Improve readability/learnability of extension methods

For a while now I’ve been trying to gather my thoughts on this subject so here goes.

I’d like to propose hopefully and improvement to just the new syntax for defining extension methods.
My main goal is to try improving the learnability and second readability of extension methods.

Extension methods

extend String with {
  def extMethod = this + ".ext"
}

// using the extension method
assert("abc".extMethod == "abc.ext")

This is the most basic example I could think of.
In my eyes, this code requires no explanation.

It builds on the knowledge/expectations of how class/trait inheritance works.

Such a simple example but it impacts quite substantional

  • build on what you already know (how to define a class)
  • it makes the code much more readable and learnable
  • when defining more than one extension method it gets rid of a lot of duplication/noise
  • having the extension methods grouped together will help maintainability
// current syntax
// i am sorry but my eyes can't parse this,
// and it's now obvious what to expect when you're seeing this for the first N times
// this syntax is new to everyone, current, and future scala developers
def (s: String) method1 (arg: Int) = ???
def (s: String) method2 (arg: Int) = ???
def (s: String) method3 (arg: Int) = ???

// vs

extend String with {
   def method1(arg: Int) = ???
   def method2(arg: Int) = ???
   def method2(arg: Int) = ???
}

Type-classes

The new proposed syntax would also impact/change the type-class definition, also in a positive way (at least in my eyes), by making the extension methods explicit.

Type-classes example:

trait Ord[T] {
   def compare(t1: T, t2: T): Int
   
   extend T with {
      def <(t2: T) = compare(this, t2) < 0
      def >(t2: T) = compare(this, t2) > 0
   }
}

This was just a very basic example of a type-class definition.

  • it reads quite well for java/kotlin developers.
  • it build on top of already know topics inheritance/class definition.
  • learning is easier since the extend methods are a separate subject to learn.
  • it makes the extension methods explicit and because of this, you know what behavior to expect.

Now try to look at the above example again, but this time try reading it as if you don’t know the concept of what a type-class is.

Me reading it would be something like:

Ord is just a trait (i know what a trait is).
It has just one method compare ok.
This trait also extends the type T with additional extension methods
So if I know about traits and type parameters/generics.
Wow type-classes are easy.
They are just parameterized traits with extension methods … sounds like Adhoc polymorphism

Optional naming and imports

The proposed changes should only be syntactical, so the https://dotty.epfl.ch/docs/reference/contextual/extension-methods.html#translation-of-calls-to-extension-methods would apply

Additionally, we could provide a name like

package org.example

extend String with OptionalName {
   def method1 = ???
}

Complete example:

trait Functor[F[_]] {
  def map[A,B](fa: F[A])(f: A=>B): F[B]

  extend F with {
    def map(f: A=>B): F[B] = map(this)(f)
  }
}

trait Monad[F[_]] extends Functor[F] {
  def flatMap[A,B](fa:F[A])(f: A=>F[B]): F[B]
  def map[A,B](fa: F[A])(f: A=>B): F[B] = flatMap(fa)(f `andThen` pure)
  def pure[A](a: A): F[A]

  extend F with {
   @alpha("bind")
   def >>=(f: A=>F[B]) = flatMap(this)(f)
  }
}

given listMonad: Monad[List] {
   def flatMap[A,B](xs: List[A])(f:A=>List[B]): List[B] = xs.flatMap(f)
   def pure[A](a: A): List[A] = List(a)
}
6 Likes

So far, I think we have reached one consensus regarding extension method syntax:

  • the syntax of extension methods should not obscure or complicate the semantics of this

That’s the rationale we rejected the following syntax:

def *(this: String)(n: Int): String = ...

However, currently the grouped syntax still obscures the semantics of this:

  given intOrd: Ord[Int] with
    def (x: Int) compareTo (y: Int) =
       ...
       this
       ...

  given stringOps: (xs: Seq[String])
    def longestStrings: Seq[String] = 
       ...
       this
       ...

In the code above, the two this have completely different semantics, what a surprise!

The proposal by @fkowal seems to depend on playing with the semantics of this. If we improve over the minor problem, we seem to arrive at extension clauses for objects and classes:

To improve the first proposal, and get rid of the abiguity on this we can use the standard { self =>

trait Functor[F[_]] { self => // or functor =>
   def map[A,B](fa: F[A])(f: A=>B): F[B]
  
   extend F with {
     def map[A,B](f: A=>B): F[B] = self.map(this)(f)
   }
}
1 Like

Maybe Scala could adopt a much simpler approach for extension methods.
Extension methods could be a thing syntactic sugar layer over method application.

case class A(x: Int, y: String)
...
def f(a: A)(b: B, c: C): D = ???
...
a.f(b, c)
// de-sugars to `f(a)(b, c)`, since
// * `A` doesn't have a member `f` and
// * `f: A => (B, C) => D` is in scope

It would also improve the situation, when you want to apply successively more functions. You would be able to do just this:

x.h.g.f

instead of this:

f(g(h(x))) // nested parentheses, so ugly and annoying to write

I wrote about this at more length a while back

4 Likes

I like this a lot, but I want to note in the functor example F isn’t a type that has values like this. F[A] is. I think it should be:

trait Functor[F[_]] {
  extend F[A] with {
    def map[B](fn: A => B): F[B]
  }
}

But this does call into question what the extend is? Is it a first class value like a trait? I guess in implementation it would be an anonymous value that is in implicit scope when there is an implicit Functor. The given approach has going for it the fact that it is clear what the values are and the ability to name them.

1 Like

I personally like this approach than the others. The key point is that an extension method is just syntactic sugar for function application and in fact we don’t extend anything.

Extension methods as proposed are just syntactic sugar for implicit classes.

def (string: String) ext: String = string + ".ext"

is sugar for

implicit class RichString(string: String) extends AnyVal {

def ext: String + ".ext"

}

Let’s make this as simple as possible. Let’s say “extends AnyVal” is already implied, so no need to mention it. We also don’t really care about the class name. That it is a new class is obvious since it has a pair of brackets with a def inside, so no class keyword needed (anonymous classes never needed a class keyword, and this is just another anonymous class). So this should be enough:

implicit (string: String) {

def ext: String + “.ext”
}

That’s almost as short as the main proposal and it is so clear, people would guess exactly what it does just by looking at it without further explanation.

Also, if you have more than one “extension method”, you can bundle them:

implicit (string: String) {

** def ext: String + “.ext”**

** def ext2(ext: String): String + ext**

** def blub(): Int = string.size + 42**

}

That’s even shorter than the main proposal, where you would have to write:

def (string: String) ext: String + ".ext"

def (string: String) ext2(ext: String): String + ext

def (string: String) blub(): Int = string.size + 42

3 Likes

Or maybe even better thing would be to pick not the first, but the last parameter list (it would still have to be singleton).
Then something like this would work:

class Sequence[A](...)
...
def map(f: A => B)(s: Sequence[A]): Sequence[B] = ???
def filter(p: A => Boolean)(s: Sequence[A]): Sequence[A] = ???
...
sequence.map(plus1)
// de-sugars to `map(f)(sequence)`, since
// * `Sequence` doesn't have a member `map` and
// * `map(...)(s: Sequence[A])` is in scope


sequence.map(plus1).filter(isEven)
// de-sugars to `filter(isEven)(map(f)(sequence))`

// and these same function can also be used on their own very nicely, especially with chaining
map(plus1) andThen filter(isEven)

What would you guys think? /cc @odersky

I feel that what you are proposing is the total opposite of what I was going after (learnability/readability).

  • it’s very unintuitive (show the implicit (string: String) { ... }) to a java/nonScala programmer and ask what he thinks this means.
  • implementation details / low-level meaning (implicit … extends AnyVal) leaks all the way to the top, so rather than providing a syntax that is easy to understand and conveys meaning, it hides the meaning and forces the new Scala3 users to learn about the legacy of Scala2.
  • I believe it undoes what Scala3 is trying to do, meaning uses the implicit for multiple different things.

Think of a situation where someone asks:
How do I add an extension function to an already existing type?
Answer should not be: Ok so let me tell you what implicits are.

3 Likes

If you don’t like the “implicit” keyword, because it is so Scala 2, simply replace it with whatever Scala 3 has instead. I got lost what that was. Was it “given”?

So:

given (string: String) {

** def ext: String = string + “.ext”**

** def ext2(ext: String): String = string + ext**

** def blub(): Int = string.size + 42**

}

You might call this leaky - well, I call it transparent. Because when the code you propose says “extend String”, it is really telling a lie. “Extending String” would mean defining a sub-class to String with new methods. But that is simply not what is happening. There is no sub-class of String. Those “extension methods” are members of a completely different type.

How about combining @fkowal’s syntax with @sideeffffect’s semantics / desugaring?

The syntax is just lovely. I think also you can’t be more to the point when you want define an Extension Method. An Extension Method “extends” a previously defined type, only that you “overwrite” that type with the “extended” version instead of defining a sub type. This “overwrite” is only in effect where the definition of that “extension” is in scope (so you control it—not like with Ruby’s Open Classes where such an “override” was global). Reusing the definition syntax of class types just feels most “natural” here.

(Mostly irrelevant fun fact: I arrived at more or less the same syntax thinking about that. I would, regarding syntax, go even one step further. I think you could leave out trait, class, maybe object, val, and def because the compiler can actually infer them, given some other ideas would get implemented. For sure class is not useful for anything anymore since we have Trait Parameters. I can explain the rest if someone wants to know, but that syntax change ideas are of topic here).

The issue with current Extension Methods: The current semantics under the hood, even with that new syntax, are quite complicated. “So it’s just an anonymous implicit class? What? You mean like a monoid in the category of endofunctors? Nobody understands that stuff…”

The idea that Extension Methods are in fact only a special case of function application is very interesting in my opinion. The idea is not novel, and has been implemented successfully already in a few languages. I see that as plus. “Stealing” good ideas makes them even more valuable! I think Rust is the most prominent example here, but I saw that feature on some Microsoft Research langs also (where the biggest a-ha moment was seeing something looking like JS, but then reading that it has no objects; I don’t remember how it was called). Such a desugaring of Extension Methods is easily understandable and its behavior hence predicable, even for beginners. It’s just a function application in the end, and one special rule. Compared with the overhead of an “implicit class” that’s an enormous win, imho. Especially on the axis of cognitive load when explain how it works, or thinking about the runtime details. From a more abstract viewpoint I like such a definition of Methods also because of its symmetry with functions. Methods are just functions with a self parameter! But one can hide this easily with some “extension block” syntax sugar (as proposed).

3 Likes

some Microsoft Research langs also (where the biggest a-ha moment was seeing something looking like JS, but then reading that it has no objects

you may be thinking of Koka – that’s where I draw my inspiration. I don’t like over-complicated schemes for extension methods.
The best extension methods are just plain old normal methods with appropriate syntactic sugar sequence.map(plus1).filter(isEven) ===> filter(isEven)(map(f)(sequence))

I think this would really simplify Scala

2 Likes

I really like this proposal because it makes extension methods easy to “look for” in a code base.
This might sound stupid, but that’s actually a nice feature.

Extension methods are nice but can be harmful when over-used. And as a beginner, I tend to overuse this as a brand new tool giving me a lot of power.

Making it easily searchable in any tool (even grep) helps a lot getting measurements.

One thing I’m ensure of, is the extend keyword “clashing” with extends.
If I search for extend, I’ll probably find extends too. Well, searchability was probably not the purpose of this proposal, so… Not sure it’s a real concern (also it’s called “extension” methods, so extend obviously makes sense). Possible candidates extended, enrich(ed), enhance(d), …

2 Likes

I really like at least the idea of this, and would love to see a fleshed out proposal.

Some edge-cases to consider are overload resolution (given method def foo(a: Any): Any and extension def foo(i: Int): Int which one is called when passed an int?) and that function application in scala is synthetic sugar for calling an apply method (a.foo <=> foo(a) <=> foo.apply(a) <=> apply(a)(foo) <=> apply(a).apply(foo) <=> …) which needs to be tied down somewhere to prevent infinite member search.

2 Likes