Proposal to add Extension Methods to the language

I don’t see any problem here. There will be list of rules compiler will try in order, some of them will have precedence over others. E.g compiler might try to resolve o.f as an extension method application first, then this.f(o) etc. If you want to understand code you will have to learn these rules. Is your question more about extension methods in general as a feature of the language?

I don’t understand (agree with) the claim that in CL there are no methods despite the existence of the defmethod macro. Methods do in fact exist, but don’t behave like Java methods.

A method is an instance of the class standard-method and a generic function is an instance of the class standard-generic-function which is a subclass of function. Generic functions discriminate (cousin of pattern matching) based on their arguments (either via computation at compile time or at runtime) to dispatch to the correct ordered list of methods depending on the method-combination which allows protocols such that of call-next-method, before/after/around methods etc.

There is no this special variable; however, variables may be added to the implicit scope of a method simply by defining the method inside a form which introduces the binding, such as let or lambda.

(let ((a 100) (b 200))
  (defmethod foo ((n number) (i integer))
      (* -1 n i a b)))

Because methods are defined syntactically separate from classes, it seems all CL methods are in effect extension methods, if I understand the Scala extension method concept correctly.

Yes, you’re right as long as the resolution rules are well defined, then ambiguous situations don’t arise. What is my concern? It just seems strange to me that in some contexts o.f() is the same as f(o) and in other situations it is not. If making things easier for adoption is a concern/goal, then gotcha rules about when two syntaxes are equivalent fails that goal.

Perhaps that’s not a worthy goal. According to Larry Wall (creator of Perl) humans are quite happy with context dependent syntax, as it is exists in many/most human languages.

1 Like

BTW, what does this mean in an extension method?

A method in the OO sense is a subroutine that is a member of a class or object. This implies you can call it through its class or object without bringing it into scope separately, and you have dynamic dispatch for sub-types. I don’t know CL, but your description sounds like these are not methods.

The proposed extension methods for Scala are not actually methods of the class or object they are “extending”, but they are really methods of another class or object just pretending to “extend” some class. They need to be brought into scope separately.

While real methods are dynamic dispatch with regards to their owner, extension methods have an argument pretending to be the owner, and as for any argument, we have static dispatch.

Makes one wonder why we need such obfuscation? Why can’t those methods be presented as what they really are? Why do they need to create the illusion of being something else? Is a little bit of convenience worth paying such a steep price in terms of complexity and confusion?

Are you confusing an implementation of a method in a particular set of programming languages with what a method actually is? In my mind, that a method be a member of a class is only an implementation detail. The fact that if is a routine which is executing depending on the type of the arguments is independent of whether it’s a member of a class or an object.

From what I understand about C++ and Java, their object model defines methods inside classes. This definition is both syntactical (enclosed in the curly braces of the class) and semantic (as they think of a class having methods). From discussions with programmers of those languages, many people see that as part of the definition.

However, most lisp programmers (a far far far smaller group of people) don’t think that’s essential to the definition. From a CLOS perspective a method is a function which specializes none, some, or all of its arguments, on some distinguishing characteristic of the arguments such as class, identity, etc. Since no argument is essentially distinguished, it does not make sense, neither syntactically nor semantically, to think of the method as belonging to a class. This is especially true if the method specializes on identity rather than class, or if the method specializes more than one of its arguments.

Granted, in the very special case of a method which only specializes its first argument on some class, one can think that the method is defined on the class of the argument.

In some sense it is important to agree on definitions, and discussion definitions is a fun part of mathematics and computer science. But in a practical sense it is impossible for everyone to agree on definitions. We just need to be careful that we understand what the other person is saying in a particular context.

Wiktionary has an entry for “method”, which says:

Yes, as I said many people see that as the definition. Respectfully, I think that definition is misleading. It confuses a common implementation with a definition. For one reason it would mean that multi-methods are not methods (as they are not belong to a class). I wonder whether I should edit the definition in wiktionary?

No, it is not primarily about implementation. It’s primarily that

(1) you can refer to a method through its owner without bringing it into scope separately

(2) you get dynamic dispatch by owner type

Implementation is secondary. For example, in JavaScript, you can define a function separately and then add it to an object as a method.

And I would say, yes, extension methods (like proposed for Scala) and multi-methods (like in Julia) are not actual methods. If these would be methods, then what would be left that is not a method?

Also, note that regarding the proposed Scala extension methods, you would have, for example:

object A {

** def (b: B) myExtensionMethod: Unit**

}

Now myExtensionMethod is actually a method of A, but an extension method for B. Would you now say it is a method of both A and B?

Thanks for explaining your point of view. It is helpful for me to see how people understand these things, and to understand how it is different my own understanding. I’m preparing a presentation in the next couple of months regarding looking at Scala through lisp-eyes. This will discussion be useful.

It’s best to consider methods to be contracts. Each implementation of a method
def f():X = {}
Is a function bound to the equivalent of an overridable “val”.

In lisp, it’s a tad different. Defmethod and it’s associates are macros. It’s best to realized that everything is a function. Methods don’t exist. When a specialized function is invoked, a pattern match is performed to look up the implemented function to evaluate.

From a lisp perspective, Smalltalk based languages. Have an implicit parameter “this” or “self”. The specialized function (method) pattern matches based on that parameter and nothing else. The pattern matching tables are held by the class definition rather than globally. This is faster when evaluating bytecodes, but Hotspot like optimizations mostly eliminates the advantage.

In short, lispers thing everything is a function. “Multimethods” are functions associated with a global variable. That function performs a pattern match on the types of the parameters to find the concrete implementation function to evaluate.

In Smalltalk based languages (e.g. scala), the first definition of a method creates the multimethod function. If the method does have a concrete implementation (function), that is a function that will be found during the pattern matching on the implicit first parameter.

The only real difference between a Smalltalk like method and a function is the scoping of the function name and the implicit first parameter. The pattern match only matches on the first parameter.

In later generations of McCarthy Lisp, there was a function “make-specializable”. That created a global variable and bound the pattern matching function to the global variable. Subsequent “method definitions” created anonymous functions that were found by the pattern matching function. The language and implementations are different in the CLOS, Clojure and other language implementations. The semantics hasn’t really changed.

I’m replying to this old thread, instead of opening a new one.
As of dotty 0.22 the collective extension method syntax is as follows:

extension listOps on [T](xs: List[T]) {
  def second = xs.tail.head
  def third: T = xs.tail.tail.head
}

There is a very annoying and unexpected limitation where we cannot do

extension listOps on [T](xs: List[T]) {
  def foo[R](right : List[R]) : Unit = {}
}

We get the error:
extension method cannot have type parameters since some were already given previously

This is VERY unexpected. Especially if we compare with what we have been doing with implicit classes in Scala 2.

11 Likes

I found one more annoyance and inconsistency between single extension defs and collective extensions. While collective extension methods do not require explicit import from the companion object, a single extension method does.

class Foo
object Foo {
  extension on (left : Foo) {
    def + (right : Foo) : Foo = new Foo
  }
  def (left : Foo) ++ (right : Foo) : Foo = new Foo
}

val f = new Foo
val g1 = f + f
val g2 = f ++ f //error here

From @odersky’s reply on this issue I understand that is the current spec, so I thought it’s important to emphasize this difference here as well.

It is true that the single extension method could be defined at the same level of Foo, but it does not have access to the companion object’s private methods.

class Foo
def (left : Foo) ++ (right : Foo) : Foo = Foo.doSomething //error here
object Foo {
  private def doSomething : Foo = ???
  extension on (left : Foo) {
    def + (right : Foo) : Foo = doSomething
  }
}

val f = new Foo
val g1 = f + f
val g2 = f ++ f 

I should mention though there is a possible hack via export if Foo is imported via bulk:

package FooLib
class Foo
export Foo.extensions._
object Foo {
  private def doSomething : Foo = ???
  object extensions {
    def (left : Foo) ++ (right : Foo) : Foo = doSomething
  }
  extension on (left : Foo) {
    def + (right : Foo) : Foo = doSomething
  }
}
import FooLib._
val f = new Foo
val g1 = f + f
val g2 = f ++ f

The top-level extension method will not be visible by default outside of FooLib package, would it? At least I don’t see a reason for it to be; it’s not in Foo’s implicit scope.

Indeed it must always be imported explicitly:

value ++ is not a member of Outer.FooLib.Foo, but could be made available as an extension method.

One of the following imports might fix the problem:

  import Outer.FooLib.++
  import Outer.FooLib.Foo.extensions.++

I mentioned it should be a bulk import of the lib. As in import FooLib._

There was an unmentioned here change to extension method syntax:

How about then making a dot-less extension method declaration an operator declaration? Equivalent to @infix annotation.

So to define an operator you’d omit the dot:

def (a: Int) + (b: Int): Int = ...

But to define a dot-syntax method you’d use a dot:

def [A](l: List[A]).append(a: A) = ...

That way the methods are declared the same as they’re called.

4 Likes

That’s how we expect people will write the definitions, yes. But it is not enforced by the compiler.

I propose to enforce it.

Moreover, the @infix annotation could be dropped by instead treating extension methods on this.type as special:

def (self: this.type) + (that: Int): Int = ...
def (self: this.type) :: (elem: A): List[A] = ...

// instead of @infix def +(that: Int): Int
2 Likes

I am with @kai on this one. Not enforcing it will undoubtedly mean that some developers prefer to do it another way. If the goal of Dotty is to be more opinionated I don’t think it should introduce more ways to do the same thing.

Overall though, I am pretty happy with the proposed changes to extension methods.