Supporting import in extension method blocks

Language Design
1

Way back in the early days of Scala, one of the notable innovations vs Java was that Scala permits imports at any point in a program’s structure, rather than just top level.

I’ve recently noticed that there is one exception however, where imports do not seem to be permitted, and where it would be convenient to use them: at the top of a block of extension methods.

A key use case for extension methods is to add behavior to an existing class we didn’t define and that the creator didn’t provide.

When the added behavior requires significant code across multiple methods, it would be nice to be able to bring the members of the extended class into scope using a single import, as occurs when we are inside its class.

A toy example, but hopefully enough to convey the need. I would like to be able to write the isMinor method as if it were defined on the Person class:

case class PrexistingPersonClass(name: String, age: Int)

extension (p: PrexistingPersonClass)
  import p.*
  def isMinor: Boolean = age < 18

As at Scala 3.4.0-RC1, this is not supported. We can place an import inside each method (ie within isMinor) but when there are many such methods, it adds too much overhead. A single import at the top of the extension block would be preferable.

It’s slightly odd & inconsistent that import can be used in most places, but not here.

14 Likes
2

This is something I had on my mind for a while. Here is another example where this could be really useful.

extension (quotes: Quotes):
  import quotes.reflect.*
  def foo(tree: Tree): Tree = ...
1 Like
3

The reason why this does not work currently is that extension does not create a new scope. extensions are desugared into defs and we do not have a clear place where we can place the import in the desugared tree.

1 Like
4

Maybe repeat the import in each def?
Moreover, this proposal nicely complements the already implemented support of export in collective extensions.

1 Like
5

Extension methods don’t create a new scope. So we don’t have a scope in which an import like that would be visible. The best you can do is have the import in the scope containing the extension methods.

Changing that would be a can of worms, for both spec and implementation.

6

I think that the request reveals a mismatch between the expectation one may have looking at an extension and the actual underlying semantics. So it might be worth to at least examine the can of worms.

When I see:

extension (self: Int) {
  def a(x: T) = ???
  def b(x: U) = ???
}

I really, really, really think about scopes. It still looks like a scope without the braces, which is why many people (myself included) are tempted to add a colon after the parameter list.

extension (self: Int):
  def a(x: T) = ???
  def b(x: U) = ???
6 Likes
7

Yes, but this is simply wrong. There is no way we can come up with a sensical definition of extension methods that implies a new scope. The need for braces is an unfortunate remnant of old style syntax. Indentation-based syntax (without the colon!) is much cleaner in that way.

To repeat the ground rules:

An extension method

extension (x: A) def (y: B) = ...

is simply a method that takes the first parameter in front of a dot. There obviously is no new scope starting after that parameter.

As a convenience we allow multiple extension methods to share the same prefix. But whether it’s a single method or multiple ones, there is still no new scope.

In light of this, the right approach is to avoid any additions (such as imports) that would imply that there is a new scope, since that would just add to the confusion. We need to double down on the fact that there is no scope. If the truth is surprising, there’s nothing to be gained in masking or hiding it.

2 Likes
8

Hmm – that does lead to the reverse question: is there any way we could clarify the difference?

I agree with @alvae’s intuition: this feels syntactically like it should be a scope (with braces or without), and while I intellectually understand the reason why it can’t be, that’s still a surprise. I suspect that this is going to be an ongoing FAQ and small pain point for folks. So I wonder if there’s anything that could be done to make this feel less like a scope?

2 Likes
9

It seems to me that this convenience syntax is the cause of the confusion.

I could see an argument for supporting import here as another “convenience”, maybe implemented with the semantics described above.

But this seems like a slippery slope. So coming back to this …

I say yes, we can deprecate the convenience syntax and require the extension prefix before every such method. It also doubles down on the idea that extension methods are desugared into curried defs (and can indeed be invoked like that).

2 Likes
10

Repeat the import in each def would be a leaky abstraction, since then you can’t import a type in a parameter list.

object O:
  type M
extension (x: A) 
  import O.*
  def f(x: M) = ... // does not work
1 Like
11

It seems like something like this would match the expectation created by the creation of the block (with delimiters like braces/colon or without, it still looks like a block):

Source:

case class PrexistingPersonClass(
  name: PrexistingPersonClass.Name, 
  age: Int
)
object PrexistingPersonClass:
  type Name = String

extension (p: PrexistingPersonClass)
  import p.*
  import PrexistingPersonClass.*

  def isNamed(x: Name): Boolean = name.equalsIgnoreCase(x)

Reality:

case class PrexistingPersonClass(
  name: PrexistingPersonClass.Name, 
  age: Int
)

object PrexistingPersonClassExtensions$1: 
    import PrexistingPersonClass.*

    extension (p: PrexistingPersonClass) def isNamed(x: Name): Boolean = { 
      import p.* 
      name.equalsIgnoreCase(x)
    }

export PrexistingPersonClassExtensions$1.*
1 Like
12

Maybe the solution is education through better error reporting.

For example, the compiler could detect the presence of an import after the prefix and point the user towards moving it into the def.

2 Likes
13

A previous proposal was

extension (import x: X) def f = x.y
14

Theoretically we could allow some other (than { … } or <indent> … <unindent>) syntax for non-scope-creating-antiboilerplate-convenience. If we do, we should generalize it to other “directives” (not only to extension). For example:

private (
  def …
  def …
)
extension (x: A) (
  def …
  def …
)

or

private [[
  def …
  def …
]]
extension (x: A) [[
  def …
  def …
]]

But it’s probably not worth it.

15

I appreciate the arguments given above for why we can’t readily support import at the top of an extension “block” as I’d initially suggested.

However, I do think there may be a case for an import syntax specifically targeting the members of the extension parameter, eg:

extension (import x: X) 
  def f
  def g

or alternately:

extension import (x: X) 
  def f
  def g

The effect would be for compiler to insert import x.{*, given} at the top each extension method.

I first came to Scala, and still prefer it today, because the language offers industry-leading capabilities for factoring out shared structure in programs. The ability to extract common structure in code is what underlies Scala’s conciseness, productivity, and “convenience”, in my view.

Here we have an emerging example of common structure in a relatively young feature, extension methods. In object-oriented languages, references to an instance’s this pointer are considered so common that they are typically resolved implicitly. Extension methods associated with a parameter likely need to make similarly frequent reference to members of that parameter.

“Good” code typically has many small methods, and so the overhead of an import statement at top of each method is significant; it was personally working on an extension with 10 methods, each 1-2 lines long, that first drove me to post this topic.

In terms of precedents, Kotlin’s extension methods implicitly import the members of the extended instance:

class Circle (val radius: Double) {
}
fun main() {
    fun Circle.perimeter(): Double {
        return 2*Math.PI*radius;
    }
}

While in Swift and C# the extended parameter must be explicitly referenced.

2 Likes
16

It’s only a partial solution to Nicolas Stucki’s sticking point:

as I forgot to add.

Note that the objection,

is not germane in so far as an import is an import context. It might be hairy to explain what a collective thing is and why an import context closes when it ends.

To collect is to glean, as at harvest, so a collective is for the purpose of lesen, or reading, a convenience for reading. When the harvest is over, it’s natural to close what was opened. Another agrarian pun for collective is колхоз. Imports are for the collective, which does not produce all the grains required for sustenance, and what is produced by the collective is for distribution.

17

Am I the only one who loves the current syntax? It’s really, really awesome. Maybe I’m just a naive idiot.

As long as people learn the fact “extension does not create a new scope” I think it’s really nice.

I always think of it from a teaching perspective: I’d tell students “look there is no colon here, so no scope”. But then again, I enjoy life on the sunny side, in the strictly-Scala3-only world. So I don’t have to worry about braces. (I do love Curly Brace, she’s a killer robot from the future.)

Otherwise I’d be OK with Arman’s suggestion of requiring extension keyword in front of every method. That’s also good.

I think extension should not create a new scope, regardless of whether it could or could not. That will lead to a lot more confusion. Scopes are more confusing and mentally tasking than other things in general. Students would ask: “where should I import? Up above, or inside here, or somewhere else?” There is already some confusion about whether extensions should be at top-level, or inside a class or (companion) object, etc.

You know people are gonna do some crazy stuff with that kind of mechanism. It then starts going back towards the implicit class stuff of Scala 2. As I understand, extension was created to limit that power and clarify / emphasize the intent / use case over the mechanism.

We should go in the opposite direction. I support Arman’s suggestion (but would love to keep the current totally awesome syntax if I can).

3 Likes
18

Thanks, I love how Curly Brace and Quote can wield weapons “without any additional training.”

If we could figure that out for Scala, it would reduce the burden on documentation.

As a first step, the docs could indicate which features can be deployed “without special training,” and also which features that, once deployed, ensure the app can “withstand incredible amounts of damage.”

Personally, I’m always asking “where should I import?” If a local import gets pushed to the top of the file, it means not only that this file has a dependency, but the whole file has a dependency.

I hope the people learn that extension does not create a scope, but import creates a context. There are other situations where you go, oh, import creates a context? so the knowledge generalizes.

The people deserve knowledge because knowledge is power!

1 Like
19

But that would cause confusion as well. There are many places where there is no colon, but there is a new scope. In fact, every place where there are braces or indentation except for an extension block.

2 Likes
20

Ah you’re right. I guess it’s impossible to find a consistent “rule” to teach these things, everything has to be explained in a “context-sensitive” manner. (At least I don’t have to deal with braces.)