Request for comments on exports

Hi!

The SIP committee would like to solicit (more of) your feedback on exports. This proposed Scala 3 feature intends to encourage composition over inheritance. It (thus) provides a nice migration path away from package object and especially their interaction with inheritance.

Summary: syntactically, an export looks like an import. An export gives rise to an export alias that forwards to the exported member: a term is exported by a final method definition, while a type alias is used for an exported type. An export alias can implement a deferred member, but it cannot override a concrete one.

Spec + more details on motivation: http://dotty.epfl.ch/docs/reference/other-new-features/export.html

Earlier discussions:

Feature interactions always makes for good discussion (many of which have been covered in the above threads):

  • overloading
  • visibility
  • given
  • scoping (they cannot occur as statements in a block – export aliases would be defined in that block, so it’s of limited utility – @odersky could you elaborate on this exception?)
  • path-dependent types (an export alias of a stable value is considered stable, even though it’s actually backed by a method)
  • inheritance (what if an export clashes with an inherited member? This is answered in the spec, but this could probably be clarified – see question below)

Open questions (to me, as I write this summary):

  • which part of the spec ensures there are no unintended conflicts (you have to explicitly hide exports to members not eligible for exporting because they clash with a member at the site of the export). @odersky said:

    My main reservation against automatically suppressing forwards on conflicts are the possible surprises. “Why did it not install an alias for this method? Oh it’s because a method with the same name is inherited through this sequence of traits!” That’s the kind of surprises we know from inheritance that we want to avoid here.

    I see

    It is a compile-time error if a simple or renaming selector does not identify any eligible members.

    but I’d also expect a restriction on a wildcard selecting ineligible members(?)

Let’s have a productive discussion and exploration of this feature!

Thanks – adriaan

8 Likes

I can see the only useable use case for that feature. It is to reduce boilerplate of imports:

class A extends B{
   import something;
}

It will be possible to write:

class B{
  export something;
}
class A extends B{
  
}

It can be usefull.

But, this implementation is not very good for proxy classes because they need more comfortable way of overloading. It has a lack of compatibility with overloaded methods.
It does not provide real inheritance because you cannot override exported method later.
And it is rather heavyto to use it only for scope injection.
I can imagine something like

class B{
  export C
}
def doSomeThing(b:B):Unit = ....
doSomeThing(new{
})

or

doSomeThing{
  import B._
  import B.given._
}

It seems rather heavy, so I am not sure I really understand motivation.
There is an example in the documentation but I don’t have such use cases in my real practice.

There was another thread on exports: Is the exporting feature really necessary?
I’ll quote my response there:

4 Likes

I (once again) agree with @kai. I think this is probably the most underrated new feature of Dotty. IMO inheritance is often abused to get similar functionality. export will make this simpler and more ergonomic.

2 Likes

Let me address two very good questions:

Feature interactions

Possible feature interactions should be minimal since exports are completely resolved by Namer. The “export” construct does not even exist as a typed tree. Exports are entered into the symbol table as a set of forwarders. Afterwards, it is no longer relevant that these forwarders came from exports. There’s one (tiny) exception: export forwarders of givens are never cached in a separate variable, since we know that the original given was already cached if necessary. That’s the only case whether something after Namer makes a distinction whether something is an export or not.

but I’d also expect a restriction on a wildcard selecting ineligible members(?)

No, a wildcard will simply export all eligible members.

My remark applied to something else. It is best shown in the following example:

object A {
  def f: String = ""
}
trait B {
  def f: String = "abc"
}
object C extends B {
  export A._   // error: export forwarder for `f` cannot override `f` in `B`
}
object D extends B {
  object b extends B
  export b._    // OK, but nothing is exported
}
object D2 extends B {
  object b extends B
  export b.f    // error: `f` is not eligible
}

Here, the wildcard import in C will create a forwarder for f, since f is eligible by the rules. But that forwarder will then lead to an error. On the other hand, the wildcard import in D will not create a forwarder since the f in b is not eligible because the same f is already defined in a supertrait of D. If we mention f by name as in the export in D2, we get the other error: namely that we are trying to export something that is not eligible.

3 Likes

I think this is incorrect. They are not the same f in presence of side-effects in constructor.

val global = new AtomicLong(0L)

trait B {
  val id = global.incrementAndGet()
  def f: String = s"abc-$id"
}
object D extends B { // this `f` should be `abc-1`
  object b extends B // this `f` should be `abc-2`
  export b._ // the `f`'s have ~similar~ but not stable-&-equal paths, so `b.f` should be eligible and produce an error
}

The f's have ~similar~ but not stable-&-equal paths, so b.f should be eligible and produce an error, IMHO

EDIT: @LPTK’s reply makes sense, it’s not actually about them being equal definitions

I don’t think the reason for making things ineligible is that they are supposed to have the same semantics (they are not). Rather, it’s to remove friction when exporting things from the same class hierarchy as the exporter. This includes making the methods from Any always ineligible and thus not exported (otherwise we’d get errors for all wildcard exports).

4 Likes

Do forwarders give way to those original definitions when both are imported in a use-site?

For instance, if sbt contained an export of the implicit conversions in scala.sys.process._ this would compile, right?

import scala.sys.process._
import sbt._
"echo hi".!
1 Like

No, there is nothing that says that this would compile. Export forwarders are regular definitions. We don’t want to create more special cases for them.

1 Like

That’s a shame. Today users can write manual forwarders val/def/type, but the compiler can’t assume anything about that. I had hoped that by having a construct explicitly in the language for re-exporting a definition, it wouldn’t collide. :frowning_face:

3 Likes

Can export be considered as recommended way to aggregate imports?

if it is so, and there is a compilation error in:

it is a very hard restriction.

3 Likes

It is not possible to export vals from private values, because:

Export aliases for public value definitions are marked by the compiler as “stable” and their result types are the singleton types of the aliased definitions.

scala> class A{
     |   val x: Int = 1
     | }
     | class B(a: A){
     |   export a.x
     | }
5 |  export a.x
  |           ^
  |           non-private method x in class B refers to private value a
  |           in its type signature => (B.this.a.x : Int)

Could the declared type be used instead in such situations?

2 Likes

My Feedback:
Looking at this new feature I am comparing it to Kotlin’s Implement by delegation

I am 100% onboard with supporting composition over inheritance at @adriaanm mentions in his comment.

Since this a new feature and I might not understand how to use it properly.
As i understand this feature it allows to remove boilerplate.
This is nice of course but favors coding against implementation rather than an interface, because the exported methods are opaque and obey no interface/contract .

Comparing this to Kotlin implementation by delegate, we get the benefit of removing the boilerplace + the container class obeys a contract that the interface provides.

The example from http://dotty.epfl.ch/docs/reference/other-new-features/export.html would benefit from the implement by delegate

trait Printer {
  type PrinterType
  def print(bits: BitMap): Unit = ???
  def status: List[String] = ???
}
trait Scanner {
  def scan(): BitMap = ???
  def status: List[String] = ???
}

class Copier(private printer: Printer, private scanner: Scanner) extends Printer by printer, Scanner by scanner {
  override def status = printer.status ++ scanner.status
}
object Copier {
  def apply() = Copier(new Printer { type PrinterType = InkJet }, new Scanner {})
}

def somemethod(printer: Printer) = ...
somemethod(Copier())
2 Likes

It’s important to be able to export definitions without relying on inheritance (unlike Kotlin), in order to export things a the package level without using package objects (which are being dropped).

In general, I’m not convince the limitations of the Kotlin approach make much sense. When you’re exporting something from the value of a class, you’re going through the class’ public API, not its implementation. You won’t be exporting the protected and private members. This is something people already do manually today, and it’s just unnecessary boilerplate. So I don’t fully agree that it “favors coding against implementation” in a meaningful sense.

3 Likes

I think that should be possible.

1 Like

Note
Is this a bug? https://scastie.scala-lang.org/Q6b2u2YSQIy8IlSXchqy8g
Or is there some other way to stop from exporting multiple members of a delegatee? I had assumed it was export foo.{a => _, b => _, c => _, _} but that gives an error. I’m going to assume that that is the correct syntax to blacklist multiple members for the rest of this comment. If there is actually no way to blacklist multiple members of an object from export, then I think that should be considered a really serious problem with the proposal.

Anyways…

I don’t really have any opinion on how the exporting works for the package object use-cases, but I would like to point out some ways that exports compare unfavourably to Kotlin’s delegating functionalty in the usecase of creating delegating wrappers.

Exporting only the members that conform to an interface

Dotty’s export foo._ will very often in everyday programming export too many methods, and in most delegating cases, we will want to only export the members of foo which conform to an interface rather than the concrete implementation. However, the only way to do so is to couple the delegator to either the interface members or to the delegatee’s concrete members.

Example

trait MyTrait {
  def a: Int
  def b: Int
  def c: Int
}

class MyImplementation extends MyTrait {
  def a: Int = 1
  def b: Int = 2
  def c: Int = 3
  
  def d: Int = 4
  def e: Int = 5
  def f: Int = 6
}

class Delegating(underlying: MyImplementation) extends MyTrait {
  // ??? what to export here?
}

We cannot do:

export underlying._

because that will export members d, e, f. So we must do either:
A)

export underlying.{d => _, e => _, f => _, _}

or B)

export underlying.{a, b, c}

Both are bad for similar reasons.

The A) solution forces all use-sites to directly couple against the members of MyImplementation. What happens when MyImplementation gets a new member added? Delegating will quitely have new members added to it, without warning. Or what happens when a member is removed from MyImplementation? Delegating will have to be modified to no longer blacklist any dropped members. Both of these are really prickly because as a delegator we don’t want to necessarily keep track of every single one of the members exposed by an implementation.

The B) solution forces all use-sites to directly couple against members of MyTrait. For so the same thing applies. When MyTrait has members added or removed, all delegators must change their own source, even thought their intention has not changed.

Also note that A) will become unwieldy if there becomes too many members of MyImplementation, and B) will become unwieldy if there becomes too many members of MyTrait

Kotlin solves this problem in a much more elegant way IMO

interface MyInterface {
    fun a(): Unit
    fun b(): Unit
    fun c(): Unit
}

class MyImplementation: MyInterface {
    override fun a(): Unit {}
    override fun b(): Unit {}
    override fun c(): Unit {}

    fun d(): Unit {}
    fun e(): Unit {}
    fun f(): Unit {}
}

class Delegating(private val underlying: MyImplementation): MyInterface by underlying

Here the delegator can clearly specify, “I am only exporting members of MyImplementation which are implementations of MyInterface”. I don’t have to go look at the MyImplementation source code and its entire inheritance hierarchy to figure out which members it has, I don’t have to check to make sure that that potentially very large list of members doesn’t change over time. And I don’t have to do the same thing for MyInterface either.

The way I see it, this problem alone is enough to consider the proposed delegating functionality in Dotty as only working in the most simple of cases.

Overloading

The above solutions to the problem of delegating, even with their issues, don’t seem to work in the presence of overloaded members:

trait MyTrait {
  def a(i: Int): Int
}

class MyClass extends MyTrait{
  def a(i: Int): Int = 1
  def a(i: Int, j: Int): Int = 2
}

class Delegating(myClass: MyClass) extends MyTrait {
  export myClass.a
}
@main def main(): Unit = {
  println(Delegating(MyClass()).a(1,2))  // 2
}

To my knowledge, there isn’t a way in this scenario to export only the a that corresponds to MyTrait and not the one which corresponds to MyClass, without creating an other field:

class Delegating(myClass: MyClass) extends MyTrait {
  private[this] val myTrait: MyTrait = myClass
  export myTrait.a
}

This workaround gets the job done but at the cost of an additional field which will have runtime costs, and it just seems like a feature designed expressly for the purposes of encouraging delegation shouldn’t require these sorts of workarounds.

4 Likes

The multiple member suppression is definitely a bug. Do you want to open an issue?

For the delegation example, you could also express it like this, I think:

class Delegating(underlying: MyImplementation) extends MyTrait {
  val exported: MyTrait = underlying
  export exported._
}

Maybe we should consider a tweak where export and import prefixes can also have type ascriptions. Then it would be one line shorter:

class Delegating(underlying: MyImplementation) extends MyTrait {
  export (underlying: MyTrait)._
}
10 Likes

Sure, I can open a ticket, no worries :slight_smile: .

This new type ascription would be a huge improvement in my books, thanks for that :rocket: !

I also just find that the decision to only allow blacklisting members rather than allowing overrides seems unfortunate. Blacklisting delegated members in order to “override” them results in stuttering, and feels verbose and clunky in comparison.

Here is an example that I don’t really think is unreasonably large in a real-world codebase (in terms of method count, and method name length:

trait Foo {
  def method_a(): Unit
  def method_b(): Unit
  def method_c(): Unit
  def method_d(): Unit
  def method_e(): Unit
  def method_f(): Unit
  def method_g(): Unit
  def method_h(): Unit
  def method_i(): Unit
  def method_j(): Unit
  def method_k(): Unit
  def method_l(): Unit
  def method_m(): Unit
  def method_n(): Unit
  def method_o(): Unit
  def method_p(): Unit
}

trait Bar {
  def method_bar1(): Unit
  def method_bar2(): Unit
  def method_bar3(): Unit
  def method_bar4(): Unit
}
// delegate methods a ... h
class DelegatingFooBar(foo: Foo, bar: Bar) extends Foo with Bar {
  // blacklist any members of foo we want to override...
  export foo.{
              method_a => _, 
              method_b => _, 
              method_c => _,
              method_d => _,
              method_e => _,
              method_f => _,
              method_g => _,
              method_h => _,
              _}
  
  // blacklist any members of bar we wanna override...
  export bar.{
             method_bar1 => _,
             method_bar2 -> _,
             _
  }
  
  def method_i(): Unit = ()
  def method_j(): Unit = ()
  def method_k(): Unit = ()
  def method_l(): Unit = ()
  def method_m(): Unit = ()
  def method_n(): Unit = ()
  def method_o(): Unit = ()
  def method_p(): Unit = ()

  def method_bar3(): Unit = ()
  def method_bar4(): Unit = ()
}

All this blacklisting is really work that the compiler could much more cleanly be doing for us instead:

class DelegatingFooBar(foo: Foo, bar: Bar) extends Foo by foo with Bar by bar {  
  override def method_i(): Unit = ()
  override def method_j(): Unit = ()
  override def method_k(): Unit = ()
  override def method_l(): Unit = ()
  override def method_m(): Unit = ()
  override def method_n(): Unit = ()
  override def method_o(): Unit = ()
  override def method_p(): Unit = ()

  override def method_bar3(): Unit = ()
  override def method_bar4(): Unit = ()
}

In my view the second formulation is much more pleasant to read and write, yet I don’t think anything is lost.

And lastly, I also think that the extends Foo by bar is a better syntax for the more-than-bike-shedding reason that the delegatee sits right there adjacent to the interface it is supporting:

class Foo(bar: Bar, baz: Baz) extends Bar by bar with Baz by baz 
//                                       <-->            <-->
//  only 4 chars separate trait-delegate pairs!    

compare with

class Foo(bar: Bar, baz: Baz) extends Bar with Baz {
  export (bar: Bar)._ 
  export (baz: Baz)._ // <---------------------^ 
                      // potentially many lines apart, also more typing,
                      // and always necessate a body; no one-liners!
}
2 Likes

Actually I just thought of a bigger issue. In the presence of overloads, you can only blacklist all or none of the overloaded methods:

trait Foo {
  def a(i: Int): Unit
  def a(i: Double): Unit
  def a(): Unit
  def a(i: Int, j: Int): Unit // the one I want to "override"

  def b(): Unit
  def c(): Unit
}

class Delegator(foo: Foo) extends Foo {
  export foo.{a => _, _}
  
  def a(i: Int, j: Int): Unit = ()
}
// Error:
// class Delegator needs to be abstract, since def a(i: Int): Unit is not defined 

So with the current proposal, I’d have to blacklist all “a” methods, and then re-wire up the other overloads…

class Delegator(foo: Foo) extends Foo {
  export foo.{a => _, _}
  
  def a(i: Int, j: Int): Unit = () // the one I care about

  def a(i: Int): Unit = foo.a(i) // error-prone boiler plate...
  def a(i: Double): Unit = foo.a(i) // error-prone boiler plate...
  def a(): Unit = foo.a() // error-prone boiler plate...
}

With overrides it would simply be:

class Delegator(foo: Foo) extends Foo by foo {
  override def a(i: Int, j: Int): Unit = ()
}
3 Likes

I agree it is a main drawback for delegating.
The main function of delegates is an ability to provide AOP functionality( before\after advice for methods)
But this prohibition just says it is not for AOP.

IMHO it is not very good idea to use delegates only for removing prefixes, because it just decrease сohesion.