Request for comments on exports

Yes, the scheme makes it sometimes awkward to express delegation. That’s intentional: the motivation of the proposal is to have something simpler than inheritance. If we supported full delegation, we would add another mechanism that’s roughly as powerful as inheritance, and fraught with similar problems.

That said, it might be possible to make the scheme more powerful/flexible. For instance, we could have earlier exports disable later ones of the same names. So, in

object m1 { def a: T; def b: T }
object m2 { def a: T; def c: T }
export m1._
export m2._

you’d get a, and b from m1 and c from m2, rather than a double definition error on a, as is the case today. Additionally, we could allow override definitions in a scope with exports to take precedence over any exported values.

But it’s not clear to me yet we want to add such tweaks. Maybe it’s better to keep things simple and restrictive and to avoid the feature interactions that such extensions would entail? Fortunately, we do not have to decide now. I would advise to go with the simple scheme for now, and keep possible tweaks on the table for discussion to be included in later Scala versions.

There is a good reason to keep the base proposal as currently implemented for Scala 3.0 since it is needed to replace package object inheritance, which we will deprecate. But any refinements to be more delegation-like are for power users only, so by our criteria they should come later.

5 Likes

I agree it’s best to keep it simple. Perhaps we should not call this “delegation”, as this is not rebound when we invoke an exported method (see https://en.wikipedia.org/wiki/Delegation_(object-oriented_programming); google books has a “preview” of Kniesel’s paper referenced in that article)).

2 Likes

Delegation is a very broad term.


There is an example:

class Window(val bounds: Rectangle) {
    // Delegation
    fun area() = bounds.area()
}
1 Like

Okay, sure that’s fine.

If we aren’t ready to commit to making exports work nicely for delegation, maybe we can release a limited form of exports for 3.0 specifically for the package-level usecase? Export statements could be limited to only top-level statements, and not be supported in the body of a class/trait/object.

That way we can keep our options open for when we choose to support delegation properly, rather than committing to this very imperfect system. If we decide later that exports are exactly what we want for delegation, we can always add it in 3.1.

3 Likes
1 Like

If you look at the motivation of https://dotty.epfl.ch/docs/reference/other-new-features/export.html, you will find the delegation is not listed as a goal, and I am still not convinced it should be a goal. On the other hand, restricting the proposal to package objects only would not address the primary goal of the proposal: having a simpler alternative to inheritance. So I am against such a restriction. I think we should evaluate the proposal as a solution for the stated goals and keep the option open to extend the goals to cover delegation-like behavior later, but not for 3.0.

3 Likes

Users will expect delegation, users will use it for delegation, the very words

imply delegation.
And the very first thing people think about when looking at it is delegation. Maybe the users aren’t wrong?
This feels like an arbitrary line to stop designing a comprehensive feature at and I don’t know the reason for it, unless the feature is considered experimental and scheduled for removal.

EDIT: my tone was overly combative in this message because I misread the clause its owner is not a base class of the class(*) containing the export clause, as meaning that even basic delegation such as trait X { def a: Unit }; class DX(x: X) extends X { export x._ } does not work since x.a is from a base class X of DX – but the owner referred to the owner of the latest implementation, not the earliest definition.
That means most basic forms of delegation are expressible with exports.

I guess the only tweak necessary for exports to be really good for delegation is to add a clause to eligible methods condition:
it is not overriden in the class containing the export clause

Which I do think is better done earlier than later – otherwise export will be yet another new feature that only does the basics and forces a retreat to low-level syntax for advanced usecases – i.e. doesn’t scale with the codebase.

2 Likes

This is interesting, I was not aware of that. In fact, when I proposed a delegation feature in the past, I was actually thinking of forwarding. I wonder what people actually want to have, automatic forwarding or automatic delegation?

1 Like

Are there any popular languages that implement delegation as described in the article? It seems to me that the author chooses an overly narrow definition not used in practice.

Remapping this would be highly non-trivial to implement on the JVM, I would safely assume that all references to delegation here mean Kotlin’s ‘implementation by delegation’

1 Like

Javascript.

Personally I would love this feature, which is similar to my previous proposal. However, I believe that this is not the motivation behind exports, which IIUC doesn’t aim to endorse “composition over inheritance” in general, but rather only for a few selected use cases (such as package objects).

I’ve been exploring similar mechanics in my alternative proposal for implicits, but it seem to me that they might very well be relevant for name conflicts as well. In any case, I agree that these are much more advanced mechanics that can be postponed.

No, it does aim to endorse composition over inheritance in general. Inasmuch as it’s written in the motivation above:

It is a standard recommendation to prefer composition over inheritance. This is really an application of the principle of least power: Composition treats components as blackboxes whereas inheritance can affect the internal workings of components through overriding. Sometimes the close coupling implied by inheritance is the best solution for a problem, but where this is not necessary the looser coupling of composition is better.

Moreover I don’t think anything remapping this is simpler than inheritance, going that way would be out of scope for exports and not intuitively related in any way either.

1 Like

If we can, I’d like to avoid getting into a discussion about what defines delegation precisely. The usecase I’m trying to evaluate exports against is the example given in the proposal, the one about printers and scanners. My thought is that the example given is a bit too simplistic to be really indicative of how the feature would be used in the real world, since it only is concerning 3 tiny concrete classes interacting with each other. In the real world, we are most often producing and consuming interfaces rather than grab bags of individual members, yet also need to be able to avoid delegating specific members. So a more realistic version of the example might look like this:

class BitMap
class InkJet

trait Printer {
  type PrinterType
  def print(bits: BitMap): Unit
  def print(bits: BitMap, n: Int): Unit 
  def print(bits: BitMap, colour: String): Unit
  
  def printDiagnostics(): Unit
  def tunOnPrinter(): Unit
  def turnOffPrinter(): Unit
  def status: List[String] 
}

class PrinterImpl extends Printer {
  def print(bits: BitMap): Unit = ???
  def print(bits: BitMap, n: Int): Unit = (0 until n).foreach(_ => print(bits))
  def print(bits: BitMap, colour: String): Unit = ???
  def printDiagnostics(): Unit = ???
  def tunOnPrinter(): Unit = ???
  def turnOffPrinter(): Unit = ???
  def status: List[String] = ???
  def otherPrinterImplMethod(): Unit = ()
}

trait Scanner {
  def scan(): BitMap
  def scan(n: Int): Seq[BitMap] 
  def scanColour(): BitMap
  def scanColour(n: Int): Seq[BitMap] = (0 until n).map(_ => scanColour())

  def status: List[String]
}

class ScannerImpl extends Scanner {
  def scan(): BitMap = ???
  def scan(n: Int): Seq[BitMap] = (0 until n).map(_ => scan())
  def scanColour(): BitMap = ???
  def status: List[String] = ???
  def otherScannerImplMethod(): Unit = ()
}

class Copier(private val printUnit: PrinterImpl, private val scannerUnit: ScannerImpl) extends Printer with Scanner {

  // problem 1: The fact that we use a PrinterImpl and ScannerImpl is an implementation detail. We don't want to 
  // accidentally leak members when exporting, so we must explicitly up-cast into the interface we want to 
  // export. This will also have runtime consequences as Copier must now hold 2 extra references at runtime!
  private[this] val printer: Printer = printUnit
  private[this] val scanner: Scanner = scannerUnit
  
  // problem 2: Every new method that printer gets, I must modify my code here, why do I have to type all 
  // this out? This will result in a lot of code churn as every time any of my delegatees are changed, 
  // I must change too
  
  // I could instead blacklist, but then I have the same problem in reverse, having to call out twice
  // which methods I want to avoid delegating
  
  // Problem 3: if someone is reading this code later, is it immediately obvious why I'm exporting these methods?
  // or why I'm exporting the scanner method? I must manually guess and check which traits are covered by
  // which exports
  export printer.{printDiagnostics, tunOnPrinter, turnOffPrinter}
  export scanner.{status => _, _}
  
  // problem 4: can only export/blacklist by name, so if there are overloads, exports don't work!
  // here, we only care to "override" the first print overload, but that forces us to wire up all
  // other overloads. 
  def print(bits: BitMap): Unit = {
    println("printing!")
    printer.print(bits)
  }
  def print(bits: BitMap, n: Int): Unit = printer.print(bits, n)
  def print(bits: BitMap, colour: String): Unit = printer.print(bits, colour)
  
  def otherCopierMethod(): Unit = {
    printUnit.otherPrinterImplMethod()
    scannerUnit.otherScannerImplMethod()
  }

  def status: List[String] = printer.status ++ scanner.status
}

I have listed ~4 problems that come up when you try to increase complexity from tiny to medium-small. These add up to a lot of complexity and mental burden when trying to use exports in a real world setting.

My prediction is that, since it is a headache to expose a conservative interface people will end up not paying much attention to it and just throwing out export foo._ everywhere. This would lead to unnecessarily tight coupling. In the example above, we would end up exporting otherPrinterImplMethod and otherScannerImplMethod simply because that’s easier to do / requires less thinking than upcasting.

If instead we take inspiration from Kotlin’s delegates, doing “the right thing” is simple and easy:

class Copier(private val printUnit: PrinterImpl, private val scannerUnit: ScannerImpl) 
  extends Printer by printUnit with Scanner by scannerUnit {
  
  override def print(bits: BitMap): Unit = {
    println("printing!")
    printer.print(bits)
  }
    
  def otherCopierMethod(): Unit = {
    printUnit.otherPrinterImplMethod()
    scannerUnit.otherScannerImplMethod()
  }
}
2 Likes

Some may accuse me of letting the perfect be the enemy of the good, but in this case I don’t think it’s wise to go forth with exports as an unsatisfactory delegation feature, because:

  • it will end up constraining the design of delegates if/when they are ever included
  • like I say, it is difficult to wield exports responsably
  • in general it is better to keep things simple, all else equal
  • restricting to the case of package exports already gets us to the point of parity with JS exports if I understand correctly (I’m definitely no JS expert)
2 Likes

I don’t think we need delegates as a separate/special feature from exports. Delegation is covered by exports with upcasting + a rule to ignore overriden methods.

class Copier(printUnit: PrinterImpl, scannerUnit: ScannerImpl) extends Printer with Scanner {
  export (printUnit: Printer)._
  export (scannerUnit: Scanner)._

  override def print(bits: BitMap): Unit = {
    println("printing!")
    printer.print(bits)
  }
    
  def otherCopierMethod(): Unit = {
    printUnit.otherPrinterImplMethod()
    scannerUnit.otherScannerImplMethod()
  }
}
5 Likes

This is actually a great example why we should not imitate Kotlin here! (assuming Kotlin is indeed forwarding and not full delegation, I have no first-hand knowledge of this aspect). The example looks like it should work but doesn’t.

If I run any of the print methods in your solutiion except the one that is overridden I hit ???. For instance:

Copier().print(bits, n)   // throws UnsupportedOperationExcepion

Why? Because there is no dynamic rebinding of this. The print methods in PrinterImpl will still forward to the missing first version in PrinterImpl no matter what override I install in Copier.

So this is a trap waiting to happen. In my opinion, a good solution is either very simple and restrictive (like export) or it goes all in with full delegation with rebinding of this. Everything in between promises more than it can deliver.

I don’t see full delegation coming to Scala. We have inheritance and that’s already hairy enough. So export is what’s on offer and we might push the boundaries a little bit in future versions. Or not, maybe we decide it’s too much of a minefield to go further.

2 Likes

That definitely works for me, but I can understand how this might be out of scope for this SIP, as providing a full replacement for package objects seem more urgent than a new delegation feature.

If by “full delegation” you mean including rebinding of this, then I would assume that none of us want that either. It also doesn’t seem to be the case with Kotlin.

I believe that problem #4 in @joshlemer 's example is not a problem; I would not expect the other non-overridden print methods to invoke the base print method which is overridden by the delegator. If I wanted that, I would use inheritance.

1 Like

We’re not talking about time constraints, the pace of change in dotty is extremely fast and e.g. implicit redesign is still evolving as we speak. It’s about what the development team is willing or unwilling to add or experiment with.

If we take

as a given, then we probably should stop posting and come back sometime in ‘future versions’, since the discussion appears to be over.

@odersky @eyalroth just to confirm, that’s right. I didn’t intend or expect that other overloads of print would call the overridden print, but I can see why my example made it look like that. I guess what you are calling forwarding is what I’m calling delegating. I can switch to using the term ‘forwarding’ from now on to avoid confusion.

The point I was trying to get across was just simply that exports do not handle exporting only one of many overloads, because members are exported by name. And that this results in having to manually forward all other overloads whenever there’s a single overload we don’t want to export.

I appreciate @kai’s counter-proposal, and in my view, getting this proposal to the state of supporting @kai’s solution should be the low bar for approving this SIP within the body of classes/traits/objects. I still think it might result in more confusion than the extends Foo by foo approach though. For instance take this example:

trait A {
  def a1(): Unit
  def a2(): Unit
}

trait B {
  def b1(): Unit
  def b2(): Unit
}

class C {
  def a1(): Unit = ()
}
class D {
  def a2(): Unit = ()
  def b1(): Unit = ()
}
class E {
  def b2(): Unit = ()
}

class Delegator(c: C, d: D, e: E) extends A with B {
  export c._ 
  export d._ 
  export e._
  
  // reader's thought process:
  //
  // wait, how are we implementing A and B? 
  // there's no A or B delegated to, and we aren't
  // implementing them here... hm.... 
  //
  // ..oh uhh ok so some of A is implemented by c, 
  //   some of it is implemented by d. Yet other 
  //   members of d are implementing parts of B, 
  //   and finally e implements the remainder of B.
  //   I think I need to lie down 
}

Also, following the principle of making the “right thing to do” the path of least resistence, I still think that users will tend to omit type ascriptions simply because it’s the path of least resistance. So in practice we’ll get mostly:

export printUnit._

instead of

export (printUnit: Printer)._

If it’s true that the current state of the proposal is really all that is on offer, and we aren’t open to amendments such as either releasing only for packages, or including upcasting and automatic blacklisting to avoid collisions between members, then if it were up to me I would not approve, because of the many issues that together prevent this feature from being used as a codebase grows and introduce accidental coupling.

But now I will try to not take up any more of the oxygen in this thread, and let others speak, since I’ve made many posts now, have a good day :slight_smile:

@odersky @eyalroth just to confirm, that’s right. I didn’t intend or expect that other overloads of print would call the overridden print , but I can see why my example made it look like that.

But here’s the catch: Virtually every set of overloaded methods contains cross-calls from one alternative to the next. They work fine if they are all exported uniformly. But they stop working once you add an override on the exporters side. So, I think your example was quite realistic and the problem is real.

2 Likes