Request for comments on exports

Some may accuse me of letting the perfect be the enemy of the good, but in this case I don’t think it’s wise to go forth with exports as an unsatisfactory delegation feature, because:

  • it will end up constraining the design of delegates if/when they are ever included
  • like I say, it is difficult to wield exports responsably
  • in general it is better to keep things simple, all else equal
  • restricting to the case of package exports already gets us to the point of parity with JS exports if I understand correctly (I’m definitely no JS expert)
2 Likes

I don’t think we need delegates as a separate/special feature from exports. Delegation is covered by exports with upcasting + a rule to ignore overriden methods.

class Copier(printUnit: PrinterImpl, scannerUnit: ScannerImpl) extends Printer with Scanner {
  export (printUnit: Printer)._
  export (scannerUnit: Scanner)._

  override def print(bits: BitMap): Unit = {
    println("printing!")
    printer.print(bits)
  }
    
  def otherCopierMethod(): Unit = {
    printUnit.otherPrinterImplMethod()
    scannerUnit.otherScannerImplMethod()
  }
}
5 Likes

This is actually a great example why we should not imitate Kotlin here! (assuming Kotlin is indeed forwarding and not full delegation, I have no first-hand knowledge of this aspect). The example looks like it should work but doesn’t.

If I run any of the print methods in your solutiion except the one that is overridden I hit ???. For instance:

Copier().print(bits, n)   // throws UnsupportedOperationExcepion

Why? Because there is no dynamic rebinding of this. The print methods in PrinterImpl will still forward to the missing first version in PrinterImpl no matter what override I install in Copier.

So this is a trap waiting to happen. In my opinion, a good solution is either very simple and restrictive (like export) or it goes all in with full delegation with rebinding of this. Everything in between promises more than it can deliver.

I don’t see full delegation coming to Scala. We have inheritance and that’s already hairy enough. So export is what’s on offer and we might push the boundaries a little bit in future versions. Or not, maybe we decide it’s too much of a minefield to go further.

2 Likes

That definitely works for me, but I can understand how this might be out of scope for this SIP, as providing a full replacement for package objects seem more urgent than a new delegation feature.

If by “full delegation” you mean including rebinding of this, then I would assume that none of us want that either. It also doesn’t seem to be the case with Kotlin.

I believe that problem #4 in @joshlemer 's example is not a problem; I would not expect the other non-overridden print methods to invoke the base print method which is overridden by the delegator. If I wanted that, I would use inheritance.

1 Like

We’re not talking about time constraints, the pace of change in dotty is extremely fast and e.g. implicit redesign is still evolving as we speak. It’s about what the development team is willing or unwilling to add or experiment with.

If we take

as a given, then we probably should stop posting and come back sometime in ‘future versions’, since the discussion appears to be over.

@odersky @eyalroth just to confirm, that’s right. I didn’t intend or expect that other overloads of print would call the overridden print, but I can see why my example made it look like that. I guess what you are calling forwarding is what I’m calling delegating. I can switch to using the term ‘forwarding’ from now on to avoid confusion.

The point I was trying to get across was just simply that exports do not handle exporting only one of many overloads, because members are exported by name. And that this results in having to manually forward all other overloads whenever there’s a single overload we don’t want to export.

I appreciate @kai’s counter-proposal, and in my view, getting this proposal to the state of supporting @kai’s solution should be the low bar for approving this SIP within the body of classes/traits/objects. I still think it might result in more confusion than the extends Foo by foo approach though. For instance take this example:

trait A {
  def a1(): Unit
  def a2(): Unit
}

trait B {
  def b1(): Unit
  def b2(): Unit
}

class C {
  def a1(): Unit = ()
}
class D {
  def a2(): Unit = ()
  def b1(): Unit = ()
}
class E {
  def b2(): Unit = ()
}

class Delegator(c: C, d: D, e: E) extends A with B {
  export c._ 
  export d._ 
  export e._
  
  // reader's thought process:
  //
  // wait, how are we implementing A and B? 
  // there's no A or B delegated to, and we aren't
  // implementing them here... hm.... 
  //
  // ..oh uhh ok so some of A is implemented by c, 
  //   some of it is implemented by d. Yet other 
  //   members of d are implementing parts of B, 
  //   and finally e implements the remainder of B.
  //   I think I need to lie down 
}

Also, following the principle of making the “right thing to do” the path of least resistence, I still think that users will tend to omit type ascriptions simply because it’s the path of least resistance. So in practice we’ll get mostly:

export printUnit._

instead of

export (printUnit: Printer)._

If it’s true that the current state of the proposal is really all that is on offer, and we aren’t open to amendments such as either releasing only for packages, or including upcasting and automatic blacklisting to avoid collisions between members, then if it were up to me I would not approve, because of the many issues that together prevent this feature from being used as a codebase grows and introduce accidental coupling.

But now I will try to not take up any more of the oxygen in this thread, and let others speak, since I’ve made many posts now, have a good day :slight_smile:

@odersky @eyalroth just to confirm, that’s right. I didn’t intend or expect that other overloads of print would call the overridden print , but I can see why my example made it look like that.

But here’s the catch: Virtually every set of overloaded methods contains cross-calls from one alternative to the next. They work fine if they are all exported uniformly. But they stop working once you add an override on the exporters side. So, I think your example was quite realistic and the problem is real.

2 Likes

There are also calls to private methods that make remapping this generally impossible or undesirable.
Whenever there’s a question of internal calling patterns, inheritance is the right tool, not exports - inheritance is specifically made to handle rewiring class internals, while exports are not - they’re a simple tool for the cases where we DON’T care about internal calling patterns and just treat objects as containers of stuff.

To this end i propose to add a more general exclusion rule
a symbol is excluded if it would conflict with an existing definition
This solves 2 issues:

  • Earlier exports shadow later exports of symbols with the same type (if exports are expanded in order of declaraton)
  • Overrides or symbols with the same type take priority over exports and are excluded from being exported

That is, the rule is not related to overrides directly, it just makes it easier to mix exports between objects with overlapping symbols.

1 Like

I propose an even more general exclusion rule:

it is an error when an exported symbol conflicts with an existing definition

There just isn’t any good way to avoid conflicting assumptions when you automatically shadow subsets of an interface, so the solution is to make it explicit: rename or omit it, explicitly, when exporting. It’s different when you’re importing–you know what you mean, presumably, so you can keep it in mind. But how is the poor consumer supposed to know what you had in mind?

Renaming/excluding might seem like an unreasonable burden, but the cognitive burden remains whether or not you have to type it out. So to avoid the high cognitive burden, demanding explicit handling is reasonable (lowers cognitive burden plus raises the author burden which makes it less likely that people will request it).

4 Likes

This is exactly the current behavior and this fail-fast behavior is what causes the problems we’ve been discussing since @joshlemer’s first message.

2 Likes

@kai - Then I guess I like the original proposal! I thought I was being even more restrictive, in that exports that conflicted with each other would also fail to work. But maybe I’m misunderstanding the resolution rules.

I would also ban exports of things that don’t have external visibility. Then you don’t run into the problem of exporting too much.

That’s in fact what’s implemented.

I would also ban exports of things that don’t have external visibility. Then you don’t run into the problem of exporting too much.

What do you mean by that? Currently it says that exports have to be accessible at the point where they are exported. So you could have a situation like this:

package p

object server:
  private[p] def service ...
 
object facade:
  export server.service

facade.service is then public. I thought it could be useful that the export can widen the accessibility of somethng it exports. That way you could tailor what gets exported without having to expose implementation details.

1 Like

Perhaps it would be better to make export retain the accessibility of the original definitions, but with the option to widen it?

package p

object inner {
  private[this] def secret = ???
  private[p] def packageSecret = ???
  def plain = ???
}

// default accessibility
object outer {
  export inner._
  // is equivalent to:
  private[p] def packageSecret = inner.packageSecret
  def plain = inner.packageSecret
}

// custom accessibility
object outer {
  public export inner._
  // is equivalent to:
  def packageSecret = inner.packageSecret
  def plain = inner.packageSecret
}

That would correspond to how accessibility rules behave with inheritance (by default, protected definitions remain protected when inherited).

The issue with exporting things to which you don’t have access is that you have to write synthetic forwarders to get them, making export not just the mirror of import. All import does is save you some typing, for non-givens; for givens it additionally brings them into consideration, but you had to be able to see them to begin with, so you could have done it some other way. The following are therefore equivalent:

// No imports
run(new foo.Foo, here.there.everywhere.Bar.default)

// Imports
import foo._
import here.there._, everywhere.Bar._
run(new Foo, default)

But export doesn’t just save typing if you can use it to share access-restricted members.

class Bar { def bippy = "bippy, of course!" }

// This is a compile error
class Foo(private[this] val bar: Bar) {}
myFoo.bar.bippy

// This is totally cool
class Foo(private[this] val bar: Bar) { export bar._ }
myFoo.bippy

Most of the difficulties pointed out by @joshlemer arise from this sort of potentially unintentional leaking of private data. You have an internal implementation which is not what you want for your public API…but the syntax makes it easiest to leak all the details with an export._, so you do it anyway.

It’s more work, but if you insist that the content be part of the public API already and export is just a convenience for usage, then this bad-design-by-laziness isn’t reinforced. You can still be lazy with your public API, but at least exporting doesn’t make you any more lazy, and you’re being up-front about it.

trait Pub { def foo: Foo }
private[pkg] trait Impl extends Pub { def bar: Bar }
class Baz private (private val impl: Impl) {
  export impl._   // This kind of thing provides all the trouble!
}
(new Baz).impl.bar   // Can't do this
(new Baz).bar        // So why can we automatically do this?

Here I’ve broken two kinds of access control at the same time–you can see the internals of a package-private trait, and you can gain access to the methods and fields of a private val.

Yes, it saves a lot of work. But it is exactly the saving of work that also causes the danger. If instead you could not share internal details, at least not without explicitly saying how to share them, then the danger would go away.

Idea one (no synthetic methods):

trait Pub { def foo: Foo }
private[pkg] trait Impl extends Pub { def bar: Bar }
class Baz private (private val impl: Impl) {
  val pub: Pub = impl
  export pub._      // This is now purely syntactic sugar
}
// (new Baz).pub.bar // Can't do this!
(new Baz).pub.foo    // Can do this, though!
(new Baz).foo        // So this is surely okay!

Idea two (explicitly request synthetic methods):

trait Pub { def foo: Foo }
private[pkg] trait Impl extends Pub { def bar: Bar }
class Baz private (private val impl: Impl) {
  export impl as Pub._  // Can bikeshed exact syntax
}
(new Baz).foo           // Seems reasonable; we asked explcitly

There is the other issue of having the synthetic forwarders implement the methods of some other trait (matching by name and type, not inheritance) which I don’t have clear thoughts about right now.

But at least in terms of confusion, I think the way to make things better is to get more restrictive, not more permissive. If more is allowed, there are more opportunities to think about when considering what the code does, and it’s harder to understand in the case where there are any problems.

Okay, I just had some thoughts about implementing traits with exports.

Here’s why I don’t like synthetic forwarders fulfilling trait requirements:

trait Room {
  def width: Double
  def height: Double
}

case class Person(name: String, height: Double) {}

class Office(val occupant: Person) extends Room {
  export occupant._
  def width = 15.0  // Feet, presumably
}

// We just implemented the room's height
// by exporting the height of the person in it...

We at least need some sort of sanity check to look at these things, don’t we?

Like perhaps:

trait Room {
  def width: Double
  def height: Double
}

case class Person(name: String, height: Double) {}

// Unimplemented method error
class Office(val occupant: Person) extends Room {
  export occupant._
  def width = 15.0
}

class TinyOffice(val occupant: Person) extends Room {
  export occupant._ for Room
  // export occupant._   // This would be a double export error
  export occupant.name   // This is still fine
  def width = 15.0
}

So again, my hunch for how to fix potential problems is to be more restrictive and more explicit.

1 Like

I don’t like this because it makes it more difficult to know if something is a safe, internal-only, breaking change or an API breaking change.

I recently learnt that anything that is private[mypackage] (technically “qualified private”, but I normally call it “package private”) is basically public if there’s any public type alias anywhere that leaks it into the public API. I was going to start making package private not trigger MiMa warnings, but this makes it harder (or maybe just impossible).

1 Like

I just cannot help but point out that this is essentially the same problem that I raise here: Request for comments on exports

And that the “extends Foo by foo” approach does not suffer from this problem. You could never end up with something confusing like this with the “Foo by foo” formulation because it doesn’t work:

class Office(val occupant: Person) extends Room by occupant {
  //                                       ^^^^^^^^^^^^^^^^
  // compile error: occupant does not implement Room 
  override def width = 15.0
}
1 Like

That’s a good example. I think it further begs the question of whether exports are the right tool for “delegation” (forwarding). I definitely see the potential of this capability being fulfilled by other features – such as the kotlin-like delegation – and as pointed out by @joshlemer, it is possible that the problems you brought up could be mitigated with a more specific syntax.

In my anecdotal experience, composition-over-inheritance is usually brought up when developers decide to use inheritance in order to gain access to functionality, and this can usually be overcome with either delegation or plain old composition (hold the class as a member instead of inheriting it).

I haven’t come across many cases where I needed to expose the methods of an inner component as is
other than in delegation (where I am implementing the same trait as the inner component). For example, the copier example from the SIP seems very strange to me; I would’ve expected traits to represent the print and scan functionalities, which would make the example eligible for delegation.

I do see see the need for composition of objects as modular components that are intended to be imported. Perhaps export should only be allowed in objects / top level (just a thought).

Edit: I’m wondering whether exports can eliminate the need for self-types which can be useful – for instance in testing frameworks – but somewhat violate the COI principle.

@joshlemer - The extends Foo by bar thing does not suffer from this problem, but it suffers from another problem–you can only extend by something that is passed in as a parameter. You can’t

class Foo(bar: Bar, baz: Baz) extends Quux {
  val quux = bar quuxifyWith baz
  export quux for Quux._
}

which seems perfectly reasonable to me.

4 Likes

This could still be worked-around.

If I wanted to implement Quux via a transformation of two other units, I would create a separate constructor / apply method for them:

class Foo(quux: Quux) extends Quux by quux {
  def this(bar: Bar, baz: Baz) = this(bar quuxifyWith baz)
}

If I wanted to implement Quux via transformation and have access to the underlying units – which IMHO is not a common forwarding pattern – I would just add them as parameters:

class Foo private(bar: Bar, baz: Baz, quux: Quux) extends Quux by quux {
  def this(bar: Bar, baz: Baz) = this(bar, baz, bar quuxifyWith baz)
}

On the other hand, the export x for X._ syntax suffers from additional verbosity when “overriding” methods of the delegatee, which I believe is used heavily in forwarding patterns (both the decorator and proxy patterns rely on it):

trait Artist {
  def name: String
  def create(): Any
}

class Painter(override val name: String) extends Artist {
  def create() = ???
}

class ConArtist(inspiration: Artist) extends Artist {
  export inspiration for Artist.{name => _, _}
  def name = s"It's a me, ${inspiration.name}!"
}
// vs
class ConArtist(inspiration: Artist) extends Artist by inspiration {
  override def name = s"It's a me, ${inspiration.name}!"
}

Also, I believe that the room-person-office example isn’t a forwarding pattern, but rather an adapter (from person to room). It might even be a form of duck-typing, which I’m not sure is desired.

2 Likes