Request for comments on exports

Perhaps it would be better to make export retain the accessibility of the original definitions, but with the option to widen it?

package p

object inner {
  private[this] def secret = ???
  private[p] def packageSecret = ???
  def plain = ???
}

// default accessibility
object outer {
  export inner._
  // is equivalent to:
  private[p] def packageSecret = inner.packageSecret
  def plain = inner.packageSecret
}

// custom accessibility
object outer {
  public export inner._
  // is equivalent to:
  def packageSecret = inner.packageSecret
  def plain = inner.packageSecret
}

That would correspond to how accessibility rules behave with inheritance (by default, protected definitions remain protected when inherited).

The issue with exporting things to which you don’t have access is that you have to write synthetic forwarders to get them, making export not just the mirror of import. All import does is save you some typing, for non-givens; for givens it additionally brings them into consideration, but you had to be able to see them to begin with, so you could have done it some other way. The following are therefore equivalent:

// No imports
run(new foo.Foo, here.there.everywhere.Bar.default)

// Imports
import foo._
import here.there._, everywhere.Bar._
run(new Foo, default)

But export doesn’t just save typing if you can use it to share access-restricted members.

class Bar { def bippy = "bippy, of course!" }

// This is a compile error
class Foo(private[this] val bar: Bar) {}
myFoo.bar.bippy

// This is totally cool
class Foo(private[this] val bar: Bar) { export bar._ }
myFoo.bippy

Most of the difficulties pointed out by @joshlemer arise from this sort of potentially unintentional leaking of private data. You have an internal implementation which is not what you want for your public API…but the syntax makes it easiest to leak all the details with an export._, so you do it anyway.

It’s more work, but if you insist that the content be part of the public API already and export is just a convenience for usage, then this bad-design-by-laziness isn’t reinforced. You can still be lazy with your public API, but at least exporting doesn’t make you any more lazy, and you’re being up-front about it.

trait Pub { def foo: Foo }
private[pkg] trait Impl extends Pub { def bar: Bar }
class Baz private (private val impl: Impl) {
  export impl._   // This kind of thing provides all the trouble!
}
(new Baz).impl.bar   // Can't do this
(new Baz).bar        // So why can we automatically do this?

Here I’ve broken two kinds of access control at the same time–you can see the internals of a package-private trait, and you can gain access to the methods and fields of a private val.

Yes, it saves a lot of work. But it is exactly the saving of work that also causes the danger. If instead you could not share internal details, at least not without explicitly saying how to share them, then the danger would go away.

Idea one (no synthetic methods):

trait Pub { def foo: Foo }
private[pkg] trait Impl extends Pub { def bar: Bar }
class Baz private (private val impl: Impl) {
  val pub: Pub = impl
  export pub._      // This is now purely syntactic sugar
}
// (new Baz).pub.bar // Can't do this!
(new Baz).pub.foo    // Can do this, though!
(new Baz).foo        // So this is surely okay!

Idea two (explicitly request synthetic methods):

trait Pub { def foo: Foo }
private[pkg] trait Impl extends Pub { def bar: Bar }
class Baz private (private val impl: Impl) {
  export impl as Pub._  // Can bikeshed exact syntax
}
(new Baz).foo           // Seems reasonable; we asked explcitly

There is the other issue of having the synthetic forwarders implement the methods of some other trait (matching by name and type, not inheritance) which I don’t have clear thoughts about right now.

But at least in terms of confusion, I think the way to make things better is to get more restrictive, not more permissive. If more is allowed, there are more opportunities to think about when considering what the code does, and it’s harder to understand in the case where there are any problems.

Okay, I just had some thoughts about implementing traits with exports.

Here’s why I don’t like synthetic forwarders fulfilling trait requirements:

trait Room {
  def width: Double
  def height: Double
}

case class Person(name: String, height: Double) {}

class Office(val occupant: Person) extends Room {
  export occupant._
  def width = 15.0  // Feet, presumably
}

// We just implemented the room's height
// by exporting the height of the person in it...

We at least need some sort of sanity check to look at these things, don’t we?

Like perhaps:

trait Room {
  def width: Double
  def height: Double
}

case class Person(name: String, height: Double) {}

// Unimplemented method error
class Office(val occupant: Person) extends Room {
  export occupant._
  def width = 15.0
}

class TinyOffice(val occupant: Person) extends Room {
  export occupant._ for Room
  // export occupant._   // This would be a double export error
  export occupant.name   // This is still fine
  def width = 15.0
}

So again, my hunch for how to fix potential problems is to be more restrictive and more explicit.

1 Like

I don’t like this because it makes it more difficult to know if something is a safe, internal-only, breaking change or an API breaking change.

I recently learnt that anything that is private[mypackage] (technically “qualified private”, but I normally call it “package private”) is basically public if there’s any public type alias anywhere that leaks it into the public API. I was going to start making package private not trigger MiMa warnings, but this makes it harder (or maybe just impossible).

1 Like

I just cannot help but point out that this is essentially the same problem that I raise here: Request for comments on exports

And that the “extends Foo by foo” approach does not suffer from this problem. You could never end up with something confusing like this with the “Foo by foo” formulation because it doesn’t work:

class Office(val occupant: Person) extends Room by occupant {
  //                                       ^^^^^^^^^^^^^^^^
  // compile error: occupant does not implement Room 
  override def width = 15.0
}
1 Like

That’s a good example. I think it further begs the question of whether exports are the right tool for “delegation” (forwarding). I definitely see the potential of this capability being fulfilled by other features – such as the kotlin-like delegation – and as pointed out by @joshlemer, it is possible that the problems you brought up could be mitigated with a more specific syntax.

In my anecdotal experience, composition-over-inheritance is usually brought up when developers decide to use inheritance in order to gain access to functionality, and this can usually be overcome with either delegation or plain old composition (hold the class as a member instead of inheriting it).

I haven’t come across many cases where I needed to expose the methods of an inner component as is
other than in delegation (where I am implementing the same trait as the inner component). For example, the copier example from the SIP seems very strange to me; I would’ve expected traits to represent the print and scan functionalities, which would make the example eligible for delegation.

I do see see the need for composition of objects as modular components that are intended to be imported. Perhaps export should only be allowed in objects / top level (just a thought).

Edit: I’m wondering whether exports can eliminate the need for self-types which can be useful – for instance in testing frameworks – but somewhat violate the COI principle.

@joshlemer - The extends Foo by bar thing does not suffer from this problem, but it suffers from another problem–you can only extend by something that is passed in as a parameter. You can’t

class Foo(bar: Bar, baz: Baz) extends Quux {
  val quux = bar quuxifyWith baz
  export quux for Quux._
}

which seems perfectly reasonable to me.

4 Likes

This could still be worked-around.

If I wanted to implement Quux via a transformation of two other units, I would create a separate constructor / apply method for them:

class Foo(quux: Quux) extends Quux by quux {
  def this(bar: Bar, baz: Baz) = this(bar quuxifyWith baz)
}

If I wanted to implement Quux via transformation and have access to the underlying units – which IMHO is not a common forwarding pattern – I would just add them as parameters:

class Foo private(bar: Bar, baz: Baz, quux: Quux) extends Quux by quux {
  def this(bar: Bar, baz: Baz) = this(bar, baz, bar quuxifyWith baz)
}

On the other hand, the export x for X._ syntax suffers from additional verbosity when “overriding” methods of the delegatee, which I believe is used heavily in forwarding patterns (both the decorator and proxy patterns rely on it):

trait Artist {
  def name: String
  def create(): Any
}

class Painter(override val name: String) extends Artist {
  def create() = ???
}

class ConArtist(inspiration: Artist) extends Artist {
  export inspiration for Artist.{name => _, _}
  def name = s"It's a me, ${inspiration.name}!"
}
// vs
class ConArtist(inspiration: Artist) extends Artist by inspiration {
  override def name = s"It's a me, ${inspiration.name}!"
}

Also, I believe that the room-person-office example isn’t a forwarding pattern, but rather an adapter (from person to room). It might even be a form of duck-typing, which I’m not sure is desired.

2 Likes

Those are good points. The constructor thing doesn’t work as nicely for case classes etc., but it’s a decent workaround.

I’m not sure what I think about the override case. I hadn’t considered it very thoroughly. If it’s a common use case, I think you could allow in-block overrides over a wildcard export. The export ident for Type form is still more verbose. I think it also gives more precise control.

Right now, traits are filled by type, even if the inheritance hierarchy isn’t followed. For instance,

class Fish { def fish = "salmon" }
trait FishBird{ def fish: String; def bird: String }
class PredatorPrey extends Fish with FishBird {
  def bird = "osprey"
}

is fine because the name and type of fish: String matches, even though FishBird has no inheritance relationship with Fish.

With the Type by instance mode, either you lack the control to exclude things that you don’t mean to include, or you have to restrict instance to being part of the inheritance hierarchy of Type.


Then there’s the whole issue of what happens if you do innocuous forwarding that doesn’t seem to need a synthetic method:

class Foo {
  // No point in creating synthetic forwarder
  export Foo._
}
object Foo {
  def whatever = "salmon"
}

trait Whatever {
  def whatever: String
}

// Oops, this only works if we did create a synthetic forwarder
class Iffy extends Foo with Whatever {}

This suggests that if you allow exports to fulfill trait contracts, you must always create synthetic forwarders if there’s any possibility of extension; export isn’t just a compiler fiction. Or you have to transitively keep track of these exports and write synthetic methods the subclasses that rely upon the exports being available.

The problem with not allowing exports to fulfill traits in the Iffy case is that you add an extra bit of information for every method–did you get this method from inheritance or from exporting? This is a huge headache. If the feature is going to be used heavily enough to be worth including, you’ll have to keep track of this. (This is one of the annoyances of extension methods, though admittedly you could just say that exports are syntactic sugar for extension methods.)

So, alternatively, we either have to restrict exports to things that can’t be extended, or everything in an extensible context has to be implemented as a forwarder.

If the latter, then I would argue that greater control is a virtue.

trait Base { def a: Int; b: Int; c: Int }
trait One extends Base { def a = 1 }
trait Two extends Base { def b = 2 }
trait C { def c = 3 }
class Count(one: One, two: Two) extends Base with C {
  export one for Base.a
  export two for Base.b
}

I don’t think the by form has this kind of flexibility.

But note that given the synthetic forwarder thing,

class Count(one: One, two: Two) extends Base with C {
  export one.a
  export two.b
}

would have to be a compile-time error precisely because of extends Base (otherwise it would be fine), and you could get around it with

abstract class AbstractCount(one: One, two: Two) extends C {
  export one.a
  export one.b
}
class Count(one: One, two: Two) extends AbstractCount(one, two) with Base {}

and this would have to work. You can’t disallow this, I don’t think, without making people memorize whether each method is an export or not (ugh!). But you don’t want to make them write it this way, I don’t think. You want it the single-class way, and you can’t handle that with by notation. But you can handle it with export for notation.


Anyway, I am rather leery of this whole thing. It seems like there are a lot of potential feature interactions that could get ugly.

If export is syntactic sugar for creating extension methods, then I think everything is cool, or at least as cool as it’s going to be anyway. If it’s any more, then I think there are dragons, and I’m not totally convinced that they’re tamed yet.

1 Like

I would love to further discuss the points you brought up in this comment, but I feel it might be a digression at this point, especially since I mostly agree with this bottom line.

Forwarding patterns, especially decorator and proxy, could benefit a lot from a syntactic sugar to alleviate their boilerplate, but my gut tells me that export is not the right tool for this task, and so this leaves it out of the scope of this SIP.

I don’t see exports as the one-size-fits-all solution for composition over inheritance. As I’ve mentioned earlier, there are also self-types which encourage inheritance over composition, and I don’t see how exports change that.

I do see export as a means of composing stateless bundles of functionalities. As already demonstrated in the SIP, they can be used to merge multiple objects so users could import libraries’ content at ease, and they also provide an alternative to the dropped package objects. That’s why I brought up the suggestion to allow them only inside objects or top-level definitions.

I also wish that exports – or another complementing construct — would allow to resolve all matter of conflicts when merging functionalities together. Right now they allow only for resolution of naming conflicts, but I would’ve liked them to allow resolution of implicits (which I’ve explored in alternative implicits proposal in the form of lenses). This is an advanced feature, which is also coupled with the work on implicits, so I think it should be left out of the scope of this SIP.

I like substance of this words, it seems that the documentation promises much more than it has been planned in implementing. It can lead to misunderstandings and abuses.

1 Like

IMO the ability to delegate only methods of a specific trait would be very useful.

Copied my 50 cents from support “proper” delegation pattern:

Currently, there are two options with export:

  1. write a wildcard, excluding all unrelated members:
    export fieldDelegateTo.{unrelatedMember => _, _}
  2. list all methods from the trait:
    export fieldDelegateTo.{foo1, foo2, ... fooN}

The first option is not only a boilerplate,
it has a high risk of exporting too many unrelated public members.
You have to explicitly list all members to “mute”. You can easily skip some.
If a new member is added to the class, most likely you will forget to update all the exports.
It may lead to context pollution.
It will potentially increase compilation time, show a lot of unexpected members in the completion list in IDE, etc…

The second option is also a boilerplate code.
You have to list all the members.
You can accidentally export some unrelated member.
You have to update all exports in case a new method is added to the trait.

Without “proper” delegation, export feels a little bit incomplete.
The motivation for export was to make “composition over inheritance” require less boilerplate code.
It’s very likely that wildcard import will be overused (cause programmers are lazy), leading to all the cons listed above.

We need some new syntax, which would allow us to only export methods of a specific trait.
Something like export delegate.{_: MyTrait1}:

trait MyTrait1 {
  def traitFoo: String
}
trait MyTrait2 {
  def traitBar: String
}
class A extends MyTrait1 with MyTrait2 {
  override def traitFoo: String = ???
  override def traitBar: String = ???
  def unrelatedMethod1: String = ???
  def unrelatedMethod2: String = ???
}
class B extends MyTrait1 with MyTrait2 {
  private val impl = new A()
  export impl.{_: MyTrait1, _: MyTrait2}
}

From the first glance, this feature shouldn’t require a lot of effort.
(patch import selector parsing and restrict filtering of exported methods in Namer.scala).
The most difficult is as always to agree on the syntax.

2 Likes

There are two ways this bounded export could be done - either we look for definitions of Trait1 meaning the object being exported from needs to inherit from that trait, or do you want a structural filter, so we match methods by name and signature?

Probably the proposed syntax ^ is a little confusing.
Like @smarter mentioned in the ticket:

export impl.{_: MyTrait1, _: MyTrait2}
In any case it looks like it’s exporting values of type MyTrait1 rather than members of MyTrait1.
I think this was discussed before on contributors but my preferred syntax would be:
export (impl: MyTrait1).*
which would be roughly equivalent to:

val impl1: MyTrait1 = impl
export impl1.*

I meant somthing like this ^

Then why not just write:

class B {
  private val impl: MyTrait1 with MyTrait2 = new A()
  export impl._
}

Your original impl field can have type MyImplType, not just the list of traits that are required to be exported (it can be a restriction)
You need to create a duplicated separate field just to export it.
The feature could nicely avoid this.

Okay, so this could be useful for import too:

import (impl: MyTrait1).*

But is avoiding a duplicated separate field really worth it?

The workaround looks awkward.
You will have strange fields that:

  1. will take space in the source code, distracting from the important code
  2. come up in the completion list
  3. come up in the bytecode.

You will have more than one field in case you need to delegate to several interfaces.

It doesn’t look like the feature requires that much effort, from what I’ve already seen in the Namer.scala.

I believe (although could be wrong) the proposed changes in the thread Proposed Changes and Restrictions For Implicit Conversions (2. Bulk Extensions > 2. Allow the qualifier of an export clause to be an expression instead of a stable identifier) would cover your usecase, too.

So something like
export (impl: MyTrait1 & MyTrait2).*

should eventually be possible, as per my understanding and if everything goes right with said proposal.

Please take a look at the Transparent term aliases thread, because the same applies to exports.

There are two different uses cases for the export keyword:

  • to define members “from a template” (to copy the declaration and add a forwarding),
  • to automatically import members from another object on a wildcard import.

In the first case, the definition should be opaque, in the second - transparent. However, currently export seems to imply opaque export. Although “forwarding exports” can seemingly also be used as “dual of imports”, this is an implementation detail, not the semantics, which is especially clear in IDEs.

It may be useful to formally distinguish between opaque exports and non-opaque export, in the same way Scala distinguishes between opaque types and non-opaque type aliases. (Another option is to consider transparent exports or inline exports.)

4 Likes