Having Another Go at Exports (Pre-SIP)


#1

Several previous initiatives proposed a “export” or “public import” feature that expresses forwarding.
The latest thread on this forum, opened by @japgolly a year ago contains links to several earlier proposals.

I would like to propose a new Pre-SIP for this feature that comes with detailed rules and an implementation. The rest of the proposal here quotes from the doc page, which contains several additional details.

Example

An export clause defines aliases for selected members of an object. Example:

  class BitMap
  class InkJet

  class Printer {
    type PrinterType
    def print(bits: BitMap): Unit = ???
    def status: List[String] = ???
  }

  class Scanner {
    def scan(): BitMap = ???
    def status: List[String] = ???
  }

  class Copier {
    private val printUnit = new Printer { type PrinterType = InkJet }
    private val scanUnit = new Scanner

    export scanUnit.scan
    export printUnit.{status => _, _}

    def status: List[String] = printUnit.status ++ scanUnit.status
  }

The two export clauses define the following export aliases in class Copier:

  final def scan(): BitMap            = scanUnit.scan()
  final def print(bits: BitMap): Unit = printUnit.print(bits)
  final type PrinterType              = printUnit.PrinterType

They can be accessed inside Copier as well as from outside:

  val copier = new Copier
  copier.print(copier.scan())

Rules

An export clause has the same format as an import clause. Its general form is:

  export path . { sel_1, ..., sel_n }
  export implied path . { sel_1, ..., sel_n }

It consists of a qualifier expression path, which must be a stable identifier, followed by one or more selectors sel_i that identify what gets an alias. Selectors can be of one of the following forms:

  • A simple selector x creates aliases for all eligible members of path that are named x.
  • A renaming selector x => y creates aliases for all eligible members of path that are named x, but the alias is named y instead of x.
  • An omitting selector x => _ prevents x from being aliased by a subsequent
    wildcard selector.
  • A wildcard selector creates aliases for all eligible members of path except for
    those members that are named by a previous simple, renaming, or omitting selector.

A member is eligible if all of the following holds:

  • its owner is not a base class of the class(*) containing the export clause,
  • it is accessible at the export clause,
  • it is not a constructor, nor the (synthetic) class part of an object,
  • it is an implied instance (or an old-style implicit value)
    if and only if the export is implied.

It is a compile-time error if a simple or renaming selector does not identify any eligible
members.

Type members are aliased by type definitions, and term members are aliased by method definitions. Export aliases copy the type and value parameters of the members they refer to. Export aliases are always final. Aliases of implied instances are again implied (and aliases of old-style implicits are implicit). There are no other modifiers that can be given to an alias. This has the following consequences for overriding:

  • Export aliases cannot be overridden, since they are final.
  • Export aliases cannot override concrete members in base classes, since they are
    not marked override.
  • However, export aliases can implement deferred members of base classes.

Export aliases for value definitions are marked by the compiler as “stable”. This means that they can be used as parts of stable identifier paths, even though they are technically methods. For instance, the following is OK:

  class C { type T }
  object O { val c: C = ... }
  export O.c
  def f: c.T = ...

Export clauses can appear in classes or they can appear at the top-level. An export clause cannot appear as a statement in a block.

(*) Note: Unless otherwise stated, the term “class” in this discussion also includes object and trait definitions.

Motivation

It is a standard recommendation to prefer aggregation over inheritance. This is really an application of the principle of least power: Aggregation treats components as blackboxes whereas inheritance can affect the internal workings of components through overriding. Sometimes the close coupling implied by inheritance is the best solution for a problem, but where this is not necessary the looser coupling of aggregation is better.

So far, object oriented languages including Scala made it much easer to use inheritance than aggregation. Inheritance only requires an extends clause whereas aggregation required a verbose elaboration of a sequence of forwarders. So in that sense, OO languages are pushing programmers to a solution that is often too powerful. Export clauses redress the balance. They make aggregation relationships as concise and easy to express as inheritance relationships. Export clauses also offer more flexibility than extends clauses since members can be renamed or omitted.

Export clauses also fill a gap opened by the shift from package objects to toplevel definitions. One occasionally useful idiom that gets lost in this shift is a package object inheriting from some class. The idiom is often used in a facade like pattern, to make members of internal compositions available to users of a package. Toplevel definitions are not wrapped in a user-defined object, so they can’t inherit anything. However, toplevel definitions can be export clauses, which supports the facade design pattern in a safer and more flexible way.

Syntax changes:

TemplateStat   ::=  ...
                 |  Export
TopStat        ::=  ...
                 |  Export
Export         ::=  ‘export’ [‘implied’] ImportExpr {‘,’ ImportExpr}

#2

I’d really like to see export in the language but I’m not a fan of implementing them using forwarders, in my experience compiler-generated forwarders are problematic, in particular here are some of my concerns with this proposal:

  • Outlining: so far in Scala, we’ve been able to get a list of the declarations
    in a class just by running the parser, this is useful for example with the
    Language Server Protocol where one frequently-sent server request is for an
    outline of the current document.
  • Parallel compilation: https://github.com/lampepfl/dotty/pull/4767 generates
    tasty outline files to allow for parallel compilation, having forwarders that
    need to be pickled makes this more complicated (not sure how much).
  • Leaky encoding: when a forwarder method is created, should it keep all the flags
    from the forwarded methods ? Should it keep all its annotations ? Both options have upsides and downsides.

Alternatively, we could try to define export as a mechanism that changes name resolution when importing something that exports something else, and does not generate any extra code.


#3

I am not sure we have a new situation wrt outlining. We already generate some definitions that the user did not write, such as apply/unapply/copy for case classes, or default getters. How are these handled in the outlines?

I don’t think parallel compilation is affected by the proposal. As things stand, there is no trace of export clauses in Tasty; they are fully expanded in Namer.

Leaky encoding: when a forwarder method is created, should it keep all the flags
from the forwarded methods ? Should it keep all its annotations ? Both options have upsides and downsides.

The proposal as is would keep none of the flags and annotations except for implied, if it’s an implied export.


#4

They won’t be shown which is OK since the outline is really a way to navigate in the file, I guess the same reasoning apply for export forwarders so I retract my point.

Right, but that require Namer to type the export expression to figure out what members need to be forwarded, so it’s an extra thing that needs to be done to be able to generate outlines.

So access restrictions like private[foo] will be dropped in the forwarders ?


#5

Yes, I think that makes sense. If you want something else, write the forwarder explicitly.


#6

Currently people (over)use inheritance and cakes because there is no way to have the forwarder semantics without writing tons of boilerplate. This is the gap that export is supposed to fill IMHO.

So the comparison should be between export and inheritance, not between export and manual definitions. Do you think export introduces anything fundamentally more difficult to deal with than inheritance?


#7

@odersky what would be the drawback of changing the lookup rules instead of synthesizing forwarders? It would avoid lots of code being generated, allow talking about precedence, and maybe also be simpler to deal with name conflicts.


#8

Would it be possible making proxy classes by exports?

trait Session{
  def call1():Unit
  def call2():Unit
  def call3():Unit
}

class SessionProxy(val session:Session) extend Session{
  export session
  
  def call1():Unit={
    doSomeThing()
    session.call1
  }
  
}

#9

I was thinking about a lookup scheme first. Below is a draft indicating how far I got before I abandoned that line of thought.

The problem with a lookup based scheme is that it complicates further what is already complicated: lookup. This means many interactions with other parts of the compiler. By contrast, the forwarder scheme is simple in the sense of being independent from the rest of the compiler. Once forwarders are generated in Namer we are done; no other part of the compiler needs to be concerned with exports. Also, the Tasty format does not need to have entries for them.

About size of generated code: Yes it’s a concern, but not worse than what we already do for mixins. If size of the generated byte code was of primary importance we could decree that all export alias methods are inline, which means no byte code is generated for them. That would give us power roughly on par with a lookup based scheme. But it would mean that export aliases would be invisible to Java and that they could not implement interface methods.


layout: doc-page
title: “Export”

Syntax changes:

TemplateStat      ::=  ...
                    |  Export
Export            ::=  ‘export’ ImportExprs

Similarly to imports, an export with several parts

export iexpr_1, ..., iexpr_n`

is a shorthand for

export iexpr_1
...
export iexpr_n`

To explain how export clauses influence type checking we explain in the following one possible scheme. As always, compilers are free to pick a different scheme if the observable results are the same.

An export export prefix . selectors in a template is legal if prefix is a path that refers either to a member of the template or to a globally accessible object.
Such an export generates a public “export” member with a compiler-generated name that’s inaccessible to user programs and globally unique, and a special type

ExportType(PT, E)

where E is the export clause itself and PT is the type of prefix. The PT part of an ExportType behaves as usual with regards to type maps such as substitution or as-seen-from.

When typing a selection p.m, if p does not have a member named m, the following adaptation is tried before searching for an implicit conversion of p:

Let es be all the export members of p. We first search all explicit exports and then, if that returns no results, all wildcard exports for a match with m, exactly as it is done when processing import statements. If there are several matching members they all must have the same prefix type PT, or an ambiguity error is reported. Otherwise, pick an arbitary element of the set of matching members, say member e defined in class C with type ExportType(PT, export q . ss). The selection p.m is then rewritten to one of the following alternatives, depending on the form of q.

  • If q refers to a globally accessible object: q.m
  • If q is of the form C.this . q_1 . ... q _n: p . q_1 . ... . q_n . m

#10

You can make proxy classes, but your example would not work, since call1 would be doubly defined. But you can express it like this:

trait Session{
  def call1(): Unit
  def call2(): Unit
  def call3(): Unit
}

class SessionProxy(val session:Session) extends Session {
  export session.{call2, call3}

  def call1(): Unit = {
    println("call1")
    session.call1()
  }
}

A variation of the proposal would automatically suppress export aliases if they would lead to a conflict. If that was done, you could write export session._. It would be easy to do but I am not sure we want to go that far. I’d be interested to read people’s opinion on this.


#11

It is difficult to maintain.
Actually when one release of library contains call4 and previous releases does not contain it. It is real headache.

I think it will be killer feature for proxies.


#12

It is very useful for dynamic dependency injection.


#13

If I understand the proposal correctly, you can do export session.{ call1 => _, _ }.


#14

If I understand the proposal correctly, you can do export session.{ call1 => _, _ }

Yes indeed. I think that’s the best solution. Lightweight and makes clear what gets generated.

My main reservation against automatically suppressing forwards on conflicts are the possible surprises. “Why did it not install an alias for this method? Oh it’s because a method with the same name is inherited through this sequence of traits!” That’s the kind of surprises we know from inheritance that we want to avoid here.


#15

How will it work with overloaded methods?

IIUC Such way to exclude ones is not very convenient for overloaded methods.


#16

Hey - I’d love to see some functionality along these lines.

I think there are two possible semantics here and I think we should be clear which of the two we are targeting. The first is a very light-weight semantic of simply pulling identifiers into scope automatically when you pull other identifiers in. The second is a heavyweight semantic of binding new identifiers as aliasses for underlying identifiers.

class Copier {
  private val scanUnit = new Scanner
  export scanUnit.scan
}

So the two potential semantics I can see here are:

  • lightweight: wherever I have val c: Copier in scope, I have import c.scan._ also in scope, with any needed magic to navigate visibility restrictions
  • heavyweight: the class Copier is augmented with a bunch of synthetic members def foo = this.scanUnit.foo, adjusted as needed for signature

The distinction really, really matters. In the former case, we don’t make any new values. There are no new implied instances, for example. It lets you lift members out of objects to package level for facades, or out of instances to support call-sight delegation, for example.

In the latter case, it becomes possible to use the mechanic to implement API-level delegation. Classes get new, concrete members proxying in the delegate members. However, this comes with the potential cost of aliasing, for example, resulting in clashing implied instances pointing to the same value, or accidentally holding things in memory through unexpected references.


#17

Overloaded methods will be excluded in bulk. So, yes, if you want to replace one of them you have to replace all of them. I believe that’s still an acceptable price to pay.

Holding things in memory is not an issue because all aliases are defs. Clashing implicits: I am not yet sure whether this would be an issue in practice. If it is an issue one could tweak the resolution rules to
declare two implicits the same if they forward to the same value.


#18

Thank you it is clear.
It seems I am in other camp :slight_smile:
I just do not understand which gain will we receive for that price?
I can see name clashing. I think it is the same question as variable shadowing.
For example let us try to imagine which gain we will receive if variable shadowing is not enabled. Will it be better?
I can see troubles but I cannot see much gain comparable to that troubles .
I think such name clashes just increase coupling in the case when we use

export someVal._ 

I do not think that

export someVal.{method =>_, _ }

is a good statement.

Let us imagine that instead of override def someMethod() we would use something like ‘class A extends B.{method =>_, _ }’.
I think it is something very ugly.


#19

Good question. The difference is two (or three?) degrees of implicitness vs one. For variable shadowing, a variable you write could shadow a variable further out. For export hiding, some member that you think is accessed by a wildcard export is in fact not aliased since there is some preexisting definition in some baseclass with the same name and signature. There’s not a single syntactic instance that could tell you this is even happening. I believe this is not comparable with variable showing, and is clearly a lot more problematic.


#20

IIUC thare are 2 points

  1. It is not obvious whether exports should “shadow” the base member or it should be shadowed.
  2. there can be shadowing which is made by mistake

I think if the first point is forbidden nobody will really suffer.
The second point can be solved very easy with some annotation:

trait Session{
  def call1(): Unit
  def call2(): Unit
  def call3(): Unit
}
trait SomeAspect{
  def call1(): Unit = {}
}
class SessionProxy(val session:Session) extends SomeAspect with Session {
  export session._
  
@overimport
  override def call1(): Unit = {
    super.call1()
    session.call1()
  }
}

I cannot see any advantages in export someVal.{someMethod=>} when I want to exclude method. But disadvantages is obvious at least for overrided method.
Just to note in PreparedStatement
it seems that more than half methods are overrided.