Idiomatic Imports

Language Design
11

I’d like to throw one fact into the ring: factoring out imports is a very common pattern in Python. A typical library will have a bunch of files in a folder, and the folder’s __init__.py will have a bunch of imports from these files that get re-exported (as all imports in Python do!).

Here are some examples:

I wonder if it’s possible to make exports work with wildcards, the way imports can work with wildcards? That would make the language more consistent than jt is now, and allow us to use the “in language” exports feature to accomplish this rather than an “out of language” compile option

5 Likes
12

But exports do work with wildcards?

13

Wildcard exports were disabled for packages (for incremental compilation reasons)

2 Likes
14

C# 10 has added project wide imports, declared in source files, with its “global using directives”:

3 Likes
15

Ah, I note it’s an implementation restriction caused by a zinc issue. Maybe this can be fixed?

6 Likes
16

Fun fact: exactly this problem (composable import statements: i.e. how to package a few import statements into one). was a motivation for annotated imports proposal in 2012. scala @exported imports pre-SIP - Google Docs

The idea was a reuse a package object to have something like (in old syntax)

package object  myLayer {
    @exported import myDependency1._
    @exported import myDepencency2._
    ....
}

and then use

import myLayer._

instead

import myDependency1._
import myDepencency2._
....

Eventually, scala3 receive exports, which partially close near half of the original problems. But exports are implemented as a proxy instead changing in compile-time search, which make this use-case non-implementable.

How about a variant, to reintroduce non-proxy exports as ‘transparent export’?
I.e.

package myLayer
object exports {
    transparent export myDependency1.*
    transparent export myDependency2.*
}

and then use

import myLayer.exports.*

instead

import myDependency1._
import myDepencency2._
....

Can this be interesting?

I.e. transparent export does not generate proxies, but adds placement of exported definitions to the search path of enclosing scope).

(In addition, one can say, that having a big list of imports can be an indication of modularity problems, but often this is just a track of used software stack because scala has no 'all batteries includes` library and people assemble their stack from a set of relatively small independent libraries.).

2 Likes
17

@rssh What is the added value of all that “object export transparent” stuff, compared to this simple scheme (if it could “just work”, even with incremental compilation)?

package myLayer

export myDependency1.*
export myDependency2.*

All files beginning with package myLayer would then share the same name space carved out by all package-level exports.

18

And an import myLayer.* would give direct access to the dependencies…

19

This would be analogous to something we use in Scala 2, and new devs consistently struggle with the difference between these two:

package foo.bar.baz
package foo
package bar.baz

Oh, and it’s also surprising for them that definitions in src/main/scala/foo/package.scala become unavailable if you add src/test/scala/foo/package.scala (because packages are otherwise open), so something that didn’t depend on this would be a much easier fix to work with.

20

Because by definition, each export generates a proxy-object – export alias. (Export Clauses ). So, export myDependency1.*. will create export aliases for each object from myDependency1. in myLayer. Implementation restrictions are caused by the necessity of creating these export aliases.

Maybe in some ideal future world, the compiler will so smart, that he can deduce – where to implement exports via proxies and where - via changes in the search path. In such case, you ‘just work’. variant will just work. But many programmers are afraid when the compiler is too smart because they want to know exactly how scala code is compiled down to bytecode, looking on it.

For this case, we can introduce transparent export which does not create proxies.

About the additional object: you are right, this is not essential. I.e. we can think that top-level exports belong to the package.

package myLayer

transparent export myDependency1,*
transparent export myDependency2.*

Mb we can receive from this ‘just works’ variant again, by specifying that exports in package scope are transparent, but this introduces some ‘irregularity’.

1 Like
21

Hmm, in analog to the function definition, it’s more like inline than transparent.
Also, if experimental erased classes will go into the mainstream, then the best fit will be erased export.

22

I think transparent and inline and erased in general often might be difficult to grasp by a beginner programmer, but shaping a namespace is something a beginner programmer indeed wants to, and export is really cool for that.

But having to explain the underlying translation model and the need for transparent or inline or erased when using wildcard export on package level seems to me like something a teacher/learner would consider an unnecessary newcomer friction…

23

Also I view transparent, inline and erased as advanced concepts that are used when you care about optimization or want to do meta-programming or type-level stuff.

But “normal” simple stuff should not require transparent, inline, erased or other “unnecessarily advanced” concepts.

24

Looks like having an 'external complexity` vs ‘internal complexity’ dilemma ;).

2 Likes
25

For me, packaging imports is already an advanced concept, so this argument doesn’t convince me
(In the sense that a course can easily wait until the more advanced subjects to introduce it)

Also I believe most people learning the language can perfectly use an “advanced keyword” without understanding what it means in general, but by just knowing what it does in this case
(Just look at introductory courses to Java)

26

Of course a student has to initially accept some of the taught concepts with a surface understanding, but each such extra “cryptic moment” is still a burden as the grit of students and pedagogical design and teaching time can be better spent. I have taught beginner Java and beginner Scala and doing away with things like public class Bla { public static void main(Array[String blabla is a really great win for almost all students that don’t have any pre-knowledge.

I also have first hand experience with using pedagogical principles of concept comparison (contrasting, generalizing, separating, fusing), in particular with the concepts of import versus export rather early on with the goal to understand name spaces and visibility rules. It has turned out to work really great for helping beginners to understand in more depth how name spaces work by comparing wildcard import versus wildcard export on top level / package level / inside singleton.

1 Like
27

I would go so far as to say that, for student work, this is a feature they shouldn’t need to ever see. Student projects don’t tend to get large enough for imports to be onerous, nor do they usually get into stuff like Cats, so the main drivers for this capability are kind of irrelevant to them.

2 Likes
28

Actually, this discussion makes me realize I’m not clear on what export does when a class(-like) thing is exported. Condensed from @Swoorup example:

object Prelude:
  export cats.effect.{IO, IOApp}

IO is an abstract class and associated companion object. IOApp refers to a trait and companion. What do these two exports do?

The Scala 3 export docs lead with “An export clause defines aliases for selected members of an object.” But I doubt that can literally apply here… since IO/IOApp aren’t “members” (in the usual sense of val/def/var), nor is cats.effect an object, but rather a package.

In preceeding posts, both @rssh and I have mentioned the generation of “forwarders” (synthetic methods that forward on references to the exported member). But I’m unclear, when eg IO is exported, what forwarders are created?

This is most relevant because @rssh’s proposed extension to export allowed it to declare “non-proxy exports” (ie exports without forwarders).

As he put it, to add “placement of exported definitions to the search path of enclosing scope”. I take this to mean: such an export affects the set of imports the compiler searches at the code site where the export is visible, as if an import was declared there.

I think Ruslan started from the position that, as a change to the specified behavior of export, this would require a distinguishing keyword, eg transparent. The only other option would seem to be to realign or clarify the behavior of export when it refers to the contents of packages, that forwarders are not generated but rather the search path is expanded. I do think such an export keyword would be doing “double duty” and this might be ambiguous.

Also, there are phase differences surely? Import search paths are a compile-time concept, while generated forwarders exist at runtime. Thus, @bjornregnell, I do think different keywords are required to delineate two different meanings of export.

29

Exported Classes (even abstract ones) are turned into type aliases, and then also a def is created to point to any companion object, e.g.:

object Ex:
  object Inner:
    abstract class Foo
    object Foo
  export Inner.*
  def test = new Foo {}

the export expands to

export Ex.Inner.*
final def Foo: Ex.Inner.Foo.type = Ex.Inner.Foo
final type Foo = Ex.Inner.Foo
def test: Ex.Inner.Foo = (new Ex.Foo() {}): Ex.Inner.Foo
1 Like
30

@bishabosha Could it perhaps expand to this:

inline final def Foo: Ex.Inner.Foo.type = Ex.Inner.Foo

So that a forwarder does not cost an extra call?