Hmm, in analog to the function definition, it’s more like inline than transparent.
Also, if experimental erased classes will go into the mainstream, then the best fit will be erased export.
I think transparent and inline and erased in general often might be difficult to grasp by a beginner programmer, but shaping a namespace is something a beginner programmer indeed wants to, and export is really cool for that.
But having to explain the underlying translation model and the need for transparent or inline or erased when using wildcard export on package level seems to me like something a teacher/learner would consider an unnecessary newcomer friction…
Also I view transparent, inline and erased as advanced concepts that are used when you care about optimization or want to do meta-programming or type-level stuff.
But “normal” simple stuff should not require transparent, inline, erased or other “unnecessarily advanced” concepts.
Looks like having an 'external complexity` vs ‘internal complexity’ dilemma ;).
2 Likes
For me, packaging imports is already an advanced concept, so this argument doesn’t convince me
(In the sense that a course can easily wait until the more advanced subjects to introduce it)
Also I believe most people learning the language can perfectly use an “advanced keyword” without understanding what it means in general, but by just knowing what it does in this case
(Just look at introductory courses to Java)
Of course a student has to initially accept some of the taught concepts with a surface understanding, but each such extra “cryptic moment” is still a burden as the grit of students and pedagogical design and teaching time can be better spent. I have taught beginner Java and beginner Scala and doing away with things like public class Bla { public static void main(Array[String blabla is a really great win for almost all students that don’t have any pre-knowledge.
I also have first hand experience with using pedagogical principles of concept comparison (contrasting, generalizing, separating, fusing), in particular with the concepts of import versus export rather early on with the goal to understand name spaces and visibility rules. It has turned out to work really great for helping beginners to understand in more depth how name spaces work by comparing wildcard import versus wildcard export on top level / package level / inside singleton.
1 Like
I would go so far as to say that, for student work, this is a feature they shouldn’t need to ever see. Student projects don’t tend to get large enough for imports to be onerous, nor do they usually get into stuff like Cats, so the main drivers for this capability are kind of irrelevant to them.
2 Likes
Actually, this discussion makes me realize I’m not clear on what export does when a class(-like) thing is exported. Condensed from @Swoorup example:
object Prelude:
export cats.effect.{IO, IOApp}
IO is an abstract class and associated companion object. IOApp refers to a trait and companion. What do these two exports do?
The Scala 3 export docs lead with “An export clause defines aliases for selected members of an object.” But I doubt that can literally apply here… since IO/IOApp aren’t “members” (in the usual sense of val/def/var), nor is cats.effect an object, but rather a package.
In preceeding posts, both @rssh and I have mentioned the generation of “forwarders” (synthetic methods that forward on references to the exported member). But I’m unclear, when eg IO is exported, what forwarders are created?
This is most relevant because @rssh’s proposed extension to export allowed it to declare “non-proxy exports” (ie exports without forwarders).
As he put it, to add “placement of exported definitions to the search path of enclosing scope”. I take this to mean: such an export affects the set of imports the compiler searches at the code site where the export is visible, as if an import was declared there.
I think Ruslan started from the position that, as a change to the specified behavior of export, this would require a distinguishing keyword, eg transparent. The only other option would seem to be to realign or clarify the behavior of export when it refers to the contents of packages, that forwarders are not generated but rather the search path is expanded. I do think such an export keyword would be doing “double duty” and this might be ambiguous.
Also, there are phase differences surely? Import search paths are a compile-time concept, while generated forwarders exist at runtime. Thus, @bjornregnell, I do think different keywords are required to delineate two different meanings of export.
Exported Classes (even abstract ones) are turned into type aliases, and then also a def is created to point to any companion object, e.g.:
object Ex:
object Inner:
abstract class Foo
object Foo
export Inner.*
def test = new Foo {}
the export expands to
export Ex.Inner.*
final def Foo: Ex.Inner.Foo.type = Ex.Inner.Foo
final type Foo = Ex.Inner.Foo
def test: Ex.Inner.Foo = (new Ex.Foo() {}): Ex.Inner.Foo
1 Like
@bishabosha Could it perhaps expand to this:
inline final def Foo: Ex.Inner.Foo.type = Ex.Inner.Foo
So that a forwarder does not cost an extra call?
It does not matter, the JVM will do the inline anyway. And the forwarder might actually implement an abstract method in a super trait, so inline does not always work.
1 Like
OK that’s cool. Nice that optimizations are done automatically and that my code does not need to be cluttered with performance optimizations 
It would be nice if this could work:
$ cat >test.scala
package myPack:
export math.*
@main def run =
import myPack.*
println(sin(0))
$ scala-cli run .
Compiling project (Scala 3.1.3, JVM)
[error] ./test.scala:2:10: Implementation restriction:
package scala.math is not a valid prefix for a wildcard export,
as it is a package.
[error] export math.*
[error] ^^^^
Error compiling project (Scala 3.1.3, JVM)
Compilation failed
and if it was prohibited because of a Zinc isssue and that could be solved so the above could be allowed that would great.
1 Like
Looking at original source of the problem: https://github.com/sbt/zinc/issues/226. looks like erased exports will not help: the issue will remain. (If you have wildcard export you should add a dependency from each file in the exported package. And even with this, if I will add in the exported package a file with a name, which will shadow some name in the file F which use export, zink should recompile F. I.e. dependency tracker should trigger recompilation of file when some new file is added and file consume exported package)
From another side, this looks not like a bug, but like a piece of work: reengineer zink to support such types of dependencies.
2 Likes
On -Yimports in Scala 2, note that it has downsides, because it breaks existing tooling. AFAIK currently there’s no way to export the needed -Yimports in the library’s JAR artefact, so tooling looking at library source files no longer have everything needed to compile those sources.
One problem is that we tend to rely on the IDE too much, so we tend to use complicated package hierarchies; and unfortunately this is a losing battle, as most people use IntelliJ IDEA, and even Metals can suggest imports, which makes packages to be very user unfriendly, by design; we have no “flat is better than nested” philosophy, so unlike Python.
Also, several popular Scala libraries rely on orphaned type class instances and orphaned extension methods (orphaned means not globally available without an explicit import). Scala 3 took big steps in dealing with this by making import suggestions for implicit values that may match. Discovery still suffers.
On the project I work on, many of the source files have at least 1 of these imports, most often 2 or more:
import cats.syntax.all._
import io.circe.syntax._
import akka.actor.typed.scaladsl.adapter._
import scala.concurrent.duration._
These are sort of a de facto standard for us, however if you have a less popular project, having users rely on yet another import syntax._ or import instances._ is a tough sell.
Truth be told, I’m less bothered by import cats._ and more bothered by import cats.syntax.all._. Because the former is about types (which can be discovered easily), whereas the latter is about extension methods or implicits.
If we can make export work with wildcards for a whole package, that would be great 
3 Likes
Resurrecting this thread because the problem remains and we are no closer to any solution. Actually, no solution is visible even on long-range radar 
What’s the problem? Modern Scala code written on top of library-ecosystems such as Typelevel is dependent on a huge number of imports to work correctly. The import section reaches 50 lines long in some of my application files, which rivals the actual application code.
Ensuring imports are consistent across files is a source of errors & low-value busy-work. Changes to import lines clog up diffs and distract from more interesting changes. Verbose imports add friction to refactors and experiments.
And at the root, the imports across multiple source files within a project are highly correlated and repetitive, so it is natural to wonder how this repetitive structure could be factored out. Right now, I can’t find any nice factoring solution up to Scala 3.3.
We’ve heard that, over a decade ago, Ruslan was thinking about solutions to this problem with his pre-SIP.
This was abandoned when it was hoped that export would solve the problem. But export, as it exists today, doesn’t allow whole package namespaces to be re-exported due to it’s focus on generating runtime forwarders. It can’t address the import problem in its present form.
We’ve heard that this is an “implementation limitation” in the Zinc compiler. But the relevant issue has stalled. There’s been no activity in years.
Personally, the most viable proposal I’ve heard is @rssh suggestion of a syntactic mechanism to declare “non-proxy” exports. Such exports do not have any runtime footprint, but rather change which namespaces the compiler looks at when compiling code in the scopes where the export applies.
1 Like
So if I understand this right, there are two different aspects:
- allowing whole-package exports to allow for “layering” as discussed above
- avoiding explicit forwarder generation for exports
Perhaps inline export somepackage.* would be a natural way to signal an opt-in for “recompile all the things”, potentially without creating forwarders, so the compiler reliably can find all names to export at the expense of efficient incremental compilation. It seems logical to me as inline in general means “more work at compile time”…
And when the compiler complains that export somepackage.* is not allowed, it can hint about adding inline in front and also explain that this might give longer compile times.
Question to someone with more compiler knowledge than me: would the above be feasible?
I’m not sure what you have in mind for “recompile all the things”, but the issue right now is that Zinc only tracks dependencies between class and has no notion of dependencies over a whole package, so when a class is added to a package, it doesn’t know that wildcard imports/exports of that package need to be recompiled. I’m currently working on various improvements to incremental compilation and I’m going to see if I can improve the situation here, but even if I succeed it’s going to require some changes to both Zinc and the compiler so it’ll take some time to get there.
7 Likes
I’m going to see if I can improve the situation here
That would be really nice!