Idiomatic Imports

Language Design
1

Scala has really good tools for factoring out repetition and duplication from code. If you take care, you can write Scala where almost every line of code has a high information content.

With one exception… there’s one section of a Scala source file where boilerplate is so routine that IDEs like Intellij hide it by default: the imports.

When I look at almost all Scala codebases I work on, the imports across different source files are highly correlated. If, for example, I have

import cats.*
import cats.syntax.all.*

…in one source file, it’s extremely likely I’ll have the same, or similar imports, in the others in the same project.

(As an aside, I’ll mention that I do like wildcard imports, despite the fact that IDE tooling can automate some of the tedium of importing individual symbols. Firstly, pragmatically, they reduce noise in code diffs, because the import lines change less. But also, they can signal intent. If I import cats.* I’m signaling that I want to be able to refer to any symbols from that package without qualification.)

This post is a request for some language & tooling mechanism to define default, idiomatic imports across a project scope. Imports in a source file would be only customizations or overrides to the default, project-wide import configuration.

We came close in Scala 2.x with -Yimports, which gives the ability to pass a default set of imports to the compiler, but it was limited to whole packages / wildcards, which is not going to suit everybody. And this feature appears not to have made it into Scala 3 anyway.

But the itch for a way to factor out the repetitive boilerplate of imports across a codebase remains…

4 Likes
2

You can get there somewhat? with export I am not sure if anyone uses it this way (I don’t either) :stuck_out_tongue:

Prelude.scala

object Prelude:
  export cats.effect.syntax.all.*
  export cats.effect.{IO, IOApp}
  export cats.syntax.all.*
  export fs2.Stream
  export cats.effect.std.Queue
  export scala.concurrent.duration.{FiniteDuration, DurationInt}

Main.scala

import Prelude.*
5 Likes
3

I don’t think exports will ever substitute for imports in full generality because they can’t replicate package-wide imports like import cats.*

This in turn stems from the fact that exports need to enumerate the things they are exporting, to generate forwarders for each. Whereas imports are a place to search for a named symbol at compile-time. I think of it as akin to the difference between intentional and extensional definitions.

4

Maybe -Yimports can be added to Scala 3?

5

I’m personally not a fan of compiler options, I would prefer something with more precise control

Maybe we could make it so if there is a file called imports.scala in a directory, those imports are in the scope in all files in this directory and in subdirectories ?

This would allow us to enable features that are illegal unless imported to be accessible in a big part of a project, but not everywhere
(Implicit conversion being a good example of something we might want often, but not everywhere)

Edit:
Upon further thinking, I don’t think this would be a good idea, it would be better to find and fix issues with the “object containing exports” solution mentioned above, as it offers a lot of expressive power in an intuitive and transparent way

6

I am actually quite doubtful about any of this. Needing complex “idiomatic” imports is often a design smell by itself. Essentially, you are establishing a DSL that only an expert can be comfortable in. For newcomers and learners it’s better to be explicit.

Future changes to the language should make it easier to see what goes on rather than providing more ways to sweep complications under the carpet.

2 Likes
7

@edmundnoble assessment was blunt, but IMO accurate, when he observed that -Yimports was “really not useful in practice”.

The problem with -Yimports was that it exposed too cut-down a subset of Scala’s import functionality, and was missing critical pieces: no ability to import part of a package or object, no ability to rename or hide symbols.

What I am seeking is to express project-wide, baseline imports, in the same language used to write regular Scala source file imports.

8

I used the adjective “idiomatic” to apply not so much to the imports, as to the codebases that result under particular sets of imports. I believe idiomatic scala is already a reality. If one works in a Cats-based codebase, that will work differently to a ZIO codebase, and again differently to Akka. Each will have idioms and stylistic patterns, which could be viewed as a DSL, that are enabled by the imports of each library.

I think the thriving library ecosystems in Scala demonstrate that Scala developers want to opt-in to these idioms.

I work in industry, rather than education, and in my world, while approachability to newcomers and learners is undoubtedly a goal, it is a secondary goal to productivity and maintainability. In an eg Cats-based codebase, developers will be expected to know/learn Cats to work in it. That’s true independent of whether they must punch out the same paragraph of imports at the top of each file, or not.

Things that I observe happening in practice are:

  • Use of idiomatic Scala associated with particular libraries/ecosystems is already widely adopted and shows no signs of reversing
  • In codebases that use specific libraries, there’s a large amount of duplicated code (I consider it “boilerplate”) at the top of every file
  • Developers have become reliant on IDE tools like Optimize Imports or Import templates to automate the generation or maintenance of this boilerplate
  • The need to setup imports introduces a small but noticeable cost to splitting code into a new file. I find myself hesitating sometimes when considering introducing a file, thinking “I’ll need to get the imports just right if I move this… can I be bothered or just leave it here?”
5 Likes
9

I understand the tradeoffs, but I think it’s a risky attitude to take. An ecosystem that does not have a laser focus on onboarding and learning will die at some point. With everything we do we have to consider possibilities of abuse. And if we allow imports to be swept under the carpet the possibilities of abusing this for dialects and language fragmentation are huge.

2 Likes
10

So if we have duplicate code, and we place that code in one shared location, we call it “factoring out common code”. Generally, Scala is excellent at enabling this.

If we have duplicated imports across a project, and we could put those imports in a single shared location, would this not also be “factoring out common code”?

Why should this form of “factoring” earn the label “sweeping under the carpet”?

My post was not motivated by a frustrated desire to write idiomatic code (Scala already offers excellent support for idiomatic code, at the cost of imports). Rather by the simple observation that, in refining my practice of eliminating duplicate code, one of the major remaining sources of duplication is now the import statements, because there are no language mechanisms to factor them out

2 Likes
11

I’d like to throw one fact into the ring: factoring out imports is a very common pattern in Python. A typical library will have a bunch of files in a folder, and the folder’s __init__.py will have a bunch of imports from these files that get re-exported (as all imports in Python do!).

Here are some examples:

I wonder if it’s possible to make exports work with wildcards, the way imports can work with wildcards? That would make the language more consistent than jt is now, and allow us to use the “in language” exports feature to accomplish this rather than an “out of language” compile option

6 Likes
12

But exports do work with wildcards?

13

Wildcard exports were disabled for packages (for incremental compilation reasons)

3 Likes
14

C# 10 has added project wide imports, declared in source files, with its “global using directives”:

4 Likes
15

Ah, I note it’s an implementation restriction caused by a zinc issue. Maybe this can be fixed?

7 Likes
16

Fun fact: exactly this problem (composable import statements: i.e. how to package a few import statements into one). was a motivation for annotated imports proposal in 2012. scala @exported imports pre-SIP - Google Docs

The idea was a reuse a package object to have something like (in old syntax)

package object  myLayer {
    @exported import myDependency1._
    @exported import myDepencency2._
    ....
}

and then use

import myLayer._

instead

import myDependency1._
import myDepencency2._
....

Eventually, scala3 receive exports, which partially close near half of the original problems. But exports are implemented as a proxy instead changing in compile-time search, which make this use-case non-implementable.

How about a variant, to reintroduce non-proxy exports as ‘transparent export’?
I.e.

package myLayer
object exports {
    transparent export myDependency1.*
    transparent export myDependency2.*
}

and then use

import myLayer.exports.*

instead

import myDependency1._
import myDepencency2._
....

Can this be interesting?

I.e. transparent export does not generate proxies, but adds placement of exported definitions to the search path of enclosing scope).

(In addition, one can say, that having a big list of imports can be an indication of modularity problems, but often this is just a track of used software stack because scala has no 'all batteries includes` library and people assemble their stack from a set of relatively small independent libraries.).

2 Likes
17

@rssh What is the added value of all that “object export transparent” stuff, compared to this simple scheme (if it could “just work”, even with incremental compilation)?

package myLayer

export myDependency1.*
export myDependency2.*

All files beginning with package myLayer would then share the same name space carved out by all package-level exports.

18

And an import myLayer.* would give direct access to the dependencies…

19

This would be analogous to something we use in Scala 2, and new devs consistently struggle with the difference between these two:

package foo.bar.baz
package foo
package bar.baz

Oh, and it’s also surprising for them that definitions in src/main/scala/foo/package.scala become unavailable if you add src/test/scala/foo/package.scala (because packages are otherwise open), so something that didn’t depend on this would be a much easier fix to work with.

20

Because by definition, each export generates a proxy-object – export alias. (Export Clauses ). So, export myDependency1.*. will create export aliases for each object from myDependency1. in myLayer. Implementation restrictions are caused by the necessity of creating these export aliases.

Maybe in some ideal future world, the compiler will so smart, that he can deduce – where to implement exports via proxies and where - via changes in the search path. In such case, you ‘just work’. variant will just work. But many programmers are afraid when the compiler is too smart because they want to know exactly how scala code is compiled down to bytecode, looking on it.

For this case, we can introduce transparent export which does not create proxies.

About the additional object: you are right, this is not essential. I.e. we can think that top-level exports belong to the package.

package myLayer

transparent export myDependency1,*
transparent export myDependency2.*

Mb we can receive from this ‘just works’ variant again, by specifying that exports in package scope are transparent, but this introduces some ‘irregularity’.

1 Like