Proposal: Implied Imports

Scala Improvement Process
1

In Scala so far, a wildcard import like:

import A._

will make all members of A visible in the current scope, and will also make all implicit definitions in A available when doing implicit search. A typical Scala file will have many wildcard imports, but only a couple if any of these imports will bring implicits in scope, this means that it’s very hard to answer the question “Where do implicits come from in this file ?” simply by reading the code, this probably contributes to the feeling among some users that implicits in Scala are scary and hard to control.

The Implied Imports proposal attempts to address this by adding an explicit syntax to import implied instances into scope. This proposal is one part of the set of proposals known as Contextual Abstractions which are being discussed in Principles for Implicits in Scala 3 and Proposal To Revise Implicit Parameters, please make sure to post your comments on the most appropriate thread to keep the discussion manageable.

1 Like
2

Do you have data to back that up?

I find that not to be the case at all with my code: implicits come in all the time.

Also, in many cases, they come along with companion objects, which presumably they still would. Those would be possibly even harder to find because first you’d be tempted to check the spurious explicit imports.

Futhermore, this seems to be on the wrong side of history: the trend it to use more typeclasses, not fewer, and thus implicits are liable to become more common, not less. So even if you have good data indicating that

  1. Wildcard imports rarely import any implicits, and
  2. The prevalence of wildcard imports bringing in implicits is a major barrier to preventing discovery of implicits

it is not clear that this will continue.

So depending on data I range from skeptical to strongly negative on the proposal. Indeed, I usually have the opposite problem: it is not easy enough to get the implicits in scope, causing myself and other users to go on a scavenger hunt for them!

3

In imports? I think of that as excepionally bad design!

Implicits in companion objects are much less problematic since they are always avaiilable and have been co-designed with the types they work with. The problem that many people have brought up is implicits coming from “magic” imports. These are too easily hidden at present, and too puzzling if they are forgotten.

So, yes, it will make importing implicits more explicit and more verbose, and that’s very much the point! It will hopefully have some influence on people to look for better designs, and make it clearer what imports are problematic. Furthermore, by making implicit imports more explicit it is much easier to remember them, the problem of forgetting some magic import should at least be partially addressed by this.

4

Okay, I’m a bit confused here. Say that I have a typeclass instance, and therefore need implicits to bring that into scope. Does that require the new import implied mechanism?

I generally agree with @Ichoran here – I import implicits all the time, especially in this context, and certainly don’t want more ceremony in order to do so. It sounds like you are saying that it isn’t needed in this case – but if that’s true, I definitely don’t understand the rules of this proposal, and when it applies…

5

I’m a bit afraid that imports become overcomplicated like in TypeScript. There
import blah from 'ns'
is not the same as
import { blah } from 'ns',
which confuses sometimes. You can get used to that, but it is not obvious.
Another fear is that there will be a lot of code duplication like:

import A._
import implied A._
import B._
import implied B._
import C._
import implied C._

Which doubles the size of all imports and make it less concise.

And by the way, most of the libraries have special package for implicits (like cats.implicits._, etc). That approach sounds simpler than adding new language construction for this case, in my opinion.

6

Okay, I’m a bit confused here. Say that I have a typeclass instance, and therefore need implicits to bring that into scope. Does that require the new import implied mechanism?

If the typeclass instance is an orphan instance, yes. If it isn’t, you don’t need an import anyway.

I generally agree with @Ichoran here – I import implicits all the time, especially in this context, and certainly don’t want more ceremony in order to do so.

I believe, at least in the future, if you need to import implicits all the time, that’s a code smell that should be addressed. The fact that you are saying this makes me feel that import implied is even more important than I assumed before because it forces you to note the code smell and hopefully do something about it.

I am saying “in the future”, since unfortunately today many imports are still necessary because of a lack of flexibility in the implicit constructs Scala offers. For instance you often need to import implicit decorators to get infix notation. That’s going away with extension methods.

7

I do this ubiquitously in order to provide a pleasant user experience instead of having them run afoul of the relatively limited set of defaults for implicit search paths. I’ve attempted to use numerous libraries like so:

import my.lib._
import my.lib.thingy._

def run = theThingy(foo)

only to get some mysterious error. Then in looking through the docs I eventually find a complete runnable example that was using theThingy and realize that I needed

import my.lib._
import my.lib.implicits._
import my.lib.thingy._
import my.lib.thingy.implicits._

and now it works the way it was always intended.

This is exceptionally bad design. Core functionality should just work when you bring into scope those things that make up the core functionality. Heck, I even wrote some of my own libraries like that and I was constantly frustrated by it.

I’m not sure about the future, but presently I use package objects primarily to hold implicits that are essential for the functioning of a package. For instance, if I have some sort of validation class–let’s call it Ok–and I want it to interoperate seamlessly with Either and Option and Try, I want to have something like

implicit class EitherCanBeOk[L,R](private val underlying: Either[L,R]) {
  def toOk: Ok[L, R] = underlying match {
    case scala.util.Right(r) => Yes(r)
    case scala.util.Left(l) => No(l)
  }
}

come along for the ride. If I create a new serialization format with a typeclass that provides default serialization, I want implicits for ToBase85[String] and stuff coming along too.

Sometimes there are legitimately cases where you might want to use something without the implicits mucking stuff up (e.g. you can use JSON without having all the serializer/deserializer machinery implicitly around, so maybe there it should be separate; or you may not want the compatibility layer there because you don’t want to use the stuff you’re compatible with).

But mostly I find that splitting out implicits into their own namespace–which, incidentally, we can already do, and could be implemented as a lint instead of a core part of the language–mostly brings a lot of inconvenience.

3 Likes
8

Another fear is that there will be a lot of code duplication like:

import A._
import implied A._
import B._
import implied B._
import C._
import implied C._

Which doubles the size of all imports and make it less concise.

That’s exactly according to plan. If you have pervasive imports like that, that’s a code smell that should be addressed. This is precisely one of the problems of the implicit rut we are in, and it will take some effort to get out of it.

And by the way, most of the libraries have special package for implicits (like cats.implicits._ , etc).

Then nothing much will change, since you need a separate import anyway. It will mean that future libraries will not have to split the implicits into separate packages anymore. import implied is more visible (since it will be syntax-highlighted) and predictable.

If all libraries would do this, then maybe import implied would not be absolutely necessary (even though it would not hurt either in case). But in my experience this organization is still the exception, not the rule.

4 Likes
9

If this is so bad, and the good cases are all covered by improvements in default places to look for implicits and so on, do we actually need to import implicits at all? Why not just drop the feature entirely?

It seems to me that the design space where a feature is so problematic that you want to discourage it, yet so vital that you must include it, is pretty narrow. Getting out of that chasm would seem to be highly desirable.

10

I believe that right now, you generally should not need to import your implicits. As mentioned, having the “imports wrong” is the #1 usability problem with implicits, since there is zero guidance from the compiler or otherwise to help you figure out how to fix it.

This means that implicits should as much as possible be defined in companion objects, to be brought in automatically. This is done in uPickle, Requests-Scala, OS-Lib, and other libraries I maintain, and works pretty well. Occasionally you need to define-and-import orphan implicits when you need a T[V] and both T and V are from external libraries, but those are uncommon cases and not something that you do “by default”

In fact, I’d go a step further and say wildcard imports:

import my.lib._

Are themselves a code smell, implicits or not. I would be very happy if we could remove this functionality and make people import things explicitly; in many other languages (e.g. Python, JS) wildcard imports are explicitly discouraged.

Name the things you want to import! Implicits being imported are a bit awkward because the imported name doesn’t get used, but good, importing implicits should be slightly awkward and not the seamless default! If the explicitly-named imports are getting hairy because the library needs a huge pile of random names and implicits imported to do anything, that’s very much a problem with the library.

If we decide to remove wildcard imports (big “if”!) we could end up with a syntax like:

import a.b.{c, d, e}
import a.b.{c, implicit d, e}
import a.b.{c, implicit d, implicit e}

to demarcate exactly which names we import are implicit, and which are not. I think that would be a pretty good state of affairs to get into, e.g. allowing people to import both the global ExecutionContext and the parasitic ExecutionContext for convenience, but decide which one they want to be implicit:

import scala.concurrent.ExecutionContext.{implicit global, parasitic}
import scala.concurrent.ExecutionContext.{global, implicit parasitic}
1 Like
11

I think this principle is too restrictive in how it’s expressed by the programmer. The programmer should want either to import all but implied instances, or all instances, including the implied ones (when using wildcards). I don’t understand the reasoning why we force the programmer to express two different import statements instead of just annotating whether or not the import includes implied or not.

12

What about implied exports (assuming export gets accepted into the language)?

I disagree. I think that we have messy imports due to lack of export. With export we can finally compose importable objects that are good as wildcard imports.

13

I think this is more common than you believe. Besides the canonical library of these for Cats and the Standard Library (alleycats), a trivial search brings up three independent projects providing this for Cats and Scalacheck. There are also additional projects along these lines for Cats & Scalatest, and equivalent projects if you replace “Cats” with “Scalaz” in the preceding examples.

That’s aside from the internal libraries designed to correct ergonomic issues with Scalatest, Scalacheck, Mockito, Anorm, and etc that I currently help maintain in my day job, which depend heavily on implicits to make them usable. Better discoverability and error messages would be very helpful, as currently we rely on an import convention similar to cats.

Making define-and-import more difficult would greatly increase the adoption barriers of these libraries, and removing this capability altogether would put us right back into the space of relying on the library designers to anticipate every use case. Based on the number of libraries providing this functionality, there’s clearly an unmet need which the library designers aren’t able to fulfill.

Some of them are great about trying to provide as many of these instances as possible (shout out to enumeratum, who do an exceptional job of this), but others wouldn’t be able to keep up (scalacheck is a good example of an excellent library with a dearth of maintainers). Regardless, given the number of possible connections between libraries, I don’t think it’s reasonable to put this burden on them in the first place.

14

What about implied exports (assuming export gets accepted into the language)?

They also get an implied variant, with the same semantics.

1 Like
15

I believe that most of the need for implicit imports today comes from extension methods, like the toOK method you describe:

implicit class EitherCanBeOk[L,R](private val underlying: Either[L,R]) {
  def toOk: Ok[L, R] = underlying match {
    case scala.util.Right(r) => Yes(r)
    case scala.util.Left(l) => No(l)
  }
}

In the future, if you want to make this method universally accessible, you’d define or export the extension method at the toplevel of your project. No implicit needed.

  def (underlying: Either[L, R]) toOK[L, R] = underlying match {
    case scala.util.Right(r) => Yes(r)
    case scala.util.Left(l) => No(l)
  }

So, let’s analyze when you would still need an implicit import by classifying the different kinds of implicits.

  1. Typeclasses: You need imports only for orphan instances, which should be avoided anyway if possible. If that’s not possible, a highly visible import is good since it alerts the reader that this code uses some orphan instances.
  2. Extension methods. Can be handled directly.
  3. Contexts such as ExecutionContext. If you have a root context, define it with the context class, as is done in ExecutionContext.global. If you want to override a third party root context with something in your project, this deserves a highly visible import.
  4. Capabilities, configurations. Same remarks as for contexts apply.
  5. Implicit conversions. If they are defined with the source or target type, no import is necessary. If they are defined neither with the source type nor with the target they are evil. Requiring a highly visible import is the least we should do in that case (in fact we currently require a language import in addition to this).

Are there other classes of implicits? It would be good to add them to the classification.

In summary, so far, it seems that in future well-designed systems implicit imports should be both rare and would profit from being very visible. That’s what this proposal achieves.

The same cannot be said for today’s systems because of extension methods implemented by decorators. That’s why the proposal exempts current implicits from the import implied tax. I.e. as long as you use implicit classes for extension methods, your users can get them with the same imports as before. But once a package switches to Scala 3 implicits, the import implied is required.

2 Likes
16

Are you also considering idea of importing something into implicit scope. Something like:

object ExecutionContext {
  val global: ExecutionContext = ???
}

import implied ExecutionContext.global

val a = implicitly[ExecutionContext]
1 Like
17

import implied ExecutionContext.global

You don’t need the import for that use case.

18

Well, it’s just an example. global is not defined as implicit (val global: ExecutionContext = ??? but not implicit val global: ExecutionContext = ???). We don’t need implicit keyword for such a case at all, because we can control implicit/not implicit scope only by a way how we import stuff. Just an idea from the top of my head, maybe a stupid one.

19

Ah yes, I overlooked that global in your example was not implicit. Yes, maybe. We considered that in the SIP committee and the general reaction was rather luke-warm. So, we can leave it for later as well.

2 Likes
20

I do not know how this cases can be named, so I just have tried:

  • serialization\deserialization
    • sql
    • orm
    • rpc
    • xml
    • json
    • etc

There are libraries which provide types and such library do not know about “serialization\deserialization” and there are libraries which provide “serialization\deserialization”.

How can such libraries be easily integrated?

It is not rare tasks in my practice. And it is ironic cases. I want to use implicits and I do not like it in such cases, so I don’t use it if I can :slight_smile:

1 Like