Proposal: Implied Imports

#5

I’m a bit afraid that imports become overcomplicated like in TypeScript. There
import blah from 'ns'
is not the same as
import { blah } from 'ns',
which confuses sometimes. You can get used to that, but it is not obvious.
Another fear is that there will be a lot of code duplication like:

import A._
import implied A._
import B._
import implied B._
import C._
import implied C._

Which doubles the size of all imports and make it less concise.

And by the way, most of the libraries have special package for implicits (like cats.implicits._, etc). That approach sounds simpler than adding new language construction for this case, in my opinion.

#6

Okay, I’m a bit confused here. Say that I have a typeclass instance, and therefore need implicits to bring that into scope. Does that require the new import implied mechanism?

If the typeclass instance is an orphan instance, yes. If it isn’t, you don’t need an import anyway.

I generally agree with @Ichoran here – I import implicits all the time, especially in this context, and certainly don’t want more ceremony in order to do so.

I believe, at least in the future, if you need to import implicits all the time, that’s a code smell that should be addressed. The fact that you are saying this makes me feel that import implied is even more important than I assumed before because it forces you to note the code smell and hopefully do something about it.

I am saying “in the future”, since unfortunately today many imports are still necessary because of a lack of flexibility in the implicit constructs Scala offers. For instance you often need to import implicit decorators to get infix notation. That’s going away with extension methods.

#7

I do this ubiquitously in order to provide a pleasant user experience instead of having them run afoul of the relatively limited set of defaults for implicit search paths. I’ve attempted to use numerous libraries like so:

import my.lib._
import my.lib.thingy._

def run = theThingy(foo)

only to get some mysterious error. Then in looking through the docs I eventually find a complete runnable example that was using theThingy and realize that I needed

import my.lib._
import my.lib.implicits._
import my.lib.thingy._
import my.lib.thingy.implicits._

and now it works the way it was always intended.

This is exceptionally bad design. Core functionality should just work when you bring into scope those things that make up the core functionality. Heck, I even wrote some of my own libraries like that and I was constantly frustrated by it.

I’m not sure about the future, but presently I use package objects primarily to hold implicits that are essential for the functioning of a package. For instance, if I have some sort of validation class–let’s call it Ok–and I want it to interoperate seamlessly with Either and Option and Try, I want to have something like

implicit class EitherCanBeOk[L,R](private val underlying: Either[L,R]) {
  def toOk: Ok[L, R] = underlying match {
    case scala.util.Right(r) => Yes(r)
    case scala.util.Left(l) => No(l)
  }
}

come along for the ride. If I create a new serialization format with a typeclass that provides default serialization, I want implicits for ToBase85[String] and stuff coming along too.

Sometimes there are legitimately cases where you might want to use something without the implicits mucking stuff up (e.g. you can use JSON without having all the serializer/deserializer machinery implicitly around, so maybe there it should be separate; or you may not want the compatibility layer there because you don’t want to use the stuff you’re compatible with).

But mostly I find that splitting out implicits into their own namespace–which, incidentally, we can already do, and could be implemented as a lint instead of a core part of the language–mostly brings a lot of inconvenience.

2 Likes
#8

Another fear is that there will be a lot of code duplication like:

import A._
import implied A._
import B._
import implied B._
import C._
import implied C._

Which doubles the size of all imports and make it less concise.

That’s exactly according to plan. If you have pervasive imports like that, that’s a code smell that should be addressed. This is precisely one of the problems of the implicit rut we are in, and it will take some effort to get out of it.

And by the way, most of the libraries have special package for implicits (like cats.implicits._ , etc).

Then nothing much will change, since you need a separate import anyway. It will mean that future libraries will not have to split the implicits into separate packages anymore. import implied is more visible (since it will be syntax-highlighted) and predictable.

If all libraries would do this, then maybe import implied would not be absolutely necessary (even though it would not hurt either in case). But in my experience this organization is still the exception, not the rule.

4 Likes
#9

If this is so bad, and the good cases are all covered by improvements in default places to look for implicits and so on, do we actually need to import implicits at all? Why not just drop the feature entirely?

It seems to me that the design space where a feature is so problematic that you want to discourage it, yet so vital that you must include it, is pretty narrow. Getting out of that chasm would seem to be highly desirable.

#10

I believe that right now, you generally should not need to import your implicits. As mentioned, having the “imports wrong” is the #1 usability problem with implicits, since there is zero guidance from the compiler or otherwise to help you figure out how to fix it.

This means that implicits should as much as possible be defined in companion objects, to be brought in automatically. This is done in uPickle, Requests-Scala, OS-Lib, and other libraries I maintain, and works pretty well. Occasionally you need to define-and-import orphan implicits when you need a T[V] and both T and V are from external libraries, but those are uncommon cases and not something that you do “by default”

In fact, I’d go a step further and say wildcard imports:

import my.lib._

Are themselves a code smell, implicits or not. I would be very happy if we could remove this functionality and make people import things explicitly; in many other languages (e.g. Python, JS) wildcard imports are explicitly discouraged.

Name the things you want to import! Implicits being imported are a bit awkward because the imported name doesn’t get used, but good, importing implicits should be slightly awkward and not the seamless default! If the explicitly-named imports are getting hairy because the library needs a huge pile of random names and implicits imported to do anything, that’s very much a problem with the library.

If we decide to remove wildcard imports (big “if”!) we could end up with a syntax like:

import a.b.{c, d, e}
import a.b.{c, implicit d, e}
import a.b.{c, implicit d, implicit e}

to demarcate exactly which names we import are implicit, and which are not. I think that would be a pretty good state of affairs to get into, e.g. allowing people to import both the global ExecutionContext and the parasitic ExecutionContext for convenience, but decide which one they want to be implicit:

import scala.concurrent.ExecutionContext.{implicit global, parasitic}
import scala.concurrent.ExecutionContext.{global, implicit parasitic}
1 Like
Wildcard imports considered harmful
#11

I think this principle is too restrictive in how it’s expressed by the programmer. The programmer should want either to import all but implied instances, or all instances, including the implied ones (when using wildcards). I don’t understand the reasoning why we force the programmer to express two different import statements instead of just annotating whether or not the import includes implied or not.

#12

What about implied exports (assuming export gets accepted into the language)?

I disagree. I think that we have messy imports due to lack of export. With export we can finally compose importable objects that are good as wildcard imports.

#13

I think this is more common than you believe. Besides the canonical library of these for Cats and the Standard Library (alleycats), a trivial search brings up three independent projects providing this for Cats and Scalacheck. There are also additional projects along these lines for Cats & Scalatest, and equivalent projects if you replace “Cats” with “Scalaz” in the preceding examples.

That’s aside from the internal libraries designed to correct ergonomic issues with Scalatest, Scalacheck, Mockito, Anorm, and etc that I currently help maintain in my day job, which depend heavily on implicits to make them usable. Better discoverability and error messages would be very helpful, as currently we rely on an import convention similar to cats.

Making define-and-import more difficult would greatly increase the adoption barriers of these libraries, and removing this capability altogether would put us right back into the space of relying on the library designers to anticipate every use case. Based on the number of libraries providing this functionality, there’s clearly an unmet need which the library designers aren’t able to fulfill.

Some of them are great about trying to provide as many of these instances as possible (shout out to enumeratum, who do an exceptional job of this), but others wouldn’t be able to keep up (scalacheck is a good example of an excellent library with a dearth of maintainers). Regardless, given the number of possible connections between libraries, I don’t think it’s reasonable to put this burden on them in the first place.

Wildcard imports considered harmful
#14

What about implied exports (assuming export gets accepted into the language)?

They also get an implied variant, with the same semantics.

1 Like
#15

I believe that most of the need for implicit imports today comes from extension methods, like the toOK method you describe:

implicit class EitherCanBeOk[L,R](private val underlying: Either[L,R]) {
  def toOk: Ok[L, R] = underlying match {
    case scala.util.Right(r) => Yes(r)
    case scala.util.Left(l) => No(l)
  }
}

In the future, if you want to make this method universally accessible, you’d define or export the extension method at the toplevel of your project. No implicit needed.

  def (underlying: Either[L, R]) toOK[L, R] = underlying match {
    case scala.util.Right(r) => Yes(r)
    case scala.util.Left(l) => No(l)
  }

So, let’s analyze when you would still need an implicit import by classifying the different kinds of implicits.

  1. Typeclasses: You need imports only for orphan instances, which should be avoided anyway if possible. If that’s not possible, a highly visible import is good since it alerts the reader that this code uses some orphan instances.
  2. Extension methods. Can be handled directly.
  3. Contexts such as ExecutionContext. If you have a root context, define it with the context class, as is done in ExecutionContext.global. If you want to override a third party root context with something in your project, this deserves a highly visible import.
  4. Capabilities, configurations. Same remarks as for contexts apply.
  5. Implicit conversions. If they are defined with the source or target type, no import is necessary. If they are defined neither with the source type nor with the target they are evil. Requiring a highly visible import is the least we should do in that case (in fact we currently require a language import in addition to this).

Are there other classes of implicits? It would be good to add them to the classification.

In summary, so far, it seems that in future well-designed systems implicit imports should be both rare and would profit from being very visible. That’s what this proposal achieves.

The same cannot be said for today’s systems because of extension methods implemented by decorators. That’s why the proposal exempts current implicits from the import implied tax. I.e. as long as you use implicit classes for extension methods, your users can get them with the same imports as before. But once a package switches to Scala 3 implicits, the import implied is required.

2 Likes
#16

Are you also considering idea of importing something into implicit scope. Something like:

object ExecutionContext {
  val global: ExecutionContext = ???
}

import implied ExecutionContext.global

val a = implicitly[ExecutionContext]
1 Like
#17

import implied ExecutionContext.global

You don’t need the import for that use case.

#18

Well, it’s just an example. global is not defined as implicit (val global: ExecutionContext = ??? but not implicit val global: ExecutionContext = ???). We don’t need implicit keyword for such a case at all, because we can control implicit/not implicit scope only by a way how we import stuff. Just an idea from the top of my head, maybe a stupid one.

#19

Ah yes, I overlooked that global in your example was not implicit. Yes, maybe. We considered that in the SIP committee and the general reaction was rather luke-warm. So, we can leave it for later as well.

2 Likes
#20

I do not know how this cases can be named, so I just have tried:

  • serialization\deserialization
    • sql
    • orm
    • rpc
    • xml
    • json
    • etc

There are libraries which provide types and such library do not know about “serialization\deserialization” and there are libraries which provide “serialization\deserialization”.

How can such libraries be easily integrated?

It is not rare tasks in my practice. And it is ironic cases. I want to use implicits and I do not like it in such cases, so I don’t use it if I can :slight_smile:

1 Like
#21

Do you categories cover scala.collection.JavaConverters?

#22

Do you categories cover scala.collection.JavaConverters ?

Yes: they are extension methods, so will need no implicit support in the future.

@Matveev:

Third party serialization packages are typical examples of orphan instances. They require import implied.

#23

It is very good news, is there any description of it.

IIUC

Ok, It is important to know.

Although It is a pitty that such popular library like anormsql can be considered as rare case.
Why cannot it be simplified as extension methods?

#24

Again, can someone please explain the motivation behind the import implied semantics. Even if I agree that import should not import implied instances, why doesn’t import implied just import all instances including the implied ones? I see no viable use case for just a import implied A._ statement if it imports just implied instances in A (and even if we found such a use-case, we can always just created object implicits with the relevant implied instances).

Additionally, what will happen to the implicit conversion inside Predef? Will the Predef object be imported by default to include the implicits?