Principles for Implicits in Scala 3

[virusdave]

As to anti-patterns: I believe most of them have to do with busting the limited implicitness budget.

[snip]

That’s why I think that the implicit-as-a-modifier syntax can be a trap for the uninitiated. We are normally used to the fact that a modifier can be put on anything.

[snip]

In fact there are very tight criteria that have to be met before you should make a definition implicit . Having a completely separate syntax serves as a better marker that something different is going on.

Amen! This is exactly what i was trying to express previously! I’m glad to see this as part of the guiding principles for the future of implicits.

[mdedetrich]

Can someone please specify what this

In fact there are very tight criteria that have to be met before you should make a definition implicit

“very tight criteria” is?

As far as I can tell, there is no coherent definition of such a thing since there are so many exceptions which makes the exercise fruitless. I can say right now that apart from implicit conversions (which we should stop talking about) that for every orthogonal combination of implicit right now there are multiple valid cases.

I have personally written every such combination at least once, and I think that if you ask library authors of major libraries they would have probably done it 10 times more.

I honestly would like to see some concrete cases of what is considered as “wrong usage of implicits” (and more importantly wrong usages of implicits that never have an equivalent right usage) so that we have some basis to make some argument.

[tarsa]

The theory being, if that’s what you do, then the fact that this becomes now an implicit is the most important part, so we want to draw people’s attention to it. The mechanics how you do that is secondary.

Sounds good.

Actually, in the current system getting the mechanics right is also very hard, in particular if your implicits are polymorphic or conditional or if they define operators. If all of these come together, getting this right is so far the domain of true experts (that is, everybody else who attempts it is likely to screw it up one way or another). In the new system, it’s completely obvious how to achieve this.

Could you give some examples?

[virusdave]

Could you give some examples?

examples of “getting the mechanics right is hard”?

I’d point to Slick’s use of Shape, and the various mechanisms used there. Despite the sophistication of them, for some rather nontrivial polymorphic use cases i have to manually thread implicits around often. In about 2 weeks (post lambda jam conference) i should be able to point to some concrete examples if reminded.

[tarsa]

Don’t forget about this part:

In the new system, it’s completely obvious how to achieve this.

[yangbo]

Would you explain what do you mean as “run-away success”?.
For example I think scala collection library is a success which is impossible without implicits.

I think “run-away success” means a language feature that is copied into other main stream languages.

[tarsa]

In about 2 weeks (post lambda jam conference) i should be able to point to some concrete examples if reminded.

I’m reminding you

[joshlemer]

I think that in designing the next version of implicits for Scala 3, it would be nice to have some kind of answer for the “trait object” use-case, as described here. Basically, it would be nice to talk about a collection of data, where the underlying types of the data elements are different, but all of which have an instance of some typeclass defined.

[BardurArantsson]

Since nobody’s mentioned Haskell in this thread…

From a prefers-Haskell-but-will-use-Scala person’s perspective: Implicits look like a “mechanism” bleeding through to the layer of semantics. It’s hugely powerful in many ways, e.g. implicit resolution can be used as logic programming, etc. etc., but reading “implicit […]” in code doesn’t actually tell the reader what that thing is going to do. FWIW, Haskell has also somewhat succumbed to this when talking about dependent types, etc.

I personally think that it’s more important to make intent explicit in the language.

[odersky]

Here’s an update: The SIP committee voted to accept provisionally the changes to implied instances and given clauses. This means:

• We will go forward with the design of these two components as they are currently specified and implemented in the Dotty codebase.
• Before we ship 3.0 final (sometimes in 2020) we will evaluate whether the new features saw significant use and what the usage experience was. If these are judged unsatisfactory we keep the possibility open to drop one or both of the changes. In this case there would be no alternative proposal for a dropped change.

So if you are keen to have the revised implicits, you should use them extensively and report back to fix any usability problems! That’s the best way to make sure they stay in.

This decision does not yet extend to import implied. We have a tweak for this which will be presented on its discussion thread.

[soronpo]

Does it mean the revised implicits will only be available under an experimental flag like was done for macro paradise?

[odersky]

Does it mean the revised implicits will only be available under an experimental flag like was done for macro paradise?

No. Revised implicits will be available as a standard feature. As far as I am concerned, they are every bit as solid as the other Scala 3.0 additions and I will use them and teach them extensively. There was just an open question in the SIP committee whether they would find enough adoption. For instance, if all the widely used libraries keep on using the old implicits, we have obviously failed convincing the community that the change was worth it. For this case the SIP committee reserved the possibility to reconsider.

I think that’s fair. Revised implicits are probably the most significant change we do to Scala. We should get feedback from extensive usage experience before they are finalized.

[soronpo]

Maybe I missed it, but is there a change in the concept of @implicitNotFound (at the very least, a name change is required)?

[odersky]

There’s no change to implicitNotFound I know of. If something’s not working, maybe file an issue?

[odersky]

I think that in designing the next version of implicits for Scala 3, it would be nice to have some kind of answer for the “trait object” use-case, as described here . Basically, it would be nice to talk about a collection of data, where the underlying types of the data elements are different, but all of which have an instance of some typeclass defined.

As far as I can see, the encoding you give works well. So maybe it’s more a matter of promoting this design pattern?

[b-studios]

From my experience of using the new implicit design (aka. given) in my Scala Effekt code base, I have to say it works really great! I love explicitly providing arguments with given and being able to omit (not having to come up with) term names. I finally can throw away most of my type aliases to create implicit function types and replace them with given. For example:

def drunkFlip given Amb given Exc = for {
  caught <- Amb.flip()
  heads  <- if (caught) Amb.flip() else Exc.raise("too drunk")
} yield if (heads) "heads" else "tails"

That’s beautiful!

The only improvement I see there would be to be able to mix the order of given parameter sections and explicit parameter sections in order to allow path dependencies on given parameters. IIRC this was possible at one stage of the design. I like to use implicits to guide type inference with path dependent types and this is not possible if implicits always have to be last.

WRT implied instances, I am not very happy with the design. Please correct me if my impression is outdated since the design changed. My major pain points are:

1. Implied instance declarations always have to provide a type. This easily leads to Java-esque repetition patterns implied for SomeLongName = new SomeLongName.
2. Overloading for, it maybe just me – but I have a hard time remembering the syntax and order of identifier, for token, type etc. With implicit def x: T this is not so much of a problem since I already learnt def x: T
3. Only supporting implicit defs. I know that there are workarounds for most cases – however in very special and advanced cases one might want to rely on object identity (i.e. singleton types) even for implicits which we loose with unstable defs.

As I mentioned earlier on Github, to me binders (def, val, …), creating instances (new, object, …) and making something implicit are orthogonal concepts. Yes, this might lead to a combinatorial explosion, but I didn’t have problems learning those three concepts on their own and then combining them. As opposed to most people, who argue type classes are easier to understand, I don’t yet buy that this is the case – in particular for people without a FP background.

[odersky]

Thanks for the experience report!

Implied instance declarations always have to provide a type. This easily leads to Java-esque repetition patterns implied for SomeLongName = new SomeLongName .

In this case you would just write: implied for SomeLongName. No = is needed.

Only supporting implicit defs: If you need object identity, or there is some other reason to create a val you can always use this pattern:

   val x = e
   implied for T = x

That is for all intents and purposes equal to

   implicit val x: T = e

[b-studios]

Thanks for the clarifications, Martin.

I wasn’t aware of these details. How do you determine whether implied for will correspond to implicit val or implicit def? Is this specified somewhere? Will the two following examples both generate implicit vals or only the first?

val x = e
implied for T = x
implied for T = { println("hello"); x }

EDIT: from playing around with Dotty, it looks like both will generate (lazy) val’s. That is the side effect is only executed once, when the implicit is used the first time (and the result is cached).

[odersky]

How do you determine whether implied for will correspond to implicit val or implicit def? Is this specified somewhere?

It’s described here: https://dotty.epfl.ch/docs/reference/contextual/relationship-implicits.html

According to this description, you’ll get a def for both instances you wrote.

val x = e
implied for T = x
implied for T = { println("hello"); x }

In the first case, the def is simply a forwarder to the val. It would be useless and wasteful to create another field for this. In the second case, you’d get def over a cached var. The def is needed since the expression is impure. To get something that behaves like a strict val, you have to follow the same pattern as in the first case.

val y = { println("hello"); x }
implied for T = y

[b-studios]

Thanks for the quick response.

Implied alias instances map to implicit methods. If an alias has neither type parameters nor a given clause, its right-hand side is cached in a variable. There are two cases that can be optimized

That explains a lot! But then the next question is: how can you encode implicit defs that have side effects every time the implicit is used? Maybe sometimes
you don’t want to cache the results.

Of course you could always create a newtype that thunks the computation, but that’s quite roundabout. Are these usages of implicits you want to discourage?