Updated Proposal: Revisiting Implicits

Let’s fix the compile-time messages, not allow multiple ways to call things and then force the user to switch between them in order to get useful messages.

If anything, this is even more of an argument to forbid going both ways: during debugging, people will randomly switch from postfix, as intended, to function-with-arguments style, and then leave it after it compiles.

3 Likes

That’s a nice goal, but until we get there, we should probably avoid crippling our ability to debug extensions when things go wrong :slight_smile:

Linters can help us remember to switch things back, until the error messages become clear enough to make this facility redundant

If extensions are so hard to debug that they motivate adding an irregularity to the language, we shouldn’t be using them. There shouldn’t be any “until”, except during development. If the feature goes live, it should be pleasant to work with.

Our current mechanism for extensions is worse, as the only way to debug them is to convert something like value.pure[F] to something like this:

new ApplicativeOps(value).pure[F]

So, while not ideal, the new way is considerably less painful to use than the old way - and despite this wart extension methods still provide enough benefits to justify their existence.

1 Like

Based on the feedback I got from here I have run some limited experiments and tried some alternative syntaxes. They can be summed up as follows:

  • use witness instead of given in instance definitions: PR #7928
  • use default instead of given in instance definitions: PR #7941
  • use with instead of given for parameters: PR #7973

The experiments reflect my belief that the semantics of the new contextual abstractions are sound and work well, but that there might still be room to make the syntax clearer.

Feedback on either the PRs or here is very welcome.

2 Likes

I think it could be a good idea to consider and decide upon some principles for the syntax before considering concrete keyword alternatives. I think it should be considered in this order:

  1. What “kind of word” should be used and how should the constructs be read in real-world language?
  2. Should the same keyword be used for definitions and parameters?
  3. Based on this, which word has the best connotations of “something that is applied implicitly”?

Some comments on the first point:

Some of the previous debate has included the question whether it should be a noun, adjective, or verb. Let’s investigate by example:

given Foo

To me, this is read as “a given instance of Foo”. As “an instance of X” could be shortened to “an X” we could read it simply as “a given Foo”. So given is here a modifier and thus an adjective. We might even say that it modifies the type Foo from a normal type to a given type.

(The verb variant, give Foo, would have a completely different meaning: the difference would be akin to the difference between imperative and declarative.)

But what does “given” mean? So far, that does not matter. If we decide that it should be a modifier, any adjective would do. It could be a word that has some of the “implicit” connotations: magic, enchanted, implied, auto, default, given, provided… or not: red, big, innocent, beautiful, strange. Yes, we could exchange given with beautiful and it would work just the same.

On the other hand:

witness of Foo has a slightly more complicated meaning. As a noun, it does not modify Foo, it is a thing in itself, which has semantic relationship to that which it it is a thing of. Thus we cannot replace it with any other noun, and the meaning of the word must be considered immediately. Previous proposed alternatives of this kind have included representative (repr) and instance (and probably others I don’t remember). We could generalize it to thing of Foo (which probably wouldn’t work that well in itself).

Actually, we could unify the two variants by saying that the general form is:

special instance of Foo

In the “given” variant, we leave out “instance of” and shorten it to special Foo and then exchange “special” with whatever adjective we like.

In the “witness of” variant, we exchange “special instance” with a word that has similar connotations to “special instance”.

After playing with these three alternatives for a while, here’s my evaluation:

There are three different “levels” of implicit definitions: instance definitions, context parameters, and context functions (i.e. implicit function types and closures). I believe it is best if each level has a different syntax. It’s less regular, but a lot easier to parse. That sentiment was also brought up in several comments on this thread.

There are several classes of implicit uses. The most important ones are

  • Context passing
  • Typeclasses
  • Proofs
  • Conversions
  • Extensions

Extensions have their own syntax now, and proofs and conversions can be seen as special cases of typeclasses. So that leaves “context passing” and “typeclasses” as the two principal flavors of implicits.

Let’s name the three explored alternatives after the name of the instance. followed by the name indicating contextual parameters. So it’s witness/given, default/given, and given/with. Here’s my evaluation how suitable these three combinations are for the two principal use classes:

                          context passing      typeclasses
witness/given             -                    +
default/given             ++                   -
given/with                +                    +

given/with has the edge in that it works for both use classes equally well, so I am pursuing this alternative further. PR 8017 is a complete implementation. In this implementation, both the previous given/given syntax and the new given/with syntax are supported, but the alternative to use => for conditional givens introduced in 0.20 has been removed. My plan is to get this merged by the next Dotty release early February, and to switch everything to the new syntax afterwards. In the PR the tests already use the new syntax but the main implementation does not.

We would then use one or two 6 weeks release cycles to try the new syntax in depth, and hopefully come to a final decision afterwards. I had hoped that we would be in feature freeze by now, but it’s very important to get this right, so I think we should give ourselves the time needed to reflect on this.

11 Likes

This prompted a question in my mind: What other use cases are there in the wild? :slight_smile:

1 Like

I read through the docs and a few examples of the PR. I like the big picture, in particular

  • the separation of concerns (given instances, implicit conversions, extension methods)
  • the way extension methods are defined
  • with clauses (I could live with a different keyword, but no strong feelings)

But I still have a hard time getting used to the definition syntax of given instances. I have two concerns.

1 – The syntax for defining given instances is different than defining ordinary values or methods. In given i as T { }

  • the definition’s type is T, the new as keyword is somehow doing what : does for normal definitions
  • i doesn’t introduce a new type, but it’s defined with a block (not with =), similar to an object definition
  • as can be read the wrong way around: “foo as bar” can mean “i take foo and give it the name bar”. Here it’s the other way around, I guess the meaning is “foo is defined as bar”.

The situation also reminds me of Java annotations, where a new syntax was invented to define annotation types.

2 – One has to remember how given instances are represented / desugared (val vs lazy val vs def). I’m pretty sure advanced users (and people defining given instances are advanced users) at least need to know, and some of them would be able to easily control it.


So I basically prefer to use the syntax of ordinary definitions. The only thing that maybe looks less good is defining anonymous instances, but I think it’s still a better compromise. We could use a marker (_) or even just leave the name away.

given object intOrd extends Ord[Int] {}
given object _ extends Ord[Int] {}
given object extends Ord[Int] {}

given val intOrd: Ord[Int] = ...
given val _ : Ord[Int] = ...
given val : Ord[Int] = ...

given def intListOrd with Ord[Int]: Ord[List[Int]] = ...
given def _ with Ord[Int]: Ord[List[Int]] = ...
given def with Ord[Int]: Ord[List[Int]] = ...

given def listOrd[T] with (ord: Ord[T]): Ord[List[T]] = ...
given def _ [T] with Ord[T]: Ord[List[T]] = ...
given def [T] with Ord[T]: Ord[List[T]] = ...
13 Likes

Just out of curiosity, are we actually considering changing implicits again between Dotty 0.22 and Scala 3? I’m trying to gauge feature stability and syntax is a large part of that. The way given works in 0.22 is really good and more then enough for my needs. Can I actually start writing a large framework on top of it or is everything going to be ripped right out from under me in the next SIP meeting?

2 Likes

given object intOrd …

This proposal would defeat one of the purposes of given, which is to remove the implementation-level, mechanical concern between implicit defs and implicit vals that is currently a source of confusion and difficulty for new Scala developers.

Actually current approach does not defeat the implementation-level. It chooses default behavior by probabilistic algorithm. It is unintuitive behavior. So It just increase amount of magic code and force me to write ugly code to emulate def.

2 Likes

Close. I’d read

given ord as Ord[T] { ... }

“given ord as the Ord[T] instance where …”

If the “ord as” part is missing it’s just “given the Ord[T] instance where…”.

1 Like

As far as I am concerned I hope that we have reached a fixed point. I am actually quite happy with the current proposal. If someone has a brilliant idea how to improve some aspect of it further over the next weeks I am happy to consider, of course. But my expectation is that we are by and large done now.

Generally, supporting votes like yours are very valuable in these discussions. I have found that, time and time again a proposal has very little support and a lot of negative comments. It’s only when the proposal is changed that the former silent supporters speak up in favor for what was by then lost. So to regain balance it’s also important that the people who like (some part) of a proposal voice their support.

2 Likes

I agree that given ... with ... as is the best of the proposed variants.

However I think that the witness ... given ... of form would be superior if one could find an alternative word to witness that would work for both typeclasses and context-passing.

The main reason is that a solid noun reads better when defining instances. The combination witness ... with ... of could in that case also be considered.

Btw I’m happy that this discussion continues. I agree it’s important to get it right and as such I think all possibilties should be explored and all aspects analysed before deciding that “this is as good as it gets”.

If it were up to me, I would put a “gag-order” on all discussions involving implicit syntax/semantics until after the 3.1 release. We should be focusing discussions and tooling and bugs!

only joking of course, but maybe I should hold back making slide decks like this: https://www.slideshare.net/pjschwarz/scala-3-by-example-better-semigroup-and-monoid :sweat_smile:

Given the rocky transition from the old SIP process, I have a feeling a gag order would be received extremely poorly

To clarify: I would understand a gag order of this type to mean, “The people with doubts about this syntax are to shut up until it’s too late to change anything.”

6 Likes

I see your point but I’m sure you understand my intention. I really, really want to see some closure on this topic. Otherwise I can’t justify spending days and weeks porting libraries to the new semantics.

I think given/with/as improves the situation. I still find it irregular, and I don’t like the tense being used, but it at least is quite clear what everything is, which is arguably the most important consideration. I no longer think that the syntax is a flat-out mistake. I really like that the with captures all the arguments in one block so there is zero ambiguity about whether a parameter is given or not.

I continue to think the underlying capability is good, but that the syntax (and the capabilities!) are insufficiently regular.

Anyway, I’ve said plenty about this before. Summary: thumbs more up than before, but with only modest enthusiasm.

Incidentally, with clauses can apparently be interspersed with regular parameter blocks. This seems a really bad idea to me. I’ve said this before, but I just want to make it clear that the new syntax does not in any way help.

def foo(x: X) with Y (z: Z = zero)

either can be called as

foo(x)(nonZero)

which makes it confusing as to what is an explicit parameter and what is given, or cannot and instead must be called as

foo(x) with (y) (nonZero)

which both makes the with clause pointless, as you have to supply it anyway, and has the strange construct (y) (nonZero) that looks like it ought to have tighter binding than to with, but doesn’t.

So I think with clauses ought to have to come last, after all normal parameter blocks. They add a lot of confusion without pulling much weight if they are allowed in the middle.

4 Likes