Updated Proposal: Revisiting Implicits

Implicit Conversions

Unfortunately they don’t have parity with existing implicit defs yet. There are at least three differences:

  1. Conversions do not support path-dependency currently - [1]
  2. Combining them with macros aka inlines is very odd now because they’re “values”. They can be upcast and they can lose their “inline” status through upcasting. Because of that the compiler will emit an error on straightforward definition of a macro Conversion, you’ll have to define two traits and override a runtime method with an inline method. example: [2]
  3. Given and conversion names cannot be overloaded, because they are not methods – even when they take parameters [3]. This gets in the way of implicit punning technique – replacing a typeclass’s companion with a slew of implicit defs to ensure that whenever a typeclass is imported, its syntax is ALWAYS available with it. examples: [usage] [implementation with overloading]. Dotty’s extension methods and the fact that they’re available whenever an implicit is in lexical scope, which happens in strictly more cases than when a type name is imported make this trick unnecessary in most cases, except when extension methods are insufficient like in @morgen-peschke’s example.
1 Like

Honestly I don’t find the syntax great. given doesn’t seem to give us anything that implicit doesn’t: if we find-and-replace all the givens with implicits, we’d get all the benefits and much less disruption. The usage of :, and how strongly it binds left and right, also goes against what someone programming in Scala would expect.

Some more specific issues:

  • This syntax is pretty confusing: it looks like some kind of refinement type, but it isn’t:
given intOrd: Ord[Int] {
  def compare(x: Int, y: Int) =
    if (x < y) -1 else if (x > y) +1 else 0
}

If Scala didn’t have refinement types maybe this could be ok, but as is I find this extremely hard to parse. And the precedence is all wrong: normally { has higher precedence than :, but here it’s reversed.

  • I also think the anonymous given instances look much worse than what we currently have with implicits:
given Ord[Int] { ... }
given [T](given Ord[T]): Ord[List[T]] { ... }

Neither of these look particularly good. The top one has a type expression in a place that looks like no other language construct in existence, while the bottom looks like keyword soup with the two givens that do not match to any normal english usage of the word.

  • Alias givens honestly would look fine as implicit
given global: ExecutionContext = new ForkJoinPool()
implicit global: ExecutionContext = new ForkJoinPool() 

In fact, that’s almost the syntax now! We just need to make the def/val/lazy val optional, and we’re done.

  • I am personally not a huge fan of the extension method syntax. The precedence between given, :, extension and { seems all wrong in the given examples
given stringOps: extension (xs: Seq[String]) {
  def longestStrings: Seq[String] = {
    val maxLength = xs.map(_.length).max
    xs.filter(_.length == maxLength)
  }
}

Overall the usage of : in the given syntax doesn’t work out. It looks too much like something we’re already familiar with, and comes with baggage of the precedence we’re already used to, but is parsed in a totally different way. Using a new keyword would be far better than overloading : in this way

12 Likes

Can you outline (again if I missed it) why extend is too common but given is rare enough? What’s the threshold (in say of scala files with a particular identifier) ? Was this measured on some corpus?

I find extend or extension for making extension methods far more straight forward and important enough to merit a keyword IMO. Reusing keywords was part of what made implicit confusing (conversions vs the static-dependency injections).

4 Likes

Both extend and given are common, given maybe more than extend. So both are a problem, as would be any other common name we take as a keyword. That said, the disruption is cumulative. So taking away two common words is roughly twice as bad as taking away one.

1 Like

I think there is a misunderstanding about precedence. When I write

given f: TC { ... }

the precedence of : is (as always) less than the precedence of {, not greater as you state. The crucial point to convey is that f: is a label for the whole definition that follows. It’s not that f has just type TC but that f stands for the whole anonymous class TC{...} including any type refinements it might contain. So the relationship with refinement types is fully intentional.

That’s also a problem with the alias syntax proposed by @smarter and @julienrf. If I would write

given f: TC = new TC { ... }

then f would not get any type refinements in the right hand side.

1 Like

I forgot to reply about that part:

Indeed, the fact that we can’t anymore have abstract given definitions is an important limitation to me. I guess the problem is that it wouldn’t play well with the given instance definition syntax. Consider for example the following definition:

given foo: Foo

Is it an abstract definition, or an instance definition (whose members are missing and should cause a compilation error).

We wouldn’t have this limitation if we had to put an = between the lhs and the rhs of a given definition:

given foo: Foo // abstract given definition
given foo: Foo = ... // concrete given definition that requires a right-hand side

Also, one comment about the fact that we can define anonymous instances: we probably judge this capability from the eyes of programmers that are used to always having to write names for implicit instances, and probably see here an improvement because we’d have less things to write, but we lack experience with this feature to really estimate its cost. For example, I often find it useful to be able to refer to an implicit value from its term identifier. Typically, when an implicit value is not found and I need to understand the reason why, the ability to explicitly refer to one specific implicit value lets me see the problem if that value were picked. (I’m not sure I am clear…!)

Error reporting might worsen as well: how do we refer to an anonymous given definition in an error message? Here is an example. The nice thing with the “named” version is that the code produced by the error message is actual Scala code that can be copy-pasted.

5 Likes

I am not sure to follow. Why would we want to have a type refinement in the rhs? If we want to define a given instance for a type refinement of TC we would write the following, right?

given f: TC { type A = Int } = new TC { type A = Int; ... }

Precisely. The problem is the duplication this entails.

2 Likes

Would using extends instead of : for given instances help ?

given intOrd extends Ord[Int] {
  def compare(x: Int, y: Int) =
    if (x < y) -1 else if (x > y) +1 else 0
}

In particular I think this makes it much more clear that the thing after the extends is a constructor which can take arguments and not just a type.

7 Likes

That said, the disruption is cumulative. So taking away two common words is roughly twice as bad as taking away one.

But the motivation says clearly we want to move from mechanism to intent; I believe that’s only possible if we distinguish also linguistically between the different forms, otherwise the critique stated in various posts holds that we’re just replacing implicit everywhere with given everywhere.

Could one not run a crawler across, say community build, and see what the actual collisions would be?

4 Likes

Here my summary - mainly a selection of what others have already said:

positive:

  • anonymous parameters; this is very useful
  • multiple given lists are good
  • requirement to qualify given parameter when explicitly passed is very good
  • import *.given is good
  • extension methods, calling like a function, e.g. max(arg), very good, I always wished we had that

negative:

  • unclear when instantiation happens, e.g. what soronpo said: given global: ExecutionContext = ... (because val, lazy val and def are gone)
  • what lihaoyi said, there is a conflict/ambiguity with refinement types in given intOrd: Ord[Int] { def compare(x: Int, y: Int) = ??? }

open:

  • there’s an open question by katrix whether we get less contrived way to make sure there are priorities between implicits

I have another question for the extension methods ‘function call syntax’; what if there are arguments, like

def (i: Int) absDif (j: Int): Int = math.abs(i - j)

I know I can invoke as 33 absDif 44, but how would the function call syntax work here – is it absDif(33, 44) or absDif(33)(44) or simply not defined? From my use cases, putting them into one parameter list would probably be the most useful, e.g. atan(dy, dx), randomRange(lo, hi) etc.

4 Likes

Aside from the glaring issues with syntax (the given syntax in general is completely out of place from the rest of the syntax in Scala as others have commented) I would actually like to see some real world non trivial examples (i.e. something equivalent to cats) on the usages of given because

Seems like a very serious limitation, I definitely know that I wouldn’t be able to use given because there is a big difference between something being initialized val/lazy val/def (its also completely out of the place in the Scala language as has been commented elsewhere, everywhere else in Scala when a variable is initialized either implicitly or explicitly you can control how its initialized).

I honestly get the pursuasive feeling that this is being shoved into the language last minute without even properly addressing considerations or seeing if this is going to hold up in the real world. Typically such features would have a proper RFC/hidden behind a flag /plugin in other languages to see if the feature actually justifies itself and that can only be seen when its used in non trivial ways.

2 Likes

I think this syntax works only for static instances, but not for instances that take type parameters or given parameters.

What do you mean by “work”? You could use it with parameterized instances too, just replace : by extends:

given bar[T](given x: T) extends Bar[T](x) {
  def ...
}
3 Likes

I meant that is becomes very different from what we are used to (so far, only object definitions can use extends).

But actually, your example reads nicely to me.

Classes can also extends, and incidentally this is what a parameterized given instance desugars into :).

2 Likes

Great, we’re almost back to implicit classes… :roll_eyes:
Again, can someone please clarify what was wrong with implicit classes other than the word implicit?

4 Likes

I don’t think that’s true. In the new syntax it’s

given ops: extension(x: T) {... }
given extension(x: T) { ... }

That makes it clear that (1) it’s a given instance, and (2) it defines extension methods. Both aspects are important. I have the impression that people here would argue that it should not be a given instance. But then the whole semantic construction falls apart. You could not do typeclasses anymore the way we do them. This was covered in detail two years ago when extension methods were invented.

3 Likes

This shouldn’t stop it from being in the proposal proper though. The entire point of these documents is to be a one stop shop so people do not need to read discussions from years ago.

Arguably the most important part of these documents is the detailed discussion of various alternatives, with good reasons why they were not chosen. That is what makes a “design doc” different from a “reference doc”. I do not see that in this set of docs, which is probably why these “redundant” questions keep coming up

6 Likes

Basically none of these require an invasive keyword change though. Except for extension methods and imports, which modify member/scope lookup, the rest can be accomplished by minor tweaks to the existing implicit syntax to make some keywords/identifiers optional. Extension methods and imports would also look fine if the lookup semantics were applied to the current implicit syntax (s/given/implicit/)

1 Like