Updated Proposal: Revisiting Implicits

Scala Improvement Process
141

Is it a dead simple? I just do not understand. I am sorry.

142

That’s not clear from the docs, which give explicit materialization semantics.

object equivalent semantics are detailed under Alias Givens:

The first time global is accessed, a new ForkJoinPool is created, which is then returned for this and all subsequent accesses to global .

def equivalent semantics are detailed under Given Instance Initialization:

If a given definition has type parameters or a given clause, a fresh instance is created for each reference.

So, from my perspective, including this explicitly would change nothing, cost the user very little, and reduce confusion considerably.

143

It is scala approach:

It is kotlin approach:

lateinit var subject: TestSubject

Which approach is user oriented?

There are many way to provide users clear means to solve their tasks. I am sure the punishment is the worst way. I have never seen that it works well. The end result always has been ironic. I hope I have mistaken in this case.

144

While you’re right that Unpleasant Design is generally a bad idea, this feels more like not being willing to expend additional effort to support something that’s very much a niche usage

You’re certainly free to use implicits to introduce additional race conditions and global mutable state into your program, but there are so many other things that need attention (see: this entire thread :wink:) that support for this being low priority is entirely unsurprising.

3 Likes
145

People trust that their Haskell compiler will do the reasonable thing without having to worry about details.

GHC is also finicky and notoriously hard to optimize for, partly because it’s trying to be too smart. Scala is an imperative language that generally produces straight-forward JVM bytecode, so much that its optimizer is turned off by default and notoriously underused! Everywhere a ‘costly’ construct is used, it is apparent, harmless structural types are even hidden behind a flag, and it’s pretty easy to produce very fast, very imperative, very low level Scala (which is dark black magic in Haskell!). IMHO Scala’s control philosophy is completely opposite to Haskell’s and that’s a virtue, not a problem.

3 Likes
146

Haskell and Rust seem to do fine without the ability to control the initialization order. I wouldn’t really know why you need to think about that too often. For me, most of the time it doesn’t matter at all, and when it does it seems you can still control it though aliases.

I think one of the biggest problems with implicits is that they are too similar to ‘normal’ constructs, which has led to abuse of using them. This proposal does not magically solve that, but at least in moves more in the direct of making implicit something separate.

edit
That said, if people really want fine-grained control, it could maybe also work with some “add-on modifier”, where you get the default if you do not provide any extra information, but still have the ability to give an extra modifier to change the initialization. I think that would still be better than falling back on the current scheme because the simple case stays simple, but the more advanced case is also possible.

147

It would be true if new way were lazy by default. I had thought so at the first. But it is too simple and greedy for that. It leads the more rare case is the more confusing it will be. It is wrong at all. The more rare case is the more readable it must be.

IIUC it is the main conflict of interest. I prefer to have add-on modifiers.

But they want that most people just think in different way. So they forbidden modifiers because they think it is good.

Maybe it is not so. But why is there no one words about modifiers.

the aim of making it different must not lead making it confusing.

1 Like
148

Honestly, it never occurred to me that choosing between implicit val, implicit def, implicit lazy val etc is a problem. It is just as natural as choosing between plain val, def, lazy val, etc

OTOH the implicit conversions in Scala 2 work in a weird way (I’ve seen many Scala puzzlers involving implicit conversions) and their syntax is too close to implicit instances. Removing the def from implicit instances and conversions wouldn’t help at all with the confusion. What removed the confusion in Dotty is that implicit conversions now require implementing new special type scala.Conversion.

Haskell has lazy everything. Rust has eager everything. In Scala you have a choice:

  • eagerness (plain val) is efficient, but generally thread-unsafe. However in 90%+ cases it’s easy to manually verify that plain vals are correct (e.g. IntelliJ shows “suspicious forward references”, Scala compiler has -Xcheckinit flag, etc)
  • laziness has performance impact, so lazy vals are avoided wherever it makes sense. I’ve looked quickly at cats’ implicit instances here cats/core/src/main/scala/cats/instances at main · typelevel/cats · GitHub and there are either implicit vals or implicit defs, I haven’t seen any implicit lazy vals. Zero overhead laziness can be achieved wholesale on a global singleton level (i.e. for all members of a such singleton) as JVM ensures that class loading is thread safe.

Also for macro-derived parameterless implicit values val or lazy val often doesn’t make sense. I’ve shown that previously some time ago, but look again at scalatest/scalactic/src/main/scala/org/scalactic/source/Position.scala at 3.0.x · scalatest/scalatest · GitHub :

implicit def here: Position = macro PositionMacro.genPosition

How to achieve that with givens if a parameterless instance is inferred to be a lazy val instead of a def? It could be achieved by adding a superfluous (and maybe erased) parameter to turn a lazy val into a def. Isn’t that a heavy abstraction leak? Dotty is supposed to make writing macros straightforward.

6 Likes
149

Haskell is not Scala, it has completely different evaluation semantics (i.e. Haskell is completely lazy by default). For the same reason given is also incosistent with the rest of the language because everywhere else we do have lazy val/val/def. If Scala didn’t have these definitions than I would agree with you, but then again it wouldn’t be Scala…

3 Likes
150

Yes, but we all agree that it is a terrible mistake to mix complicated evaluation semantics with implicits. It’s a downside of the old system that it makes these abuses look simple and natural.

To appreciate the new given semantics, one has to change the viewpoint. Previously, one could do everything Scala provided and simply slap implicit on it to get implicit evaluation. That’s mechanism oriented. It gives a large space of possibilities, most of which are really bad ideas.

Now, the viewpoint is that you define your system including evaluation semantics explicitly. Once that’s done, at some key points you lift definitions to be givens. That lifting should never play cute evaluation tricks, just do the simplest thing that works. That simplest thing is: a def if there are parameters, a lazy val if there are none and an optimization to a simple forwarder if the lifted thing is simple and pure. That gives you all the possibilities and it does the right thing without you even having to think about it.

So, in summary, Scala is not Haskell, but the lifting of things to be givens should just do the right thing, without getting mixed up in Scala’s rich choices of evaluation semantics.

6 Likes
151

Actually I don’t think we all agree about this otherwise we wouldn’t be having this discussion. Furthermore in terms of the “concerns about implicits”, evaluation semantics is a complaint that I have almost never heard (most common complaints being about implicit conversions or terrible diagnostic error messages from the compiler about where implicits are coming from). If you are concerned about such evaluation semantics you are basically making the argument to code in Haskell.

The point here is that making everything lazy val or def is not always the right evaluation semantic. As pointed out before, the default evaluation semantic for cat’s is actually val and this is for performance reasons. The base case of putting coherent implicits is a val inside an object which is guaranteed to be fast and safe.

1 Like
152

So, whenever you to want a specific performance optimization, write it down explicitly and then use a given over that. I would like to see a use case where this technique is not sufficient,

2 Likes
153

That “whenever” in practice means almost always. I’ve seen many implicit val instances but no implicit lazy val instance in typelevel/cats’ source code. That was a quick glance, though.

When I make parameterless implicit instances I also use implicit val everywhere. implicit lazy val is pretty rare in the code I’m working on on daily basis.

Also inferring val after given is somewhat inconsequential.

// here putting `val` after `given` is legal makes sense
class Givens(a: Int)(given b: String) {
  println(s"$a $b")
}

@main def main(): Unit = {
  given String = "abc"
  val g = Givens(5)
  println(g.b) // compilation error, there was no `val` inferred after `given`
}
5 Likes
154

The “old” viewpoint is not the reason why implicits are confusing. Implicits are just a mechanism / operation. The new viewpoint doesn’t do much to change that; it still allows for all of these possibilities, it just uses a different syntax.

implicit is not a definition. It doesn’t define a unique “thing” in the language, nor does it modifies the definition of something. Every modifier in the language – abstract, override, private, var (“nonfinal”), final, lazy, sealed, @volatile – must be part of a definition; if they were to be applied separately after the definition, they would change the semantics and behavior of the defined “thing”. The same is not true for implicit, as it’s just an operation on a “thing”, it doesn’t change its behavior.

The exception of course is with implicit parameters, in which case being implicit is part of the definition and cannot be applied separately.

1 Like
155

… I just had a look. All implicit vals in Cats have simple and pure right hand sides, so the given is compiled to a simple forwarder which will be inlined in the JVM. This is a good demonstration that given will indeed do the right thing without you having to think about it.

156

Anyway, Discourse tells me I am posting too much, which is probably true. So I’ll take a break for a while.

1 Like
157

How does Dotty determine purity? Is there new rule for missing keyword inference?

A definitely non simple example cats/core/src/main/scala/cats/instances/option.scala at main · typelevel/cats · GitHub :

  implicit val catsStdInstancesForOption: Traverse[Option]
    with MonadError[Option, Unit]
    with Alternative[Option]
    with CommutativeMonad[Option]
    with CoflatMap[Option]
    with Align[Option] =
    new Traverse[Option]
      with MonadError[Option, Unit]
      with Alternative[Option]
      with CommutativeMonad[Option]
      with CoflatMap[Option]
      with Align[Option] {

      def empty[A]: Option[A] = None

      def combineK[A](x: Option[A], y: Option[A]): Option[A] = x.orElse(y)

      def pure[A](x: A): Option[A] = Some(x)

      override def map[A, B](fa: Option[A])(f: A => B): Option[B] =
        fa.map(f)

      (...)
      }

How can anyone (i.e. humans) tell that right hand side of catsStdInstancesForOption is pure?

2 Likes
158

This is my favorite #metoo quote.

3 Likes
159

The overview (in criticism 4) states:

The syntax of implicit parameters also has shortcomings. It starts with the position of implicit as a pseudo-modifier that applies to a whole parameter section instead of a single parameter. This represents an irregular case with respect to the rest of Scala’s syntax.

If this is the motivation, one infers that this is fixed. But apparently it is not. Given also applies to the entire parameter block, not to individual parameters.

There are no illustrations of this one way or the other in the examples.

The grammar does state that this is impossible (by not admitting f(x, given y)), but reading grammars is not the easiest (though it is the most precise) way to figure out what syntax is legal.

Even the grammar doesn’t say whether you can give only parts of an argument list. Under Given Clauses, in section Multiple Given Clauses, this example is given:

f(global)(given ctx)(given sym, kind)

but it doesn’t say whether f(global)(given ctx)(given sym) with kind inferred is okay or not.

(Also, I believe that f resolving to f(global) is a mistake? If the definition was f(u: Universe = global) that would be okay, but as is there seems no way to resolve the first parameter block as it isn’t given.)

160

Good point, is there any argument against this, any reason to keep nested givens? If they don’t add anything I too think they should go.