Updated Proposal: Revisiting Implicits

To get a plain given val:

val x = whatever
given T = x

The given in this case translates to a forwarder def since the right hand side is a simple pure reference. So you have defined one value plus one forwarder, which will be inlined by the JVM. Generally, the rules have been designed so that the straightforward strategy to define your value normally and then add a given works as expected. There’s one exception to this that @AMatveev brought up, but I believe most people would classify it as an abuse of implicits anyway. That’s when you want to construct a given over a mutable variable. You have to do it like this:

var x: T = ...
given [Dummy]: T = x

Without the [Dummy] parameter, you’d get a lazy val which would not track future updates to the variable. But, I think a system that constructs givens that change according to variable mutations is dubious anyway. So I don’t mind that getting this right requires some more advanced knowledge.

1 Like

People trust that their Haskell compiler will do the reasonable thing without having to worry about details. We have to free ourselves from the old thought patterns that are too close to mechanism.

This, 1000x. :clap:

2 Likes

Is it a dead simple? I just do not understand. I am sorry.

That’s not clear from the docs, which give explicit materialization semantics.

object equivalent semantics are detailed under Alias Givens:

The first time global is accessed, a new ForkJoinPool is created, which is then returned for this and all subsequent accesses to global .

def equivalent semantics are detailed under Given Instance Initialization:

If a given definition has type parameters or a given clause, a fresh instance is created for each reference.

So, from my perspective, including this explicitly would change nothing, cost the user very little, and reduce confusion considerably.

It is scala approach:

It is kotlin approach:

lateinit var subject: TestSubject

Which approach is user oriented?

There are many way to provide users clear means to solve their tasks. I am sure the punishment is the worst way. I have never seen that it works well. The end result always has been ironic. I hope I have mistaken in this case.

While you’re right that Unpleasant Design is generally a bad idea, this feels more like not being willing to expend additional effort to support something that’s very much a niche usage

You’re certainly free to use implicits to introduce additional race conditions and global mutable state into your program, but there are so many other things that need attention (see: this entire thread :wink:) that support for this being low priority is entirely unsurprising.

3 Likes

People trust that their Haskell compiler will do the reasonable thing without having to worry about details.

GHC is also finicky and notoriously hard to optimize for, partly because it’s trying to be too smart. Scala is an imperative language that generally produces straight-forward JVM bytecode, so much that its optimizer is turned off by default and notoriously underused! Everywhere a ‘costly’ construct is used, it is apparent, harmless structural types are even hidden behind a flag, and it’s pretty easy to produce very fast, very imperative, very low level Scala (which is dark black magic in Haskell!). IMHO Scala’s control philosophy is completely opposite to Haskell’s and that’s a virtue, not a problem.

3 Likes

Haskell and Rust seem to do fine without the ability to control the initialization order. I wouldn’t really know why you need to think about that too often. For me, most of the time it doesn’t matter at all, and when it does it seems you can still control it though aliases.

I think one of the biggest problems with implicits is that they are too similar to ‘normal’ constructs, which has led to abuse of using them. This proposal does not magically solve that, but at least in moves more in the direct of making implicit something separate.

edit
That said, if people really want fine-grained control, it could maybe also work with some “add-on modifier”, where you get the default if you do not provide any extra information, but still have the ability to give an extra modifier to change the initialization. I think that would still be better than falling back on the current scheme because the simple case stays simple, but the more advanced case is also possible.

It would be true if new way were lazy by default. I had thought so at the first. But it is too simple and greedy for that. It leads the more rare case is the more confusing it will be. It is wrong at all. The more rare case is the more readable it must be.

IIUC it is the main conflict of interest. I prefer to have add-on modifiers.

But they want that most people just think in different way. So they forbidden modifiers because they think it is good.

Maybe it is not so. But why is there no one words about modifiers.

the aim of making it different must not lead making it confusing.

1 Like

Honestly, it never occurred to me that choosing between implicit val, implicit def, implicit lazy val etc is a problem. It is just as natural as choosing between plain val, def, lazy val, etc

OTOH the implicit conversions in Scala 2 work in a weird way (I’ve seen many Scala puzzlers involving implicit conversions) and their syntax is too close to implicit instances. Removing the def from implicit instances and conversions wouldn’t help at all with the confusion. What removed the confusion in Dotty is that implicit conversions now require implementing new special type scala.Conversion.

Haskell has lazy everything. Rust has eager everything. In Scala you have a choice:

  • eagerness (plain val) is efficient, but generally thread-unsafe. However in 90%+ cases it’s easy to manually verify that plain vals are correct (e.g. IntelliJ shows “suspicious forward references”, Scala compiler has -Xcheckinit flag, etc)
  • laziness has performance impact, so lazy vals are avoided wherever it makes sense. I’ve looked quickly at cats’ implicit instances here cats/core/src/main/scala/cats/instances at main · typelevel/cats · GitHub and there are either implicit vals or implicit defs, I haven’t seen any implicit lazy vals. Zero overhead laziness can be achieved wholesale on a global singleton level (i.e. for all members of a such singleton) as JVM ensures that class loading is thread safe.

Also for macro-derived parameterless implicit values val or lazy val often doesn’t make sense. I’ve shown that previously some time ago, but look again at scalatest/scalactic/src/main/scala/org/scalactic/source/Position.scala at 3.0.x · scalatest/scalatest · GitHub :

implicit def here: Position = macro PositionMacro.genPosition

How to achieve that with givens if a parameterless instance is inferred to be a lazy val instead of a def? It could be achieved by adding a superfluous (and maybe erased) parameter to turn a lazy val into a def. Isn’t that a heavy abstraction leak? Dotty is supposed to make writing macros straightforward.

6 Likes

Haskell is not Scala, it has completely different evaluation semantics (i.e. Haskell is completely lazy by default). For the same reason given is also incosistent with the rest of the language because everywhere else we do have lazy val/val/def. If Scala didn’t have these definitions than I would agree with you, but then again it wouldn’t be Scala…

3 Likes

Yes, but we all agree that it is a terrible mistake to mix complicated evaluation semantics with implicits. It’s a downside of the old system that it makes these abuses look simple and natural.

To appreciate the new given semantics, one has to change the viewpoint. Previously, one could do everything Scala provided and simply slap implicit on it to get implicit evaluation. That’s mechanism oriented. It gives a large space of possibilities, most of which are really bad ideas.

Now, the viewpoint is that you define your system including evaluation semantics explicitly. Once that’s done, at some key points you lift definitions to be givens. That lifting should never play cute evaluation tricks, just do the simplest thing that works. That simplest thing is: a def if there are parameters, a lazy val if there are none and an optimization to a simple forwarder if the lifted thing is simple and pure. That gives you all the possibilities and it does the right thing without you even having to think about it.

So, in summary, Scala is not Haskell, but the lifting of things to be givens should just do the right thing, without getting mixed up in Scala’s rich choices of evaluation semantics.

6 Likes

Actually I don’t think we all agree about this otherwise we wouldn’t be having this discussion. Furthermore in terms of the “concerns about implicits”, evaluation semantics is a complaint that I have almost never heard (most common complaints being about implicit conversions or terrible diagnostic error messages from the compiler about where implicits are coming from). If you are concerned about such evaluation semantics you are basically making the argument to code in Haskell.

The point here is that making everything lazy val or def is not always the right evaluation semantic. As pointed out before, the default evaluation semantic for cat’s is actually val and this is for performance reasons. The base case of putting coherent implicits is a val inside an object which is guaranteed to be fast and safe.

1 Like

So, whenever you to want a specific performance optimization, write it down explicitly and then use a given over that. I would like to see a use case where this technique is not sufficient,

2 Likes

That “whenever” in practice means almost always. I’ve seen many implicit val instances but no implicit lazy val instance in typelevel/cats’ source code. That was a quick glance, though.

When I make parameterless implicit instances I also use implicit val everywhere. implicit lazy val is pretty rare in the code I’m working on on daily basis.

Also inferring val after given is somewhat inconsequential.

// here putting `val` after `given` is legal makes sense
class Givens(a: Int)(given b: String) {
  println(s"$a $b")
}

@main def main(): Unit = {
  given String = "abc"
  val g = Givens(5)
  println(g.b) // compilation error, there was no `val` inferred after `given`
}
5 Likes

The “old” viewpoint is not the reason why implicits are confusing. Implicits are just a mechanism / operation. The new viewpoint doesn’t do much to change that; it still allows for all of these possibilities, it just uses a different syntax.

implicit is not a definition. It doesn’t define a unique “thing” in the language, nor does it modifies the definition of something. Every modifier in the language – abstract, override, private, var (“nonfinal”), final, lazy, sealed, @volatilemust be part of a definition; if they were to be applied separately after the definition, they would change the semantics and behavior of the defined “thing”. The same is not true for implicit, as it’s just an operation on a “thing”, it doesn’t change its behavior.

The exception of course is with implicit parameters, in which case being implicit is part of the definition and cannot be applied separately.

1 Like

… I just had a look. All implicit vals in Cats have simple and pure right hand sides, so the given is compiled to a simple forwarder which will be inlined in the JVM. This is a good demonstration that given will indeed do the right thing without you having to think about it.

Anyway, Discourse tells me I am posting too much, which is probably true. So I’ll take a break for a while.

1 Like

How does Dotty determine purity? Is there new rule for missing keyword inference?

A definitely non simple example cats/core/src/main/scala/cats/instances/option.scala at main · typelevel/cats · GitHub :

  implicit val catsStdInstancesForOption: Traverse[Option]
    with MonadError[Option, Unit]
    with Alternative[Option]
    with CommutativeMonad[Option]
    with CoflatMap[Option]
    with Align[Option] =
    new Traverse[Option]
      with MonadError[Option, Unit]
      with Alternative[Option]
      with CommutativeMonad[Option]
      with CoflatMap[Option]
      with Align[Option] {

      def empty[A]: Option[A] = None

      def combineK[A](x: Option[A], y: Option[A]): Option[A] = x.orElse(y)

      def pure[A](x: A): Option[A] = Some(x)

      override def map[A, B](fa: Option[A])(f: A => B): Option[B] =
        fa.map(f)

      (...)
      }

How can anyone (i.e. humans) tell that right hand side of catsStdInstancesForOption is pure?

2 Likes

This is my favorite #metoo quote.

3 Likes