Updated Proposal: Revisiting Implicits

Scala Improvement Process
46

I am also in favor of keeping the implicit keyword. The given keyword looks nice in parameters, but it’s strictly less informative. Probably harder to google, too. All previous Scala material use “implicit” and it’s already known outside the community as a distinguishing feature of Scala. Also, other languages like Haskell, Agda and Coq have adopted similar terminology for arguments that can be passed implicitly (though their own versions of the concept work differently).

One problem was that people confused implicit parameters with implicit conversions, lumping them together. I think this is already addressed by using the Conversion type class, and the change of keyword achieves nothing more here. It does makes more sense and sounds more precise to talk about an implicit Conversion[A,B] than to talk about a given Conversion[A,B]. The “given” conversion still is fundamentally implicit, as it’s applied implicitly. C++ and C# also have such implicit conversions. Changing the name of the concept to given does not seem to achieve anything useful.

Another invoked reason was that “implicit” feels wrong in definitions and arguments because… these are not implicit themselves; in fact they are explicitly written in the source. I never really bought this argument. It’s sufficient to understand that defining an implicit def is like defining a def that will be used implicitly, and that passing an (implicit arg) is like passing an argument, but for an implicit parameter (which will, in fact, be used implicitly in the body of the function). As long as there is a notion of “implicit parameter”, it’s fine to pass “implicit arguments” for them, even when doing so explicitly. I do not think that using a less specific keyword actively improves things here, either.

6 Likes
47

implicit classes only accept a single parameter which is usually non-implicit, if they accepted multiple implicit parameters I think they would be equivalent to given instances indeed (actually they would be more flexible since an implicit class can extend multiple traits, whereas this isn’t possible with given instances which is another potential issue with them).

48

My bad, it’s possible, but you have to use , instead of with (this is also supported for extends clauses of classes and traits in Dotty, but I’m not sure if it’s documented anywhere):

trait A
trait B
given A, B {
  def foo: Int = 1
}
49

It seems it is a bad idea for me. I will definitely have a lack of implicit methods.
I usually use implicits for context injection. So a usual code looks something like:

class Dao{
   var _session:Session = _
   implicit def session = _session
}
object Index{
  def apply()(implicit session:Session):Unit = ??? 
}
class SomeDao extend Dao{
 lazy val index = Index("someField")
 //it just will be broken index in scala2 
 //IIUC it will be broken dao in scala3
 val index = Index("someField2")
 ...
}

I really do not like that some insignificant error in constructor will be able to broke all class.

50

I hate to be that guy, but I find myself on the ‘nay’ side but without any compelling alternatives to suggest at this time. :confused:

51

Overall, this looks like a solid and comprehensive set of improvements to the language, elevating language past features like implicits from the level of mechanics to the level of intent, and simplifying teaching use case-driven contextual features (notably: type classes and extension methods). :clap:

My only very minor concerns revolve around reducing the number of ways to do things, and simplifying the new language features as much as possible.

Some random ideas here:

Remove Named Givens

Currently, there are two ways to define a given:

given Ord[Int]

and

given intOrd: Ord[Int]

This bifurcation makes the feature more complex and forces users to choose between two different styles; indeed, it also leaks the mechanical, implementation detail that givens are implemented by synthesizing named values (or methods).

The named given is not needed, because it can be reconstructed separately from other orthogonal language features:

given Ord[Int] { ... }

val intOrd = summon[Ord[Int]]

This factoring avoids the conflation of creation of a given with the naming and storing of a reference to that given, and also takes a more opinionated stance that the canonical way to define given instances is without names, which, thanks to summon and extension methods, are often useless and / or boilerplate.

Remove Given Extensions

As others have pointed out, there are many ways to define extension methods. The syntax for defining extension methods is straightforward and an obvious generalization of the syntax for def methods

However, given instances for extension methods introduce quite a different syntax for doing the same thing:

given stringOps: extension (xs: Seq[String]) { ... }

This new keyword and new language feature is unnecessary, because we can express the same extensions in two other ways: by defining method extensions using the new syntax, or by using given instances on AnyRef (or, indeed, by using implicit conversions!):

given AnyRef { ... }

The only additional cost for either technique is repeating the “this” parameter for every extension; but sometimes this is desirable, because for polymorphic data types, some extension methods are specialized (e.g. .flatten defined only on Future[Future[A]]); in general extension methods on polymorphic data types will not have a uniform target type.

Remove Implicit Conversions

This suggestion will be so flamed I won’t spend too time much on it, but the idea is that explicit toX or asX conversion methods can be added as extension methods, so implicit conversions are not actually fundamental or necessary in any way; and in my experience, the explicitness of extension method conversions helps tremendously with readability and tooling (e.g. toJavaCollection instead of an automatic conversion).

Removing implicit conversions would also reduce the number of ways to do extension methods.

Other Random Notes

  • I’d prefer the name Equal instead of Eql
  • Big :+1: on given imports
4 Likes
52

What about conversions between scala.Int and java.lang.Integer for example? There are lots of such implicit conversions in scala.Predef.

2 Likes
53

Remove Named Givens

Explicitly naming things is an absolute must for maintaining binary compatibility. I expect library authors to never use anonymous givens or to learn the error of their ways very soon. If anything, anonymous givens are a better target for axing, they’re nice, application-code-only, syntax sugar, but that sugar forces a creation of new language construct import {given Type} solely to accommodate for it.

1 Like
54

What about conversions between scala.Int and java.lang.Integer for example? There are lots of such implicit conversions in scala.Predef .

I don’t personally value these. I’d rather toJavaInteger if I want to accept and explicitly document the potential implications for converting one type to another.

1 Like
55

Why isn’t it sufficient to generate a name based on the package (and path) name and the type(s) for which the instance is defined?

e.g.:

package foo

object Bar {
  derive Ord[Int]
}

…gets a name like `foo.bar.Ord[Int]`

If you move the given instance around, then it will already break binary compatibility; and if you change it from Ord[Int] to Ord[String], it’s not the same instance; but within a given scope, you can have at most one given instance defined, which seems to get you stability.

Or am I missing something?

56

Why isn’t it sufficient to generate a name based on the package (and path) name and the type(s) for which the instance is defined?

There may be multiple instances for the same type with different priorities, clearly just these two names aren’t sufficient.
Currently the names are generated from type + type parameter names as follows, AFAIK:

given [T] Ord[Bar[T]]
// given_Ord_of_Bar_of_T

If the name of the type parameter changes to A, the name will be different. Same I assume if the given was changed to implement more types, e.g. given [T] Ord[Bar[T]] & Show[Bar[T]] – instances in cats tend to implement as many typeclasses as possible with one value, so if adding an additional instance broke bincompat, that wouldn’t be nice. Anyway, I doubt an optimal naming strategy can be derived that would be unambiguous and allow evolution without breaking compatibility, and allowing given names is an acknowledgment of that.

Another argument for axing anonymous instances is that names are also required for explicit passing, making implicit search the only way to get a value of a given is very fragile – the search can be spoiled by surrounding implicits AND it requires the given itself to enter implicit scope. To be frank, I wouldn’t want to remove anonymous instances altogether, but if the choice was between anonymous-only and named-only, I think named-only is much better in Scala.

4 Likes
57

For what it’s worth, it looks like the current release candidate may have some trouble generating unique names.

For example, this error just hit my console:

[error] 165 |  given FunctorLaws.Givens[Validated, Int, String, Long] = FunctorLaws.Givens[Validated, Int, String, Long]
[error]     |                                                                             ^
[error]     |cannot merge
[error]     |  method given_Arbitrary_Function in object ValidatedTest of type => sql2json.testing.Arbitrary[String => Long]  and
[error]     |  method given_Arbitrary_Function in object ValidatedTest of type => sql2json.testing.Arbitrary[Int => String]
[error]     |they are both defined in object ValidatedTest but have matching signatures
[error]     |  => sql2json.testing.Arbitrary[String => Long] and
[error]     |  => sql2json.testing.Arbitrary[Int => String]
[error]     |as members of object ValidatedTest

That appears to be caused by these two anonymous given declarations:

object ValidatedTest
  given Arbitrary[Int => String] = summon[Arbitrary[Int => String]] 
  given Arbitrary[String => Long] = summon[Arbitrary[String => Long]]

Giving one of them a name clears up the compile error.

1 Like
58

Here is my perspective on why implicits are confusing for new comers, and how they could be improved, for both new comers and veterans.

Nowadays, the implicit keyword is associated with multiple distinct mechanisms:

  1. Extension methods (implicit class).
  2. Implicit conversations.
  3. Implicit clauses.

The ambiguity between these mechanisms is the main culprit in my opinion, and the solution is quite straightforward - create a distinct keyword / syntax for each mechanism.

However, there is another ambiguity with the syntax in regards to the last mechanism – implicit clauses – which is what I believe most are introduced with in regards of implicits, more or less in this way: one can define an instance such that it will later be implicitly passed as an argument to a method in the same scope.

The problem is that the same keyword – implicit – is used to declare both the semantics of the parameter and the instance, even though the semantics are different. My proposal would be to use similar yet distinct keywords for each; for instance, implied for parameters and imply for instances:

object Future {
  def apply[T](body: =>T)(implied executor: ExecutionContext): Future[T] = ...
}

object Foo {
  imply val global: ExecutionContext = new ForkJoinPool()
  new Future {
   ...
  }
}

Moving forward with this hypothetical suggestion, it becomes less clear why there is such a thing as “implicit import”:

object Ordering {
  imply object Int extends Ordering[Int] {
    def compare(x: Int, y: Int) = java.lang.Integer.compare(x, y)
  }
}

Having “imply” declarations without a clear consumer “implied clause” is confusing. It’s also not a “strong” semantic; it’s either a shortcut, which can be replaced with “imply import” (/ “given import”); or it’s a way for the author to suggest a way to use their code, which can be achieved by other means (docs, annotation, etc.).

At the end of the day, the user who imports these declarations should decide whether to use them implicitly, not the author. In my opinion, implicitly importing implied definitions should go away.

One last note about the given keyword - it’s a very bad candidate in my opinion. This will cause a great ambiguity in basically every documentation for functions with given clauses, because many docs – in all programming languages – refer to the parameters of a function as “given”.

3 Likes
59

The reason implicits are confusing has nothing to do with syntax or keywords, but:

(1) Some libraries use implicits heavily trying to create a new level of abstraction or hide complexity, but the underlying complexity keeps leaking through

(2) Such libraries are poorly documented with regard to what actually happens, trying maintain the illusion it is all simple

(3) Failures come in surprising ways, and it is hard to understand what is possible and what is not

(4) Compiler errors are hard to understand, frequently pointing to the wrong cause

4 Likes
60

That can’t happen in the same fully qualified scope. They’d still have unique names as per the above proposal.

I agree the names of the type parameter should be irrelevant, and that should be reflected in the generated name.

Nah, you can say given summon[Ord[Int]], for example`.

My view is that more than one implicit for the same type is a smell, and not to be attempted by mainstream Scala developers, only by advanced Scala developers who really need that power and know what they’re doing. “More than one implicit per type” is part of why implicits are so confusing and painful for many developers.

1 Like
61

It’s not quite that simple. Even if there’s only one implicit for that type, it’s helpful to debug situations where the type might be wrong at the callsite (misplaced parens, typos, etc) by manually passing the value you’d expect the compiler to grab. Generally the extra hint will produce more useful compilation errors.

It’s even more helpful when there isn’t a compilation error. I’ve been playing with Dotty, and this was exceptionally useful figuring out why it was hanging at runtime - turned out there was a missing given clause upstream.

Debugging this involved caching them locally and passing them explicitly to the method. The kicker was that using summon[A] would still hang, so I had to replace them with named references to get things working again. Then I could bisect the list of givens, narrowing down which would fail when converted to summon[A].

I’m not quite sure how I would have approached this if I couldn’t refer to them by name, but whatever it was would have been considerably more convoluted.

2 Likes
62

imply / implied is intriguing, but I would look at using these as direct replacements for given in the currently proposed syntax.

imply intOrd: Ord[Int] { ... }
imply ExecutionContext = ...
def max[T](x: T, y: T)(implied Ord[T]) = ...
max(x, y)(implied intOrd)

or maybe:

max(x, y)(imply intOrd)

What do others think?

8 Likes
63

verb/adjective combos have been discussed on github before, I personally find it the most intuitive of all the options I’ve seen here, pretty much regardless of the actual word (imply/provide/assume/give/etc) used.

Also the questionable

given (given Ctx)

example would look much more legible:

imply (implied Ctx)
provide (provided Ctx)

etc.

I wouldn’t even mind mixing them up such as

provide (given Ctx)
imply (given Ctx)

so long as the verb / adjective form is retained. Actually kinda like these two.

7 Likes
64

I don’t think the tenses work properly. The declaration site is static, old, past-tense, implied-like. In fact, it’s more like a noun: impliable. The argument-site seems current, fresh, present-tense: imply. The call site is definitely present-tense: imply.

Given works from mathematics, but a pair that makes the providers clearly distinct from the use sites (e.g. give/given) is, I think, substantially better than relying upon a single term.

2 Likes
65

I don’t like this one though, passive works fine/better here I think.

1 Like