Updated Proposal: Revisiting Implicits

I’d like to propose a variant that I think maximizes readability + regularity. It’s mostly a combination of what’s been discussed before, with one novel idea.The gist of it is this:

given [T] if Ord[T] for Ord[List[T]]  { ... }

given if (outer: Context) for Context = ...

def foo(given Context) = { ... }

More examples can be found here.

There are four notable points:

(1) Keyword for precedes the type (instead of “as” or “of”).

(2) Keyword if is used for conditional instances (instead of “with” or “given”).

(3) Same keyword given for instances and parameters.

(4) Parentheses around given parameters.

I’ll comment a bit on each point.

(1) The for variant translates better into real-world language. How to read it:

Example: given for Ord[Int] { }

Two alternatives that are mostly synonymous:

a) Using “given” as adjective, i.e. “a given instance”: Let there be defined a given instance for Ord of Int with the following implementation.

b) Using “given” as verb, i.e “x is given for y”: Let there be given for Ord of Int an instance with the following implementation.

(2) I think the idea has been that conditional instances and context parameter should have the same syntax. This is a legacy from the original implicit syntax. The new, high-level abstraction is conditional given instances. There is no notion of “parameters” here, and as such it can have a completely unique syntax. with does not have any connotations of “conditionality”, thus if.

How to read it:

Example: given [T] if Ord[T] for Ord[List[T]] { }

a) Let there be defined, for any type T, if there exists a given instance for Ord of T, a given instance for Ord of List of T with the following implementation.

b) Let there, for any type T, if there exists a given instance for Ord of T, be given for List of Ord of T an instance with the following implementation.

(roughly)

(3) I think there should be some kind of symmetry between context instances and context parameters. Using the same keyword is a simple way to achieve that. The alternative with has the following problems:

  • it’s a bit overloaded
  • it does not have any connotation of “implicitness”
  • it does not have any semantic relationship with “given”
  • used without parentheses, it has a syntax collision with new Foo with Bar
  • it requires additional syntax for context function types

(4) Using parentheses around context parameters avoids the “warts” of .given and spaces around :.

conversions can be seen as special cases of typeclasses

No, unfortunately this is not the case. For example, inline conversions are NOT typeclasses. They cannot be used as values of a typeclass because they are NOT valid at runtime! Currently dotty makes you jump through some hoops to define a macro conversion – you must first define a subclass of Conversion that will error at runtime, an error situation that’s forced by the new encoding – and only then you can define an inline conversion:

trait MyInlineConversion[-A, +B] extends Conversion[A, B] {
  def apply(a: A): B = throw new Error("""Tried to call a macro conversion
at runtime! This Conversion value is invalid at runtime, it must be applied
and inlined by the compiler only, you shouldn't summon it, sorry""")
}

trait DslExpr[+A]
given MyInlineConversion[A, DslExpr[A]] {
  inline def apply(expr: A): DslExpr[A] = {
    // analyze expr and produce new DslExpr...
    ...
  }
}

More so, the Conversion typeclass rules out two more types of implicit conversions:

  1. Path-dependent conversions such as implicit def x(a: A): a.Out are inexpressible with Conversion, there’s no syntax to express Conversion[(a: A), a.Out]

  2. Path-dependent conversions that summon more implicit parameters that depend on the input value, such as implicit def x(a: A)(implicit t: Get[a.T]): t.Out are inexpressible with Conversion, there’s no way to append more implicit argument lists to the def apply(a: A): B method of Conversion!

All of the above types of conversions are heavily used in mainstream Scala libraries, such as Akka, Sbt & Shapeless. I point the specific usages in the dotty issue

I do not believe that implicit conversions deserve to be gimped by completely ruling out at least two of their currently used forms and by making the third form - macro conversions - inconvenient to define and unsafe to use (a library user always risks to summon a dud Conversion object that will error at runtime if there are macro conversions in scope). As such I think we need to address the following issues:

  1. Create a syntax specifically for conversions, that will bring back path-dependency and would better express intent than the given Conversion definitions:
 conversion on (a: A) with (t: Get[a.T]): t.Out = ...
  1. Create a new type InlineConversion[A, B] - a super type of Conversion, that would be summonable ONLY in inline defs. That way, abstractions on top of summonable macros can be easily built, but at the same time the Conversion type will be unpolluted by inline conversions that are invalid at runtime.

It’s true that inline conversions require boilerplate. But they don’t need to expose safety holes since the base trait (MyInlineConversion in the example) can be sealed. On the other hand, Scala 2 does not have inline at all, so I don’t see a regression here.

Path dependent implicit conversions are indeed not supported. Maybe we can introduce a Conversion class that allows dependent typing at some point. Or maybe that does not work and path-dependent implicit conversions are dropped for good. I personally don’t lose sleep over this, since the use cases seem to be questionable designs anyway. I would probably object to inventing a special language feature for this; that would send exactly the wrong signal.

They must inherit from Conversion, because of that the user can summon them with innocent

def x(a: A) with (Conversion[A, B]): B = a

given MyInlineConversion[A, B] { inline def apply ... }
sealed trait MyInlineConversion[A,B] extends Conversion[A, B]

Where expression x(a) will blow up at runtime because x will get MyInlineConversion from implicit search and apply it at runtime. Even if you can omit a stub definition, you can’t omit a "legal" upcast of an inline conversion to a non-inline runtime Conversion value.

Err, Scala 2 has macros, which are widely used in the exact same way:

implicit def dsl[A](expr: A): DslExpr[A] = macro DslExpr.inspectExpr[A]

Hard disagree. With weaker conversions, these “questionable designs” will transform from one-liners into multiple lines of slow, imperative and opaque inline code, that would benefit no one. At least the inline/macro conversions issues must be addressed, I do not predict their usage would decline one bit.

I have expressed my concerns about the absence of abstract given definitions and my concerns about anonymous definitions, and I would like to insist one last time on those points.

In Scala 2, abstract implicits are occasionally defined by developers (here is one example from my codebase, here is another one from cats). The current syntax in Scala 2 is the following:

implicit def functor: Functor[F]

In Scala 3, the syntax will be:

def functor: Functor[F]
given as Functor[F] = functor

The new approach is a bit more verbose, but it also goes against the principle of “intent vs mechanism”: here, we have to understand and think in terms of the underlying mechanism because there is no language construct that matches our intent.

About anonymous instances, I agree that they are much more pleasant to define than the named ones, but we lack experience in using them. How are they referred to in error messages or by IDEs? I think we should address these questions before committing to supporting anonymous instances or we should make them experimental.

6 Likes

This scheme doesn’t work well with alias instances:

given ord as Ord[T] = otherOrd

I had raised the suggestion for extensions to only use the “Collective Extension” syntax (to simplify the language). Martin had countered that the collective extension syntax isn’t sufficient to implement type classes.

I have read about type classes in Scala 2 in the past, and have some use cases where they would probably have been useful, but I shied away from them because they looked too complicated to define/understand. I wasn’t convinced that a regular engineer would be able to understand the code, and perhaps even I wouldn’t understand the code after coming back to it six months later.

I wasn’t aware of Simulacrum, but from a language user perspective this is fairly close to what I would like type classes to be. I.e. something that is fairly concisely defined in the language where the construct is easily identified.

Hence, without understanding the exact mechanics of how it works, and I appreciate that the devil is in the detail, I think that I was hoping that typeclasses in Scala 3 might look something more like this:

trait SemiGroup[T] {
  @infix def combine (x: T)(y: T): T
}

typeclass trait Monoid[T] extends SemiGroup[T] {
  def unit: T
}

typeclass instance Monoid[String] {
  def combine (x: String)(y: String): String = x.concat(y)
  def unit: String = ""
}

typeclass instance Monoid[Int] {
  def combine (x: Int)(y: Int): Int = x + y
  def unit: Int = 0
}

extension on (xs: List[T]) {
  def sum[T: Monoid]: T = xs.foldLeft(Monoid[T].unit)(_ combine _)
}


---


trait Functor[F[_]] {
  def map[A, B](x: F[A])(f: A => B): F[B]
}

typeclass trait Monad[F[_]] extends Functor[F] {
  def flatMap[A, B](x: F[A => B])(f: F[A]): F[B]
  def map[A, B](x: F[A])(f: A => B): F[B] = x.flatMap(f `andThen` pure)

  def pure[A](x: A): F[A]
}

typeclass instance listMonad as Monad[List] {
  def flatMap[A, B](xs: List[A])(f: A => List[B]): List[B] =
    xs.flatMap(f)

  def pure[A](x: A): List[A] =
    List(x)
}

// ReaderMonad omitted, I didn't under the the =>> syntax.

Note, I’m not saying the the code above can work, but just to give an indication of what I was hoping to see. With the current proposed design, I think that I would still either avoid defining typeclasses, or add additional code comments to explain what is being done. Or perhaps wait for someone to port something like Simulacrum to Scala 3.

I think ultimately, I was hoping that “givens/implicits” are treated as low level language machinery that folks don’t generally need to understand or make use of, with cleaner high level abstractions sitting on top of them.

Parts of the Scala language are exceptionally nice to use, but it worries me that parts of the language (currently need to be used by library definitions/implementations) are complex enough that I can’t fully understand the code without investing considerable mental agility.

1 Like

I think that’s very close to how typeclasses effectively look right now:

trait SemiGroup[T] {
  def (x: T) combine (y: T): T
}

trait Monoid[T] extends SemiGroup[T] {
  def unit: T
}

given Monoid[String] {
  def (x: String) combine (y: String): String = x.concat(y)
  def unit: String = ""
}

given Monoid[Int] {
  def (x: Int) combine (y: Int): Int = x + y
  def unit: Int = 0
}

extension on [T](xs: List[T]) {
  def sum[T: Monoid]: T = xs.foldLeft(summon[Monoid[T]].unit)(_ combine _)
}

Forgive me if I’ve missed some recent syntax change.

2 Likes

Yes, I agree the structure is similar (but this doesn’t match the syntax that Martin pointed me to) but please consider someone who is relatively unfamiliar with the nuances of the language and who tries to read this code. They will probably google both “given” and “extension”. The help page for extensions should be pretty straight forward, but I anticipate that the help pages for given will be a lot more complex because they can be used in different ways.

The above example doesn’t say to me that Scala 3 naturally supports typeclasses . It says to me that typeclasses can be simulated using a mixture of given, extensions, and context bounds. My perception is that that this makes the language harder for “regular” users as opposed to “power” users.

2 Likes

I would rephrase that as

typeclasses can be naturally expressed in Scala using existing features

I actually don’t know of a programming language that has an explicit typeclass keyword. Rust has traits. Even Haskell doesn’t explicitly say that something is a “typeclass”. The syntax is

class Semigroup a where
  combine :: a -> a -> a

instance Semigroup Int where
  combine x y = x + y

Haskell simply doesn’t have OOP features. Seeing as “Scala combines object-oriented and functional programming in one concise, high-level language”, it follows quite naturally that a Scala class or trait can be either or both a OO class or FP (type)class. I know there are people who disagree and say that everything has to be a separate isolated feature. But Scala has always gone the other way, being a “scalable” language.

1 Like

How about something like this?

typeclass SemiGroup[T] {
  @infix def combine (x: T)(y: T): T
}

typeclass Monoid[T: SemiGroup]
  def unit: T
}

lens Monoid {
  typeinstance StringMonoid implements SemiGroup[T], Monoid[String] {
    def combine (x: String)(y: String): String = x.concat(y)
    def unit: String = ""
  }

  typeinstance IntMonoid implements SemiGroup[T], Monoid[String] {
    def combine (x: Int)(y: Int): Int = x + y
    def unit: Int = 0
  }

  extension ListMonoidOps[T: Monoid] extends List[T] {
    def sum: T = this.foldLeft(Monoid.unit)(_ combine _)
  }
}

---

import Monoid

object Usage {
  List(1, 2).sum
  List("a", "b").sum
}

lens Usage includes Monoid

and this:

typeclass Functor[F[_]] {
  def map[A, B](x: F[A])(f: A => B): F[B]
}

typeclass Monad[F[_]: Functor]
  def flatMap[A, B](x: F[A => B])(f: F[A]): F[B]
  def pure[A](x: A): F[A]
}

lens Monad {
  typeinstance MonadFunctor[F[_]: Monad] implements Functor[F] {
    def map[A, B](x: F[A])(f: A => B): F[B] = Monad.flatMap(Monad.pure(f))(x)
  }

  typeinstance ListMonad implements Monad[List] {
    def flatMap[A, B](xs: List[A])(f: A => List[B]): List[B] =
      xs.flatMap(f)
    def pure[A](x: A): List[A] =
      List(x)
  }
}

---

import Monad

object Usage {
  List(1,2).map(_ + 1)
}

lens Usage includes Monad

But traits shouldn’t be both OO class and type classes (which are not inherently FP). They are two different models of polymorphism, and conflating both of their quite different semantics into one construct is extremely confusing.

Scala has not always gone this way; for instance, def is an “isolated” feature that could be replaced with function values.

For me the key difference between Haskell and Scala is as you say, Haskell only has type classes, so they are likely be widely used and understood by the programmers writing code in Haskell. But I’m not convinced that is the case in the Scala domain. I suspect that most Scala engineers come from an OOP background and hence are more familiar with OO, and less familiar with type classes.

Also, it looks like both Rust and Haskell do use a keyword to identify a typeclass instance (i.e. instance or impl).

1 Like

I’m not sure about “lens Monoid” construct bit, I would rather it could work without this, or just use “object Monoid”.

Presuambly your extension syntax needs a tweak to bind the variable name “xs” to the extended class. Or perhaps you intended to use “this” instead?

I’m also not sure that an extension should “extend” a class, but that might be okay.

Otherwise, I quite like the syntax for " typeinstance … implements … "

First of all, I really love where things are heading towards.
I find the summary of “several classes of implicit” a few comments above by M. Odersky very useful, even for me as a beginner to get the idea behind this full rewrite.
I also like the idea of separating concerns, and having extensions as their own separate thing. Easier to Google, easier to look for in a code base, even easier to setup “tech debt rules” or stuff like that for code analyzers (like: “not to much extensions in Int, please…”).

The single sentence “conversions can be seen as special cases of typeclasses” helped me a lot to understand the idea, too thank you.

Speaking of that, I agree with some of the comments above: the same way “Conversions are a special case of type classes”, I’m wondering if “typeclass are implemented using given” could maybe be hidden?
That definitely has drawbacks like introducing a new keyword for “something that can already be achieved using existing mechanisms” (hence being noisy).
But I find quite interesting to have a concept properly labeled and identified: a typeclass, then looking at the implementation (“oh, actually it simply relies on anonymous given instances, so look: it looks like a given instance, but for a type, OK, it all makes sense now”).
I had to read the doc multiple types and typing these lines so that it “clings” to my mind: a typeclass “instance” (as in Haskell) is a “given class” (as in the docs) acting as as a given at type level, instead of value level (as given instances do). But I’m not sure I really understood what the object Monoid { ... } is about, it’s confusing to me.

Now I think I understand the sentence a bit better: “Typeclasses are just traits with canonical implementations defined by given instances.” from the docs. But maybe (and I agree with some previous comments) I shouldn’t dive into the “survivor bias” and forget about how I struggled first (especially since I think I get why there’s an object definition but I’m not really sure).

Maybe a typeclass keyword or typeinstance is too much. Maybe just a few notes on the docs or a good tutorial may help, too (that’ll always help in every case, anyway).

I’m not that much concerned about trait. Trait means “some definition of a type, a behaviour” (to me, at least) so it doesn’t confuse me that typeclasses are defined (as in Rust) with traits. But I do agree that Rust has impl, Haskell has instance whereas Scala seem to have a combination of an obejct and a given class. object seem to be part of the definition of the typeclass (you’ll be able to create an instance by defining a given class), but… I’m hesitating. Defining the typeclass through a maybe type trait, type class or something around these lines?

I understand this is maybe not the most helpful comment, if it sounds confused, that’s because I am.

But truly, the last re-write (of the syntax, and the docs) mentioned in this thread already helped me a ton, thank a lot for this. There’s just this typeclass thing that bugs me a bit. I think I got the concept from Haskell, but here the object part confuses me.

Thanks.

EDIT: I should have read typeclasses-new.html from the doc and not the old version. Now it’s given as and not only given

Yeah, it’s mostly an object-like component for the purpose of modularity, and the main reason for the separation between the two is so that those “implicit interpretations” (extensions, type instnaces, conversions) won’t be applied automatically when importing. You can read about this more in my proposal.

Yep, it was supposed to be this, I forgot to change it. I think it should still extend a class because (a) it is an extension, and (b) this prevents extending pure generic types (extension MyExt[A] extends A). Again, you can read about it more in my proposal :slight_smile:

I wouldn’t personally mind if methods and functions could be unified in a single concept at the language level. But as it stands I guess you could also say that functions in Scala are “simulated using a mixture of methods, classes and objects”. Given that a function f is actually an object that’s an instance of a Function class with an apply method.

1 Like

The thing is they are not actually a special case of type classes – at least not type classes in their original meaning – and for two reasons:

  1. Conversions work on concrete instances, while type classes work only on bound generic types.
  2. Conversions work without invoking any explicit function.

If conversion was really a type class you would’ve need something like this to use it:

// 1. explicitly asking for `Conversion` type class
// 2. explicitly invoking the type class via `apply()` or `()`
def convert[A, B](a: A)(implicit conv: Conversion[A,B]): B = conv.apply(a)

That’s exactly my point. Scala functions are simulated by other lower-level constructs, but that is mostly irrelevant to the developer and will likely hinder their work. A lot of clarity is gained when new constructs are designed for frequently used patterns that could’ve been implemented with existing lower-level constructs – that’s the core idea behind any non-assembly programming language.

1 Like

Agreed. But IMHO typeclasses don’t require a lot of extra ceremony on top of what’s already available in dotty. Adding too much extra sugar on top might confuse things more than anything else, but I know that opinions differ widely on this point. Anyway I came here to point out that the state of the art is already virtually equal to what @rgwilton suggested, modulo the keywords. I don’t think I’m the right person to defend all the given stuff, given that I wasn’t the biggest fan myself, though I must say it’s grown on me a little.

2 Likes