Pre-SIP: Additional syntactic sugar for typeclasses

Scala 3 has taken great steps towards simplifying and differentiating the common uses cases of Scala 2 implicits. Two major annoyances when using typeclasses — the need to name things that do not need a name, and the boilerplate required to dot-call typeclass methods — have been heavily reduced thanks to anonymous givens and extension methods. This Pre-SIP argues for going a little further in this direction to remove similar annoyances that remain. I’ll first summarize the proposal, which has two separable parts, before proceeding to the motivation. I’ll add more detail about the implementation and discuss drawbacks at the end. I will build upon the typeclass examples in the Scala 3 book.

Update after discussion

I’ll leave the old proposal below so as not to confuse people, but after many useful comments, I would like to change the proposal. The new proposal is

Sugar for anonymous instance types

Let all of these declarations be sugar for each other

// new sugar
def showAll(xs: List[? : Showable]): Unit = xs.foreach(x => println(x.show))
// existing sugar
def showAll[S: Showable](xs: List[S]): Unit = xs.foreach(x => println(x.show))
// unsugared
def showAll[S](xs: List[S])(using Showable[S]): Unit = xs.foreach(x => println(x.show))

In general, the syntax ? : <type lambda> on a parameter’s type annotation should be sugar for adding a context bound <named type variable> : <typed lambda> in the method’s type parameter list.

Sugar for object methods

This section of the proposal can already be accomplished once this PR is in. I believe one can write

trait Showable[A] { self => 
  extension(a: A) def show: String
  def showMissing: A
  extension(Showable.type):
    export self.showMissing
}

(Old) Proposal

Sugar for anonymous instance types

Let all of these declarations be sugar for each other

// new sugar
def showAll(xs: List[Showable[_]]): Unit = xs.foreach(x => println(x.show))
// existing sugar
def showAll[S: Showable](xs: List[S]): Unit = xs.foreach(x => println(x.show))
// unsugared
def showAll[S](xs: List[S])(using Showable[S]): Unit = xs.foreach(x => println(x.show))

In general, any type tree with a single _ in (type) argument position will be converted a using clause with the _ argument substituted for a named type parameter that is also added to the surrounding method’s type parameters.

Sugar for object methods

Add a @objectmethod (or some better name) annotation that permits the following typeclass declaration:

trait Showable[A]:
  extension(a: A) def show: String
  @objectmethod def showMissing: A

to be sugar for

trait Showable[A]:
  extension(a: A) def show: String
  def whenMissing: A

object Showable:
  def whenMissing[A](using x$generated: Showable[A]): A = x$generated.whenMissing

so that consumers of the Showable typeclass can write

def maybeShowAll[A: Showable](xs: List[Option[Showable[A]]]): Unit = 
  xs.foreach(_.getOrElse(Showable.whenMissing).show)

In general, any method annotated with @objectmethod will be copied to the companion object, but with an additional type parameter for each type parameter to the enclosing trait and a using clause asking for an instance of the enclosing trait with those type parameters (where the type parameters copy over any bounds that exist in the trait type parameter declarations).

Motivation

The central motivation for this change is the same as stated above for some of Scala 3’s previous improvements: to avoid naming things that do not need a name, thereby avoiding sometimes significant boilerplate.

The change allows the author to avoid naming instance types for typeclasses in the common case where the instance type does matter. This change is heavily inspired by Rust’s support for typeclasses. Anonymous instance types are very similar to Rust’s impl parameters, which allows

fn foo(arg: impl Trait) {
}

to be sugar for

fn foo<T: Trait>(arg: T) {
}

Note that this syntax is not limited to typeclasses that take a single type parameter. For example, it allows a nice analogue of Rust’s Into pattern for declaring implicit conversions to be allowed at the usage site – this was my original motivation for proposing the feature here, when discussing the proposed into keyword.

type Into[-T, +U] = Conversion[T, U]
extension(t: T) def into()(using conv: Conversion[T, U]): U = conv(t)

def add(a: Into[_, Int], b: Into[_, Int]): Int = a.into() + b.into()

I use this just as well-known example, not to tie this Pre-SIP up in the discussion over the proposed into feature. Implicit conversions are not the only use for two-argument typeclasses.

The @objectmethod allows the author to avoid naming a typeclass instance (or using summon) when calling a non-instance level typeclass method: without the method generated by the sugar.

object Showable
  def whenMissing[A: Showable]: A = summon[Showable[A]].whenMissing

the caller must do one of the following:

// use `summon`
def maybeShowAll[A: Showable](xs: List[Option[Showable[A]]]): Unit = 
  xs.foreach(x_.getOrElse(summon[Showable[A]].whenMissing).show)
// name the typeclass instance (losing context bound sugar in the process)
def maybeShowAll(xs: List[Option[Showable[A]]])(using showable: Showable[A]): Unit = 
  xs.foreach(_.getOrElse(showable.whenMissing).show)

I find coming up with a name for an instance of Showable to be particularly awkward, since it is in fact the x: A that is “showable”, not the typeclass instance. Of course, in methods with multiple parameters implementing the same typeclass, one of these strategies will still be necessary. We can only hope to improve the common case where there is only one.

Using both forms of sugar in this Pre-SIP, this method can be particularly elegant:

def maybeShowAll(xs: List[Option[Showable[_]]]): Unit = 
  xs.foreach(_.getOrElse(Showable.whenMissing).show)

Implementation

Since both of these methods can be thought of as syntactic sugar, the implementation should be fairly straightforward, analogous to other forms of existing sugar. There is some subtlety in the use of the type lambda syntax x: Showable[_]: we could decide that we want to allow any type lambda, even one that does not name the anonymous argument (H/T to @rjolly for the suggestion)

type ShowableT = [T] =>> Showable[T]
def foo(x: ShowableT): Unit = { ... }

This proposal takes the position that we should not permit that. This would require delaying application of the sugar until after the typer has run, leading to what I expect to be a much more complicated implementation. Furthermore, the reader would not be able to realize the sugar is being applied without inspecting the type lambda’s declaration, which could lead to considerable confusion. The typer should continue to emit an error when using a type lambda as a parameter type, and the parser should perform the purely syntactic operation of substituting a singleton _ in a parameter’s type declaration with type parameter. An error should be emitted if there is more than one _ in (type) argument position.

The auto-generated of the object counterparts to @objectmethod declarations will have to decide whether they error when conflicting with an existing declaration or silently fail to generate (as in the case of a def equals on a case class currently). This Pre-SIP proposes that an error is emitted if there is a method matching the signature of the generated counterpart, since there would be no purpose in having @objectmethod in that case.

Drawbacks and Possible Objections

Perhaps the largest drawback to this proposal is the very high potential for confusion from the type lambda syntax. Before 3.2, s: Showable[_] is a legal parameter declaration that means something quite different – it means what will soon become s: Showable[?]. I think this is an acceptable confusion because (a) the confusion will exist with or without this proposal, though in fewer places and (b) it is arguably a nice feature that the syntax still evokes “there is a type parameter here, but I don’t care what it is”. Still, it is likely that newcomers to Scala will find themselves randomly trying ? and _ when they want to ignore the type and occasionally getting confusing compiler errors. A user who erroneously writes x: Showable[?] and then tries to invoke x.show will be confused by an error saying there is a missing given instance, while a user who erroneously writes x: Showable[_] will be confused by an error indicating a missing implicit even if x goes entirely unused or has only regular methods invoked on it.

Another objection is that the desugaring is quite leaky: the invisible additional using declaration will confuse many users and lead to strange error messages. I do not believe this problem is any worse than the analogous problem context bounds (A: Showable). One might argue that the benefit to the new sugar is not as high as for context bounds, though I would disagree. I find that any time I try to use a typeclass, the less typeclass-inclined members of my team are continually frustrated by the syntax required to use one when compared to the simplicity of using inheritance. Context bound syntax solves half the problem (not needing to write out the extra using parameter), but the other problems – not needing to name a type parameter and/or typeclass instance you do not care about not needing to summon the typeclass instance – remain a pretty high barrier.

Although the example above allows both kinds of sugar to be applied in one method, in many cases, the compiler will not be able to infer the type argument(s) to the object method and they will need be explicitly stated, forcing the user to name a type parameter that they largely don’t care about (as they do currently). In a hypothetical compiler that could infer A in def foo(using F[A]) in a context where only one implicit matching F[_] is defined, this wouldn’t be necessary, but that’s not something we should expect the Scala compiler to ever do. Still, being able to use the context-bound sugar and avoiding calls to summon is a win, even if some type parameters still need to be named.

In general, one might argue that language changes are not necessary to eliminate what is ultimately a small amount of boilerplate. I think that Scala 3 has explicitly (and rightly) taken the stance that instead of having one very powerful language feature (implicits) that can implement multiple patterns (conversions, typeclasses, extension methods, etc.), the language should explicitly support those features in a first-class way. The context bound syntax exists almost entirely to support typeclasses AFIAK, but as a I stated above, it only eliminates about half the problem in the fairly standard case where you do not need to name the type parameter to a typeclass instance because the typeclass has abstracted over it.

3 Likes

My suggestion for nullary operators:

trait Showable[A]:
  extension(a: A) def show: String
  extension def whenMissing: A
2 Likes

Thanks. Just be clear: this is non-existing syntax right? You’re proposing extension def, which currently doesn’t compile, be treated the same as the proposed @objectmethod def? That’s clever, though I think it’s a little confusing because it’s not clear that the extension is applied to object Showable and takes a using Showable[A] argument?

I had thought that it would be nice to have a version of export that does method edit:

trait Showable[A]:
  extension(a: A) def show: String
  def whenMissing: A

object Showable:
  export(using s: Showable[A]) s.whenMissing

but that’s a little magical too.

Also, I’d add that it’s not just nullary methods — any method that can’t be written as extension method on A would quality.

I largely agree with the motivation. These are two areas that could profit from a shorthand syntax. But I am not sure the proposed syntax is the last word on the matter yet.

  1. List[Showable[_]] is not well-kinded, and I would find it surprising to fix kind errors in the way that was proposed. We also want to re-inforce the analogy with context bounds. There it is [X: Showable], not [X: Showable[_]]. So maybe List[:Showable] or List[? : Showable]?

  2. We should stick in Scala 3 to the meta rule that annotations do not change typing. @objectmethod would violate that. I think it’s better to use a soft modifier. objectmethod is a bit long, but what about shared?

4 Likes

You can have this already:

trait Showable[A]:
  extension (a: A) def show(a: A): String
  extension (s: Showable.type) def whenMissing: String

But that’s a bit cryptic and falls apart when you have multiple Showable instances in scope, e.g. in def foo[A: Showable, B: Showable] = ???.

I’m really not convinced more syntactic sugar here will pull its weight given the existence of context bounds, but if we did want a new syntax, we should avoid more ungoogleable symbols. One option would be to just copy what Rust did and writing List[impl Showable].

By the way, impl in Rust can also show up in the result type, where it behaves like an existential type: Impl trait type - The Rust Reference

1 Like

That’s already quite nice IMO although this won’t work if you don’t define a (possibly empty) companion object

I thought about that and agree something like it would be nice, but I couldn’t figure out a way to make it non-ugly. In the case without a surrounding type, def foo(x: ? : Showable) is a little tough to parse, and I can almost guarantee that people will be frustrated that you can’t write def foo(x: ?: Showable) because ?: parses as a single token. Similarly, x: :Showable is almost perfect, except that most people write T: Showable (I think?) not T :Showable. That’s minor enough though that I would think it’s probably fine.

I was also a little worried about changing the parser but if you’re not worried, I’m not worried!

I wasn’t aware of the meta rule, I thought the rule was just that type-changing @ annotations couldn’t be user-defined. A soft modifier would be great too. shared sound good to me.

I should have pointed out that you might use shared in cases without typeclasses: in the Scala 3 compiler, there is this method

inline def ctx(using ctx: Context): Context = ctx

that allows you to call ctx inside a def where you didn’t name the using parameter

def foo(using Context) = ctx.withPhase(...)

An alternative pattern would be to mark withPhase as shared and then call Context.withPhase(...), since the object version of atPhase's signature would be

final def withPhase(pid: PhaseId)(using Context): Context

No need to make a magic name for all your implicit contexts – just use the type name.

While this does allow Showable.whenMissing, it doesn’t allow one to dot-call on a typeclass instance, i.e. (showable: Showable[String]).whenMissing won’t compile. You’d have to call showable.whenMissing(Showable), but maybe that’s fine.

I agree that it’s probably fine to ask users to do this and forego the shared keyword. I didn’t realize it was possible, thanks.

impl seems reasonable, though I think in Rust it intentionally mirrors the declaration of trait implementations, so the analogue in Scala would be given – but of course givens aren’t only for typeclasses in Scala, so that’s probably not great.

I take this back. In Scala 2, I would write _ <: T all the time and it wouldn’t bother me. I’m sure I could get used to ? : T.

1 Like

On the other hand, I think ? : doesn’t work for multi-arg typeclasses. You could assume you always substitute the first type argument to get

def add(a: ? : Into[Int], b: ? : Into[Int]): Int = a.into() + b.into()

but that’s confusing, and also wouldn’t let you do the equivalent of a: Into[Int, _] that original proposal would allow.

This works fine in Rust because the first type argument to a typeclass is always implicit, but it’s confusing in Scala where it’s explicit. That’s part of why I liked abusing kindedness – it was at least visually clear what type argument you are filling in.

Another option is to use : instead of _ so there’s no kindedness confusion:

def showAll(xs: List[Showable[:]]): Unit = xs.foreach(x => println(x.show))
def add(a: Into[:, Int], b: Into[:, Int]): Int = a.into() + b.into()

It’s also visually “closer” to T : Showable, although not by much.

I prefer the impl keyword from Rust. In Scala, we can let impl be followed by a higher-kinded type of kind * => *. This would just add one rule to the syntax. The examples above would then be

def showAll(xs: List[impl Showable]): Unit = xs foreach { x => println(x.show) }
// this is sugar for
def showAll[T](xs: List[T])(using T$instance: Showable[T]) = ...
def add(a: impl Into[_, Int], b: impl Into[_, Int]): Int = a.into() + b.into()
// here Into[_, Int] is a type lambda X =>> Into[X, Int], of kind * => *
// this is sugar for
def add[T1, T2](a: T1, b: T2)(using T1$instance: Into[T1, Int], T2$instance: Into[T2, Int]) = ...

This preserves the visual clarity of what type argument you are filling in.

1 Like

I don’t see a reason to save few characters here by using impl

def showAll[T: Showable](xs: List[T]) = ...
def add[T1: Into[_, Int], T2: Into[_, Int]](a: T1, b: T2)
7 Likes

It’s a fair point that it’s not a big character savings, though of course neither is using Context vs using ctx: Context. I’m curious if you think allowing anonymous usings and givens was not worth the trouble either?

I like this because then we could also use arbitrary type lambdas without worrying about confusing the reader or polluting the typer. I’m still not totally sold on impl since it doesn’t have the same analogy to the rest of the language as it does for Rust. I wonder if using would be terrible? I think it’s not ambiguous, and I think it’s suggestive in the right way – it is sugar for a using clause elsewhere.

I read impl Typeclass as "an unspecified type that implements the given Typeclass". Neither using or given can be read this way.

Sure, makes sense. Since typeclasses are not truly first class in Scala – they are just traits after all – this is just a bit of a stretch. One also talks about a class “implementing” an interface in general (or a trait), and “implement” appears nowhere in the canonical syntax for declaring a typeclass instance. I think it would be confusing if “implementation” was newly scoped to typeclass syntax.

All that said, I would happily take impl if that’s the consensus. Something like for might also make sense, although that might cause more parsing headaches than it’s worth.

1 Like

I didn’t know this syntax was expected to work. It doesn’t in 3.1.2 – I tried this in Scastie:

trait Into[A, B]

given Into[Int, String]()
given Into[Boolean, String]()

// Playground.Into[?, String] does not take type parameters
def uses3[T: Into[_, String]](t: T) = t

// ambiguous implicit arguments: both object given_Into_Int_String in object Playground and object given_Into_Boolean_String in object Playground match type Playground.Into[?, String] of an implicit parameter of method uses3 in object Playground
uses3(1)

If this syntax is expected to work, ie. if the right of the : can be any type lambda, then I’m back to agreeing with @odersky that ? : Showable or :Showable are the best.

Well, that, or we allow def foo[T impl Showable] in addition to def foo[T: Showable], but I think that won’t work because it’ll get parsed as an infix type?