Dotty Type classes

Thanks for the proposal, it looks nice.

However, I disagree with the rejection of inheritance for defining type classes. The only given justification:

Adelbert Chang wrote a paper describing in great detail why subtyping for type classes is a suboptimal approach.

seems somewhat misled. AFAIU, despite its title (which I find slightly inadequate), the cited paper does not show that subtyping per se has any fundamental limitations for encoding type classes. All it shows is that in the current Scala implementation there is no way to prioritize implicits derived from subtyping, whereas one can prioritize implicit conversions which allows to avoid ambiguity errors… but ad-hoc prioritization is a hack anyways and can lead to surprises in case the instances are truly ambiguous, so it doesn’t really matter.

The real problem is either:

  • to rule out the ambiguity in the first place, and the way that would be done is via coherence. As is mentioned in Chang’s paper, this Dotty ticket shows that it can be done with subtyping;

  • or to have a convenient way to specify which instance the user wants, and for this I have seen different tricks but can’t remember one that doesn’t work with subtyping (though there might well be).

So instead of going with the natural, idiomatic inheritance-based definition approach, your proposal basically duplicates all its features, greatly increasing the surface of the language, but for apparently no good reason.

Please correct me if I’m wrong :^)

1 Like

I would take orphan instances over coherence.

Coherence is not the problem we have with type class usage thus far. In my opinion at least. It’s a problem to think about for sure, but convention largely works fine. Personally I’ve never been bitten by violations of coherence. I know it probably happens sometimes, but with “opaque types” and education, I think it will be a non-issue.

The problem in the ecosystem is that there will be libraries that don’t want to depend on any particular library of type classes, e.g. Cats or Scalaz, and provide type class instances in separate projects, thus orphans. And the reasons for this can be very valid, like having a “no dependencies” policy for the core.

Orphans are a must. AFAIK GHC allows orphans too and that too can lead to conflicts. But it’s better than the alternative.

5 Likes

A possible feature request: better handling of the special cases induced by case classes. I haven’t used type classes much but really like the idea already. On my recent foray into Scala type classes I came across this problem, which is basically the fact that it is not easy (and there appears no standard way) to require that a case-class-like copy method exists. Maybe there just isn’t a good solution to this problem, but I can see it potentially being very nice from an ergonomics perspective.

Basically, we want to be able to add a type class member copy that can be easily satisfied by a case class’s copy:

trait Foo[F] {
  type T
  def aa(foo: F): List[T]
  def maybe(foo: F): Option[T]
}

final case class Bar(aa: List[String], maybe: Option[String])
object Bar {
  implicit val barFoo = new Foo[Bar] {
    type T = String
    def aa(foo: Bar): List[String] = foo.aa
    def maybe(foo: Bar): Option[T] = foo.maybe
  }
}

@LukaJCB I very much like your proposal. I have a syntax nit and a question.

Syntax Nit

type class CommutativeMonoid[A] : CommutativeSemigroup[A] : Monoid[A]

which to me it looks like the CommutativeSemigroup[A] instance must be a Monoid[A]. I’d prefer something that looks more like intersection.

type class CommutativeMonoid[A] : (CommutativeSemigroup[A], Monoid[A])
type class CommutativeMonoid[A] : CommutativeSemigroup[A] & Monoid[A]

Or in an extreme case

type class A : B : C: D: E

I want to read as

type class A : (B : (C : (D : E)))

So I’d rather write it as

type class A : (B, C, D, E)

Multiple Derivations

Can there can be multiple ways to define a type class? Like suppose I have a type class A, which I can define in terms of type classes B and C in an efficient way, but a default without them would work too? In other words, the following shouldn’t conflict.

type class A

extension simpleA[T]: new A { ... }    
extension fancyA[T: B & C]: new A { ... }

I swear I’ve seen something like this in the Haskell typeclass ecosystem, but it’s been years since I’ve looked at it.

I disagree – this feels (appropriately) like the context-bound syntax for specifying multiple typeclasses. I don’t think it implies anything any CommutativeSemigroup.

I get why you have this intuition, but the proposal seems consistent with the way we usually talk about types in Scala…

1 Like

I think we can easily keep the subtype encoding of type classes and resolve ambiguity errors at the same time by just narrowing the type signatures of implicit instances.

Let’s ask – what prevents us from choosing an arbitrary instance in case of ambiguity?
It’s only the fact that when we get two opaque implicit values in our implicit search: Monad[F] and Traversable[F], we can’t possibly know if the underlying Functor[F] is the same for both.

If we knew, we could pick either of them. But we could just directly check if they’re the same!

Let’s start with the usual suspects:

trait Functor[F[_]] { def map[A, B](f: A => B): F[B] }
trait Monad[F[_]] extends Functor[F]
trait Traversable[F[_]] extends Functor[F]

trait Maybe[A]
object Maybe {
  trait FunctorImpl extends Functor[Maybe] {
    final def map[A, B](f: A => B): Maybe[B] = ???
  }

  implicit val maybeMonad: Monad[Maybe] = new Monad[Maybe] with FunctorImpl { ??? }
  implicit val maybeTraversable: Traversable[Maybe] = new Traversable[Maybe] with FunctorImpl { ??? }
}

When we try to summon an implicit value, we’ll get an ambiguity error.

  implicitly[Functor[Maybe]] // error

We know that both maybeMonad and maybeTraversable are inheriting the implementation of Functor[Maybe] from FunctorImpl, we also know that map in FunctorImpl is final, so neither of them could override it.
We have ample evidence that both of these implementations are equivalent with respect to Functor, but unfortunately, we hid it from the compiler by widening the type back to Monad[Maybe] and removing the mention of FunctorImpl, let’s fix that mistake:

  implicit val maybeMonad: Monad[Maybe] with FunctorImpl = new Monad[Maybe] with FunctorImpl { ??? }
  implicit val maybeTraversable: Traversable[Maybe] with FunctorImpl = new Traversable[Maybe] with FunctorImpl { ??? }

Or just omit the signatures and let the compiler infer the narowest type:

  implicit final val maybeMonad = ???
  implicit final val maybeTraversable = ???

Now the compiler, too, has direct unfalsifiable evidence that both of the implementations are the same, it only needs to glance at the type signature to verify it. At this point it can safely pick either instance, knowing that they both point to the same trait.

  implicitly[Functor[Maybe]] // good, all the instances are just FunctorImpl in disguise

In more detail, when encountering an ambiguity over some type T, for each ambiguous instance,
the resolver should try to find the latest base class that implements T with all T methods as final, if all found base classes are the same for every instance, the compiler now has direct evidence that all implementations of Tin the search are the same and can pick any of them.
Alternatively, a more exhaustive scheme would be to extract the set of methods of T and verify that for each ambiguous instance, there is a final implementation of each method with the same ‘source’. This would allow two methods of T to be finaly implemented by two different traits, but still correctly disambiguated.

For library authors, the scheme would entail two minor changes to their workflow:

  • they would have to specify the narrowest type for their implicit values, instead of widening them
  • they would have to final their methods

For users of implicit there would be no changes at all – they would just suddenly stop getting ambiguity errors! They wouldn’t have to specify coherence domains in their implicit summons and they wouldn’t lose the ability to declare orphans and local instances.

This scheme does not substitute for coherence in all cases, but it solves the problem of ambiguity in a straightforward and obvious manner without imposing any draconian restrictions on the users.

Implementation: the scheme is a simple enough extension as to be implementable today as an implicit macro or as a compiler plugin.

Caveats: If an implicit value is taken and re-assigned to a variable with a widened type, it’s disambiguation properties are lost. However, coherent domains exhibit the same properties if an implicit in a domain is re-assigned to a domainless value.

A variant of this scheme can also be implemented if Scala gets first-class delegation, e.g:

  final val functorMaybe: Functor[Maybe] = ???

  implicit val maybeMonad: Monad[Maybe] with (Functor[Maybe] by functorMaybe.type)
  implicit val maybeTraversable: Traversable[Maybe] with (Functor[Maybe] by functorMaybe.type)

Would be resolved because the singleton types which implement the Functor class are proven the same.

4 Likes

@kai I must say I really like this proposal as a compromise between adding a lot of duplicate type system extensions and achieving the desired outcome.

I also agree with @alexandru w.r.t orphan instances.
They’re a necessary nuisance in my opinion and completely forbidding them would break a lot of people’s code. In the proposal I wanted a way to make it easier to detect bugs when orphan instances are declared instead of outright forbidding them :slight_smile:

I like the idea, but isn’t one of the main problems that this currently doesn’t work:

def foo[F[_] : Monad : Traverse] = implicitly[Functor[F]]

If I understood correctly, this still wouldn’t work with this scheme.

2 Likes

I believe so as well. We’d have to specify somehow that Monad and Traversable have the same Functor instance. That’s what I was referring to when I mentioned sharing constraints.

@odersky

We’d have to specify somehow that Monad and Traversable have the same Functor instance. That’s what I was referring to when I mentioned sharing constraints.

Well, we can somewhat awkwardly express it already:

def foo[F[_]]: FooApply[F] = new FooApply[F]

final class FooApply[F[_]] {
  def apply[FunctorImpl <: Functor[F], MonadImpl <: Monad[F], TraverseImpl <: Traverse[F]]()
  (implicit F: FunctorImpl, M: MonadImpl, T: TraverseImpl,
   finalImpl: HasFinal[Functor[F], FunctorImpl], ev1: MonadImpl <:< FunctorImpl, ev2: TraverseImpl <:< FunctorImpl) =
  ??? // implicitly[Functor[F]]
}

Where HasFinal[C, I] is a magical type class that checks whether I satisfies the canonicity requirements for C – that methods of C in I are final and don’t call non-final members (implementable with a blackbox macro)

Or with a few helpers:

def foo[F[_]](implicit M: CanonBy[Functor[F], Monad[F]], T: CanonBy[Functor[F], Traverse[F]]): Functor[F] =
  ??? // implicitly[Functor[F]]

class CanonBy[C, I <: C](val value: I)

object CanonBy {
  implicit def canonical[C, I <: C, CImpl <: AnyRef, IImpl]
    (implicit cImpl: Narrow.Aux[C, CImpl], iImpl: Narrow.Aux[I, IImpl],
     valid: HasFinal[C, CImpl], ev1: IImpl <:< CImpl, ev2: IImpl <:< I)
    : CanonBy[C, I] = new CanonBy(iImpl.value)
}

trait Narrow[T] {
  type Out
  def value: Out
}

object Narrow {
  type Aux[T, O] = Narrow[T] { type Out = O }
  implicit def narrowest[T, N <: T with AnyRef](implicit n: N): Narrow[T] { type Out = n.type } =
    new Narrow[T] { type Out = n.type ; val value = n }
}

The resolver would have to be extended to take into account HasFinal evidence …

Okay, so what if we reused the idea of a marker trait from Martin’s original proposal and used it to suppress the implicit ambiguity errors before specialization.

So that this:

def foo[F[_] : Monad : Traverse] = implicitly[Functor[F]]

Would work if e.g. Functor extends from Typeclass.
Then when you call foo and specialize its type parameter, it could check if their base instance is the same like @kai described and throw a compilation error if not.

Does that make sense?

I am not sure which Marker thread you meant?

That’s exactly what we do in our plugin: https://github.com/scalaz/scalaz-plugin/blob/master/test/files/pos/ambigious_typeclass_parameters.scala

Note the Typeclass marker.

Ah I see now, you meant TypeClass as a marker.

Other possibilities would be to require that every combined context bound is locally coherent, or to introduce new syntax for coherent context bounds. See https://github.com/lampepfl/dotty/issues/4234#issuecomment-421950625

Not quite, scalaz-plugin just assumes that any marked implicit is coherent and picks arbitrary, it doesn’t check the implementations.

I’d be fine personally with requiring every combined context bound to be locally coherent, I’m not so sure about the proposed syntax foo[A: B & C] might be confusing with & meaning something different when used as types.

One potential drawback I could see is, that if you do end up having two different base implementations, the compile error would happen a bit removed from the actual problematic definition.
Example:

def foo[F[_]: Monad: Traverse](fa: F[Int]): F[String] = fa.map(_.toString)

class ListFunctor extends Functor[List] { def map ... }
class ListFunctor2 extends Functor[List] { def map ... }
implicit object ListFunctor extends ListFunctor
implicit object ListMonad extends ListFunctor ...
implicit object ListTraverse extends ListFunctor2 ...

foo(List(1, 2, 3)) // Error List is not coherent in it's Functor instance

Though I reckon in real code this will happen pretty much never.

[A : B & C] happens to be available as syntax, since & is normally defined only for first-order types, and B and C` are higher-kinded in context bounds. But maybe that’s a bit too cute. I propose to try to require coherence for all combined bounds first and fall back to specialized syntax if that proves too restrictive.

I’d also hope that your tests would catch this.

[A : B & C] happens to be available as syntax, since & is normally defined only for first-order types, and B and C` are higher-kinded in context bounds. But maybe that’s a bit too cute. I propose to try to require coherence for all combined bounds first and fall back to specialized syntax if that proves too restrictive.

Sounds good to me :+1:

I’d also hope that your tests would catch this.

Yes, that should definitely be the case.