Allow cascading custom error messages with `@implicitNotFound`

The annotation @implicitNotFound enables us to customize the error message for missing implicits.
However, only the latest implicit in effect calls its error message, if exists.
In some cases we are missing crucial information why the error had occurred.
Example:

trait Car

@scala.annotation.implicitNotFound("missing Car")
trait Bar
object Bar {
  implicit def ev(implicit car : Car) : Bar = new Bar {}
}

@scala.annotation.implicitNotFound("missing Bar")
trait Foo
object Foo {
  implicit def ev(implicit bar : Bar) : Foo = new Foo {}
}

trait FooUser
object FooUser {
  implicit def ev(implicit foo : Foo) : FooUser = new FooUser {}
}
implicitly[Car] //could not find implicit...
implicitly[Bar] //missing Car
implicitly[Foo] //missing Bar, but don't know why
implicitly[FooUser] //could not find implicit...

I suggest that we had an optional Boolean to @implicitNotFound:

final class implicitNotFound(msg: String, showPreviousCustomError : Boolean = false) extends scala.annotation.StaticAnnotation {}

This allows the user to specifically annotate whether or not the previous custom error should be displayed as well.

That won’t work, because the compiler can’t tell which implicit should have been “previous”, in general.

This is the same problem that makes nested implicits hard to use. If you ask for an implicit[Foo], the compiler knows what is needed. If no Foo can be found, there may be multiple possible sources for implicit Foo which failed for multiple different reasons.

For example Akka HTTP can suffer from this – if you are trying to use their Routing code, it can often fail for want of an implicit ToResponseMarshaller, and the compiler can see that is missing. However there are many different implicit sources for a ToResponseMarshaller depending on what else is implicitly in scope, see e.g. PredefinedToResponseMarshallers.scala. The compiler (and often, the programmer) will not know which “should” have been used.

When we write:

implicit def foo[T](implicit bar: Bar[T]): Foo[T] = ???

it means "if there is an implicit Bar[T] then there is an implicit Foo[T]". This way the compiler can fallback to other possible sources of Foo[T] when it cannot find Bar[T].

However, sometimes we know that this is the only possible source of Foo[T]. It would be nice to be able to hint the compiler that it shouldn’t try other sources, e.g. with an annotation like:

implicit def foo[T](implicit @required bar: Bar[T]): Foo[T] = ???

Now it means: "foo provides an implicit Foo[T] and it requires Bar[T], i.e. it is an error if someone wants Foo[T] when there is no Bar[T]"

Now the compiler knows that there is no point in trying other sources of Foo[T] and it can show us an error about Bar[T] being unavailable.

1 Like

Good one, I can see this one being useful. What this option would provide is a clear logical dependence between implicit values.

What do you think @odersky? Perhaps an annotation is too severe for such a change in semantics – I think something along the same lines is worth exploring :slight_smile: .

In general I think that we should improve both the compiler and the language for users to tweak implicit resolution and constrain how expensive it is. This could also help speed up compile times.

Usually, when it comes to debugging implicit resolution, I always use -Xlog-implicits. I think, though, that this option is utterly verbose! I would really appreciate if someone took the time to improve it and clearly show the “implicit paths” that the compiler took in order to find an implicit value. In my view, the benefits of such a “short” summary are two-fold:

  1. Users can debug implicits with more ease.
  2. Users can spot inefficient implicit searches and optimize them for concrete scenarios.

Related post in our now defunct Google mailing list: Redirecting to Google Groups.

/cc People that could be interested in this: @adriaanm @milessabin.

2 Likes

If I define implicit val myImplicitFoo = new Foo[Int] somewhere in my code, would that give an error then?

No, this one would override foo[T], because it’s more specific.
What I want to tell the compiler is that the suitability of def foo[T] as provider of implicit Foo[T] does not depend on existence of implicit Bar[T]. Ultimately, Bar[T] will be required, but this check happens after foo[T] has already been chosen as the suitable implicit.

I think this should also benefit compiler performance as it makes implicit search simpler in such cases.

Hello,

So, this would be illegal?

implicit def foo[T](implicit @required bar: Bar[T]): Foo[T] =
???implicit def foo[T](implicit baz: Baz[T]): Foo[T] = ???

 Best, Oliver

In principle, no. During implicit search, the compiler would simply treat this the same way as this:

implicit def fooBar[T]: Foo[T] = ???
implicit def fooBaz[T](implicit baz: Baz[T]): Foo[T] = ???

And if fooBar was chosen then the requirement for Bar[T] would be enforced (unconditionally).

When debugging, that may be true, but in some cases we want to notify the user of library exactly what is wrong. In my use case (a DSL library) I wouldn’t even dare suggest the user to turn this on, since implicits are a useful backend tool to implement DSL language constraints (e.g, limit a vector’s length type argument to be positive).

The way I see it, at the first moment you build a DSL based on implicits, your users are already doomed and have to familiarize themselves with the underlying implicit design in the same way you do as a maintainer. It’s a case where the use of an abstraction leaks, and it happens in serialization libraries and other design patterns like the Spray magnet pattern.

Given that, it doesn’t seem a bad idea to encourage your users to use it, especially if this debug option is improved, as I suggest. This was only a thought though, and improving debugging of implicits would probably deserve a new thread and a concrete proposal, which I don’t happen to have the time to do. But if someone volunteers, I’m happy to help!

@ghik I like your idea. If more people agree with it, would you consider formalizing it in a SIP? First, we should try to gauge more support though – I see what I can do on my side. Another option to get attention is to open a ticket either in the Scala Dev or the Dotty repository.

2 Likes

Do you see a concrete use case where that would be helpful?

I tried your example out with Scala 2.12.1, Splain compiler plugin 0.1.22 with -P:splain:implicits:true passed to scalac.

scala> implicitly[Car]
<console>:12: error: implicit error;
!I e: Car
       implicitly[Car]
                 ^

scala> implicitly[Bar]
<console>:12: error: implicit error;
!I e (missing Car): Bar
Bar.ev invalid because
!I car: Car
       implicitly[Bar]
                 ^

scala> implicitly[Foo]
<console>:12: error: implicit error;
!I e (missing Bar): Foo
Foo.ev invalid because
!I bar (missing Car): Bar
Bar.ev invalid because
!I car: Car
       implicitly[Foo]
                 ^

scala> implicitly[FooUser]
<console>:12: error: implicit error;
!I e: FooUser
FooUser.ev invalid because
!I foo (missing Bar): Foo
Foo.ev invalid because
!I bar (missing Car): Bar
Bar.ev invalid because
!I car: Car
       implicitly[FooUser]
                 ^

This is not quite what you asked for, but might satisfy your use cases. This seems like a significant usability improvement over -Xlog-implicits.

If a plugin can know this, why can’t Scalac use the exact same information to display the proper @implicitNotFound messages? Or alternatively, can the plugin be modified in such a way to print these messages?

Another question: Can the Splain plugin work with Clippy? If so, then perhaps that can be good workaround for my usecase. Another option is to fork the plugin and create a dedicated one for my uses.

That would be an extremely welcome open-source contribution.

Or alternatively, can the plugin be modified in such a way to print
these messages?

It already does.

Another question: Can the Splain plugin work with Clippy
https://github.com/softwaremill/scala-clippy? If so, then perhaps
that can be good workaround for my usecase. Another option is to fork
the plugin and create a dedicated one for my uses.

It seems like they would work OK with settings where they aren’t trying
to munge the same error output. It’s worth a try, anyway.

OK, I figured how to do this with macros, similarly to the shapeless trick applied for Lazy.

//Typeclass to pass an Implicit Not Found annotation of `T` to the given symbol `Sym`
trait PassINF[T, Sym] {
  value : T  
}
object PassINF {
  implicit def ev[T, Sym] : PassINF[T, Sym] = macro ...
}

Using the above we can selectively pass an implicit error message:

trait SomeT
trait Test
object Test {
  implicit def ev(implicit t : PassINF[SomeT, Test])  : Test = new Test{}
}
implicitly[Test] // returns the same (or additional) error message that `SomeT` was not found

@SethTisue What do you think the proper construct should be, if I wish to apply a PR to the compiler?

@soronpo I would expect the PR you do to differ from that implementation. You’ll probably need to go to typer and modify the handling of errors for implicits (confirm /cc @SethTisue).

In fact, the sources handling the implicit error improvements from Splain should help you get up and running quickly.