Usability of boundary/break on the example of `Either` integration

[adamw]

I’m adding a boundary/break implementation tailored to Eithers to ox, but I noticed there are some usability issues (or simply problems in my implementation), and I’m not sure if they can be addressed using current error message customisation mechanisms.

First, here’s my (rather simple) implementation so far:

package ox

import scala.util.boundary
import scala.util.boundary.{Label, break}

object either:
  case class Fail[+A] private[ox] (a: A)

  inline def apply[A, T](inline body: Label[Fail[A] | T] ?=> T): Either[A, T] =
    boundary(body) match
      case Fail(a: A) => Left(a)
      case t: T       => Right(t)

extension [A, B, T](t: Either[A, B])(using b: boundary.Label[either.Fail[A] | T])
  inline def value: B =
    t match
      case Left(a)  => break(either.Fail(a))
      case Right(b) => b

This works nicely if used correctly, e.g.:

val ok1: Either[Int, String] = Right("x")
val fail1: Either[Int, String] = Left(1)

val r: Either[Int, (String, String)] = 
  either:
    (ok1.value, fail1.value)

However, I noticed two potential problems with usability. As a clarification: I know these are standard error messages and in that respect they work correctly. However, my premise is that if a boundary-break implementation (tailored to Either, Result or whatever) is supposed to be one of the basic building blocks when it comes to error handling in Scala, the bar on usability is quite high. So, generic error messages are not sufficient (as demonstrated below).

Problem 1 - using break outside of boundary

If we try to use .value outside of the either boundary, we get the following error:

ok1.value

[error] -- [E008] Not Found Error: test.scala:38:8
[error] 38 |    ok1.value
[error]    |    ^^^^^^^^^
[error]    |value value is not a member of Either[Int, String].
[error]    |An extension method was tried, but could not be fully constructed:
[error]    |
[error]    |    ox.value[Int, B, T](this.ok1)(
[error]    |      /* missing */summon[scala.util.boundary.Label[ox.either.Fail[Int] | T]])
[error]    |
[error]    |    failed with:
[error]    |
[error]    |        No given instance of type scala.util.boundary.Label[ox.either.Fail[Int] | T] was found for parameter b of method value in package ox
[error]    |
[error]    |        where:    T is a type variable with constraint
[error]    |
[error] one error found

I think from a user perspective the error is quite cryptic, and doesn’t really point to any possible solutions (wrapping with either:). Maybe that’s a possible enhancement to the compiler, as we are missing a Label implicit, which has a better error message:

@implicitNotFound("explain=A Label is generated from an enclosing `scala.util.boundary` call.\nMaybe that boundary is missing?")
final class Label[-T]

But that’s not used, I’m guessing because of the indirect implicit usage (via an extension method).

Ideally, I’d want to provide my own error message (via a type alias + @implicitNotFound?). (I’ll try a macro next, but that’s also not perfect from an implementors PoV.)

Problem 2 - wrong type ascription

If we use an incorrect type, the error message is confusing as well:

val r: Either[Int, Long] = either(ok1.value)

[error] -- [E008] Not Found Error: test.scala:38:42
[error] 39 |        val r: Either[Int, Long] = either(ok1.value)
[error]    |                                          ^^^^^^^^^
[error]    |value value is not a member of Either[Int, String].
[error]    |An extension method was tried, but could not be fully constructed:
[error]    |
[error]    |    ox.value[Int, B, T](this.ok1)(
[error]    |      /* missing */summon[scala.util.boundary.Label[ox.either.Fail[Int] | T]])
[error]    |
[error]    |    failed with:
[error]    |
[error]    |        No given instance of type scala.util.boundary.Label[ox.either.Fail[Int] | T] was found for parameter b of method value in package ox
[error]    |
[error]    |        where:    T is a type variable with constraint
[error]    |
[error] one error found

Again, I understand where the error is coming from, but I think we should be able to provide a better developer experience here (of course, still my implementation can simply be incorrect, so please correct me if I’m wrong). Using the custom error message from @implicitNotFound could provide some guidance, but I’m not sure if it would resolve this specific problem fully. If it’s possible to solve in any way at all?

[adamw]

Macros where a dead end, but it turns out inlines do the trick:

package ox

import scala.compiletime.{error, summonFrom}
import scala.util.boundary
import scala.util.boundary.{Label, break}

object either:
  case class Fail[+A] private[ox] (a: A)

  inline def apply[A, T](inline body: Label[Fail[A] | T] ?=> T): Either[A, T] =
    boundary(body) match
      case Fail(a: A) => Left(a)
      case t: T       => Right(t)

  extension [A, B](inline t: Either[A, B])
    transparent inline def value: B =
      summonFrom {
        case given boundary.Label[either.Fail[A]] =>
          t match
            case Left(a)  => break(either.Fail(a))
            case Right(b) => b
        case _ => error("`.value` can only be used within an `either` call.\nIs it present?")
      }

Note that the transparent is needed, as otherwise type inference fails for “correct” usages. This gives a nice error message for problem 1:

val ok1: Either[Int, String] = Right("x")
ok1.value

[error] 43 |    ok1.value
[error]    |    ^^^^^^^^^
[error]    |    `.value` can only be used within an `either` call.
[error]    |    Is it present?
[error] one error found

This might shadow some problems with constructing the extension method other than I have encountered so far - all such cases should be included in the error message, so this might need some refining.

(Problem 1.5) I found one such case - a wrong error type is explicitly given. Ideally we’d want to say “expected errors of type X, but got Y”, got maybe that’s a stretch:

val fail3: Either[String, String] = Left("x")
val r: Either[Int, String] = either(fail3.value)

[error] -- Error: test.scala:45:40
[error] 45 |    val r: Either[Int, String] = either(fail3.value)
[error]    |                                        ^^^^^^^^^^^
[error]    |                      `.value` can only be used within an `either` call.
[error]    |                      Is it present?
[error] one error found

Surprisingly, we also get a better error message for problem 2, for reasons which I don’t quite yet understand:

[error] 45 |    val r: Either[Int, Long] = either(ok1.value)
[error]    |                                      ^^^^^^^^^
[error]    |      value value is not a member of Either[Int, String].
[error]    |      An extension method was tried, but could not be fully constructed:
[error]    |
[error]    |          ox.either.value[A, B](this.ok1)
[error]    |
[error]    |          failed with:
[error]    |
[error]    |              Found:    (EitherTest.this.ok1 : Either[Int, String])
[error]    |              Required: Either[Any, Long]
[error] one error found

So unless there are some downsides of using transparent + summonFrom for such a use-case (some interactions with type inference? slower compilation?), this seems mostly resolved. Unless somebody has an idea for problem 1.5!

[adamw]

Actually, problem 1.5 can be solved by adding one more match case, for “other labels” that are in the scope:

  extension [A, B](inline t: Either[A, B])
    transparent inline def value: B =
      summonFrom {
        case given boundary.Label[either.Fail[A]] =>
          t match
            case Left(a)  => break(either.Fail(a))
            case Right(b) => b
        case given boundary.Label[either.Fail[Nothing]] => error("Error types don't match.")
        case _ => error("`.value` can only be used within an `either` call.\nIs it present?")
      }

I guess if we were picky we could still find places to improve the error messages (e.g. better error for problem 2, mentioning the error types explicitly in problem 1.5), but that’s increasingly cosmetics.

[natsukagami]

@adamw

extension [A, B, T](t: Either[A, B])(using b: boundary.Label[either.Fail[A] | T])
  inline def value: B =
    t match
      case Left(a)  => break(either.Fail(a))
      case Right(b) => b

I found out that the @implicitNotFound error would work if you include it in the using clause of the extension:

extension [A, B, T](t: Either[A, B])(using 
  @implicitNotFound("get a label!")
  b: boundary.Label[either.Fail[A] | T])
  inline def value: B =
    t match
      case Left(a)  => break(either.Fail(a))
      case Right(b) => b

[adamw]

@natsukagami

I found out that the @implicitNotFound error would work if you include it in the using clause of the extension:

Indeed, nice find! Thanks

Is this a bug in the compiler, that the @implicitNotFound defined on the Label type is not used, but the same annotation defined at usage-site works?

[Ichoran]

Oh, the custom message is a nice idea. I should do that on kse.flow, which uses boundary.break heavily for exactly this sort of thing (kse3/flow/src/Flow.scala at main · Ichoran/kse3 · GitHub for example). Funnily enough, I used that to give better error messages on math on wrong types, and didn’t think to use it here.

I suggest that rather than .value, which is almost always a safe accessor for a value, that you use .? to match Rust? Or perhaps .ask?

I’ve coded with this paradigm a lot, and having something where the control flow leaps out at you is a good idea.

Note that you also probably want to replace Try, because Try eats control flow, so it isn’t scalable. (I solve this by having safe blocks which let control flow through and threadsafe blocks which catch it.)

[adamw]

@Ichoran

I suggest that rather than .value, which is almost always a safe accessor for a value, that you use .? to match Rust? Or perhaps .ask?

Ha yes, naming things … I was considering ?, but I’ve never been a fan of symbolic methods, though maybe this would be a good exception. ask is an interesting idea, thanks, I’ll consider this one as well.

@Ichoran

Note that you also probably want to replace Try, because Try eats control flow, so it isn’t scalable. (I solve this by having safe blocks which let control flow through and threadsafe blocks which catch it.)

You mean generally, or do you see Try used somewhere in the codebase?

[Ichoran]

I mean generally. If someone uses your code but does:

val x = either:
  Try{ foo(input.value) }.getOrElse( bar(backup.value) )

you will never get your failure case off of input because Try will intercept the control flow.

So you need to provide an alternative. I do:

val x = Or.Ret:
  safe{ foo(input.?) }.getOrElse( bar(backup.?) )

[adamw]

@Ichoran

you will never get your failure case off of input because Try will intercept the control flow.

Interesting, thanks for pointing this out! It almost seems like a bug, that the break exception should be not caught by NonFatal? But I suspect this has been already discussed, so I’ll just use the search option

Anyway, that’s a good idea for another util function - though introducing a parallel-Try datatype is risky. An alternative would be to stick to try-catch for exception handling (with the assumption that

[Ichoran]

@adamw

introducing a parallel-Try datatype is risky

Oh, no, you don’t want a parallel Try. Just live in Either-land. In my world, safe{ a: A } produces an A Or Throwable. try/catch is okay too but it’s lower-level than is ideal. Better to abstract it, and if it’s inline, it really is just try/catch (unlike Try which has to form a closure).

@adamw

It almost seems like a bug, that the break exception should be not caught by NonFatal?

Well, that’s what I thought, but apparently the people who leak control flow logic out of their threads want their threads to not crash but to produce a Failure instead, so Try now catches control flow.

I always want to know where my thread boundaries are, and consider leaking control flow a critical bug not something to ignore at runtime, so I rolled my own instead.

While I was at it I also created my own success-unboxed Either clone so I can just work this way without worry. It turns out that you only need one one concept! Option[A] is A Or Unit. Try[A] is A Or Err (I did create an Err type that wraps both String and Throwable so I can create low-overhead error messages too). Either[A, B] is A Or B.

And you can always drop an Or.Ret: or Or.FlatRet: somewhere to jump out with failure cases on .? like either:/.value can in your flavor. I also have explicit remapping of the failure type with .?+ and with automatic remapping with .?* and a given. You can do without .?*, but inline .?+ avoids having to create a lambda object only to do nothing on the success case, so it’s a nice improvement over mapping the disfavored case (e.g. left in Either).

Monadic ops are nice, but a lot of times it’s way simpler conceptually to just cut through all the boilerplate/clutter.

It doesn’t work too well with the rest of the ecosystem, alas, but it sure simplifies my code and the concepts I need to employ.

[adamw]

@Ichoran

Oh, no, you don’t want a parallel Try. Just live in Either-land. In my world, safe{ a: A } produces an A Or Throwable. try/catch is okay too but it’s lower-level than is ideal. Better to abstract it, and if it’s inline, it really is just try/catch (unlike Try which has to form a closure).

Ah, you’re right, some helper functions which capture exceptions into an Either[Throwable, T]. This should be useful, noted

@Ichoran

Well, that’s what I thought, but apparently the people who leak control flow logic out of their threads want their threads to not crash but to produce a Failure instead, so Try now catches control flow.

I always want to know where my thread boundaries are, and consider leaking control flow a critical bug not something to ignore at runtime, so I rolled my own instead.

Yeah the fact that an exception is used for boundary-break feels almost like an implementation detail, which shouldn’t be visible (in fact, I think it’s sometimes optimized away and changed into a jump, which would “break” the Try). But there’s probably another side to the story as well, and that’s a different topic really.

[Ichoran]

@adamw

in fact, I think it’s sometimes optimized away and changed into a jump, which would “break” the Try

No, it isn’t, and that’s another reason why you want something else.

The JIT compiler can often be quite clever, but

either:
  Try(x.value)

necessarily wraps x into a closure which evaluates x.value on execution, because the signature is apply[A](a: => A): Try[A]. And that means that if x.value doesn’t exist, a Break[Fail[L]] throwable will have to be created to bail out of the closure.

If you actually want a jump, you need something like

inline def safe[X](inline x: => X): X Or Throwable =
  try Is(x)
  catch case e if e.catchable => Alt(e)

where e.catchable is specified by (in my case)

extension (t: Throwable)
  inline def catchable: Boolean = t match
    case _: VirtualMachineError | _: ThreadDeath | _: InterruptedException | _: LinkageError | _: ControlThrowable | _: Break[?] => false
    case _ => true

Now that it’s all inline, the boundary/break will be a jump. (Note that Break is not a ControlThrowable.)

[bruce-wayne]

New user here. I came across this topic after watching Odersky’s talk on YouTube. I just wanted to mention that the apply method shown above generates couple of warnings as follows using Scala 3.4.2.

[warn] 14 |      case Fail(a: A) => Left(a)
[warn]    |           ^^^^^^^^^^
[warn]    |           pattern selector should be an instance of Matchable,,
[warn]    |           but it has unmatchable type T | either.Fail[A] instead

Modifying the code to boundary(body).asMatchable leads to a non-exhaustive pattern match warning. I ended up suppressing the warning using a local @nowarn.

[adamw]

@bruce-wayne

New user here. I came across this topic after watching Odersky’s talk on YouTube. I just wanted to mention that the apply method shown above generates couple of warnings as follows using Scala 3.4.2.

I didn’t test on 3.4.2 (as a library, Ox is pinned to 3.3.x), but the implementation there is changed a bit, maybe it doesn’t have the problem you describe.