Pre-SIP: Checked Exceptions

@Ichoran First, let me thank you for taking this seriously and replying with a constructive criticism. It is clearly not trivial to do so given how controversial this topics is – as seen by other replies here – and I very much appreciate your way of conduct.

Although I disagree on some of the problems you listed, I do agree with the general notion that the proposal will introduce far too many new constructs and rules, serving only a particular use case.

I believe this use case is rather common, and that existing tools do not satisfy the need for a clearer code. I think I have another idea, though – less intrusive – which I will explain in a separate comment.

I did not, as I’m not familiar with Rust all that much. I did however examine it a bit now that you’ve mentioned it, and I agree that is quite a neat feature (so thanks for the reference!).

I still believe it is not sufficient – for two reasons – but then again I’m not entirely familiar with this feature, so correct me if I’m wrong.

The first reason being is that this feature is not entirely compatible with higher-order functions. For the sake of demonstration, let’s imagine that this operator exists in Scala:

def foo(): Result[Int, Error] = {
  unsafe()? + 2
  // is equivalent to
  val i = unsafe() match {
    case Ok(i) => i
    case Err(e) => return Err(e)
  }
  Ok(i + 2)
}
def unsafe(): Result[Int, Error] = ???

This surely reduces a lot of boilerplate, and it might b e worth having this feature – perhaps via macros? – regardless of error handling.

When we consider code with higher order functions, though, this feature cannot help reducing the boilerplate the same way exceptions do:

def foo(): Result[Int, Error] = {
  Seq(1,2,3).map(i => unsafe(i)?).sum
}
def unsafe(i: Int): Result[Int, Error] = ???

This example will not compile, as the anonymous function (unsafe?) returns Result of Ints, and sum is not defined on non-numeric items. It’s not necessarily a problem with the sum method in particular, but a general problem of seamlessly “reducing” a container of Results to a singular one.

It might be possible to overcome this problem by making the the operator behave differently in anonymous functions. Instead of making the function return type a Result, its return type should be the valid result, but it should still use return for the invalid result (err), and would still require the outer function to return a Result.

Say, if the compiler was smart enough to translate the above example to this:

def foo(): Result[Int, Error] = {
  val is: Seq[Int] = Seq(1,2,3).map { i =>
    unsafe(i) match {
      case Ok(i2) => i2
      case Err(e) => return Err(e)
    }
  }
  Ok(is.sum)
}
def unsafe(i: Int): Result[Int, Error] = ???

In which case the return actually throws a NonLocalReturnControl exception (if I’m not mistaken), and this functionality is said to be dropped in Dotty, so I’m not sure whether this is feasable. Also, I’m not entirely sure whether the relaxation of the “must return Result” restriction for anonymous functions would not incur other problems.

The second problem with this mechanism is that it doesn’t solve the boilerplate of multiple levels of nested exceptions:

def compile(rawScript: String): Result[Script, Err] = {
  val script: Result[Script, Err] = {
    if (rawScript.isEmpty) return Err("script is empty")
    val header = compilerHeader(rawScript)?
    ...
    compose(header, body, ...)
  }
  script.expect(s"Failed to compile script: '$rawScript'")
}

def compilerHeader(rawScript: String): Result[Header, Err] = {
  val rawHeader = rawScript.substring(0, 15)
  val header: Result[Header, Err] = {
    findBadWord(rawHeader).map { badWord =>
      return Err(s"bad word found: '$badWord'")
    }
    ... // other unsafe operations
  }
  header.expect(s"Failed to compile header: '$rawHeader'")
}

def findBadWord(s: String): Option[String] = ???

This code is a bit more cumbersome than the exception-based one; but more importantly, the explicit returns will ignore some of the expect messages so they won’t be included in the final result.

To clarify, I would expect an error message like so (could be reversed order):

bad word found: 'voldemort'
-- caused: Failed to compile header: 'voldemort rules'
-- caused: Failed to compile script: 'voldemort rules, not dumbledore'

Edit: To correct what I said earlier, nested returns are not dropped in Dotty, but merely replaced with a non-keyword mechanism that is slightly more verbose (the underlying implementation still uses exceptions).

Also, it’s probably worth noting that exceptions are only intended for error handling in sequential code. Concurrent code has other solutions (coroutines, messages, etc), albeit construction of multiple levels of nested errors is problematic (this is due the nature of concurrent code).

Inspired by Dotty’s replacement for non local returns, I created a Result that could potentially overcome the problem of multiple levels of errors:

def compile(rawScript: String): Result[Script] = {
  returning { implicit c =>
    if (rawScript.isEmpty) err("script is empty")
    val header = compileHeader(rawScript).?
    compose(header).?
  }.expect(s"Failed to compile script: '$rawScript'")
}

def compileHeader(rawScript: String): Result[Header] = {
  val rawHeader = rawScript.substring(0, 3)
  returning { implicit c =>
    findBadWord(rawHeader).map { badWord =>
      err(s"bad word found: '$badWord'")
    }
    new Header
  }.expect(s"Failed to compile header: '$rawHeader'")
}

def findBadWord(s: String): Option[String] = Some("bad").filter(s.startsWith)
def compose(header: Header): Result[Script] = Ok(new Script)

A full working example is available in this scastie (Scala 2).

I think this still need some revising. I don’t like needing to declare the implicit controller each time, nor do I like that err is just a method and not a keyword; after all, it stands for a control-flow mechanism and hides a throw call.

Coincidentally enough, this could look a lot like my “other idea” that I mentioned before. I had the intention of explaining that idea, but this experimentation with Result has led me to a similar solution.

Uh, no, if you expand x.? in Scala into x match { case e: Err(_) => e; case Ok(x) => x } then in fact unsafe(i).? is an Int. It works this way in Rust and in my macro in Scala.

So it works fine already. It never has a Result return type; ? is specifically designed to get out of the result type.

Your initial example should be unsafe().map(_ + 2), and then everything is fine.

Yeah, that’s how it works, and the feature is still there, just gated by wrapping with a returning {} block.

Also, you don’t need a separate Result type. Either works fine. (You can make ? work on Option too. I do.)

Isn’t the return type of such an expression is Int | Err?

That is, unless the err case returns the err: case e: Err(_) => return e;, which is what I showed in the next code example (“say, if the compiler was smart enough…”).

How so? The intent is to map a sequence of integers using an unsafe function, sum the mapped integers, and increment that by 2. I’d like to have an easy way to make this transformation:

val from: Seq[Result[Int, Err]] = Seq(1,2,3).map(unsafe)
val to: Result[Seq[Int], Err] = transform(from)

Which would be possible if the compiler will be “smart enough” to translate the ? in the anonymous function to code with return e. I mean, that is the default code it’s being translated to, I’m just not sure if this happens with anonymous functions as well.

Yeah, I corrected myself later. I’ve also used this pattern as an inspiration for something tailored specifically to Result.

Yeah, I don’t think I mind which one will be used eventually.

Yes, that’s exactly the point of the feature, in Rust as well as Scala.

In Rust, Result types are ubiquitous but there are only local returns.

Yes, that’s how it works. Right(Seq(1, 2, 3).map(unsafe.?).sum) is enough.

(Scala is not very good with postfix ops, so I use it as .?.)

(You can add a method to make things adopt the right branch, e.g. Seq(1, 2, 3).map(unsafe.?).sum.inRight.)

@Ichoran then it really look like a solid solution for these specific use cases of error handling; it seems that checked exceptions can safely retire (or die? they are already retired).

I’m also seeing that there is already a thread on rust-like results, so I’ll go read it.

Thank you for taking the time to help :slight_smile:

Sorry to jump in this discussion, (maybe the other link discussing Result is a better place to ask) but, even though I totally agree with @Ichoran about Result (or Try or Either) being a very good way to deal with errors, could we watch this thread from the Java interop point of view?

Would it be possible to imagine a compiler feature that turns Java’s checked exceptions into this Result we’re talking about ? (and what would happen if the Java code can throw multiple exceptions?).

I’d be really interested in such a feature, when we think about it, Java’s checked exceptions are a first step into “encoding the error into the type system”. Far from being perfect, but I think having an automatic feature to turn everything we can “understand” from Java’s libraries (non-nullable, nullable, side-effects, etc.) would make interop so great.

Please let me know if this is completely out-of-topic, or simply bad, dillusional or whatever :slight_smile:

Thank you.

EDIT: I don’t want my comment to sound as a “what if”. There’s certainly a good reason why checked exceptions cannot or should not be turned into such a Result type. Just a pointer to a good doc, a paper, or something else would be a nice answer, thank you.

1 Like

@aesteve - Try already catches exceptions.

If you want to catch just the checked exceptions, you need type unions, because you can get any one of the checked exceptions. So Scala 3 could possibly support this.

I’m just not sure it’s worth it. It’s quite a bit of compiler tooling and a bit of complexity in the language just to support a feature that overall isn’t as good as something that we already have.

2 Likes

Glad you found it useful!

I should point out that although I wrote my own .? macro and use it ubiquitously, this style of programming is not common in Scala. However, as a proof of concept that we have better-than-checked-exception capability in Scala, it works nicely. (And though it’s not common, basically all of my code uses it because it’s so effective at dealing with errors.)

2 Likes

Yes for sure, I could not tell if it’s worth the effort or not :\

It sounds “magical” (in the good sense of the term) and very useful to me.


A little back-story, in case it may help:

We’ve been working in Scala on some module that relies on an old Java module of ours. The old module dealt with data Validation through JSR-303 and added checked exceptions on top of the ValidationExceptions.
Obviously an Either-like system (or Valid/Invalid like in Vavr) would have been better here. But I can’t blame people who wrote this module ages ago to have, at least!, encoded possible errors in the Exception system. That’s all they had at this time, and to be fair, that’s really better than RuntimeExceptions, or some weird binary-encoded return code.

When we dealt with this dependency in the new Scala module, we were a bit disappointed to have to wrap every call to multiple methods within a Try. In fact, we started to feel really reliant on the compiler that helped us so much and suddenly that “compiler-as-a-security-nest” felt apart.


I’m definitely not the right person to argue in favor of such a feature, maybe it’s not worth the effort, maybe it’s not desirable at all, but hopefully the example we faced will be helpful.

As you said, when we first had a look at union types we thought about checked exceptions and it “felt like” some glue could have helped us automatically.

Thanks for your answer :slight_smile:

I’ve already experimented with early returns from for-loops over Eithers:
https://users.scala-lang.org/t/dont-use-return-in-scala/3688/24?u=tarsa
https://users.scala-lang.org/t/dont-use-return-in-scala/3688/35?u=tarsa

That was for the unbiased Either in Scala 2.11-, but it should be easily convertible to biased Either in Scala 2.12+ (I can do that if you want).

The machinery also could probably be simplified, e.g. Either[Either[Error, Result], Result] could be changed to EarlyEither[Error, Result] that has methods like def flatMap[B](A => Either[E, B]) (to allow composition with ordinary Eithers) and def extract: Either[E, A] (to designate early return boundary).

Automatically translating a java method to an Either[Coproduct of declared throws, Result] sounds kind of neat, TBH.

3 Likes

Keeping up such a fiction in the face of overriding and implementing Java methods sounds really hard though. And it would require Scala to be way more opinionated about how a developer is supposed to do error handling.

4 Likes

2 totally fair points.

1/ (It’s opinionated and should not be a first-class-citizen): absolutely. That could be something we could try to implement, test, and propose afterwards? Is it even possible? Any helpful pointer (macros should be the thing to look at?) at some implementation / module / compiler plugin / … that would do something similar we could look at to get started would help a lot.
2/ (It’s hard) : I’m trusting you on this. So we have to try-and-tell (thus => question 1).

We’d be happy to try and contribute something like that, if someone has pointers to share, it’d be really awesome.

Thanks a lot.

EDIT: (to avoid the noise of another message). Thanks @LPTK for pointing all of the issues. And sorry for the inconvenience.

I agree that it doesn’t seem feasible in general.

Consider this Scala function:

def test[T](x: { def foo(): T }) = x.foo()

Then I could call test with a Java object o whose foo method throws, which according to the Scala compiler fiction would mean returning Either. So Scala will tell you test(o) has type Either[...] when in reality it just throws, and there is no specific place for the compiler to insert the exception-to-either-conversion logic.

Another example: when implementing, in Scala, a Java class with an abstract method that throws, what should the compiler do? It would have to silently make the Scala implementation actually throw (to comply with the Java method’s runtime semantics and erasure). Then it would have to somehow mark this Scala method as Java-throwing under the scenes, so callers deal with it accordingly. But all this would make the Scala method actually incompatible with a Scala method that properly returns an Either, so you couldn’t at the same time implement a Scala interface with it, although the Scala types would seem to match.

As compiler fictions go, this one would be on the difficult-and-leaky side.

6 Likes

Hey, can I revive this discussion a bit? I skimmed over it and I personally think it got side-tracked into a Either/Result-oriented design discussion. Let’s bring it back to specifically checked exceptions and maybe try to simplify a bit.

My suggestion is this: have an analysis tool which checks Scala code for use of the throw keyword without a corresponding try ... catch which catches that exception in the same method. If it finds such usage, log a warning. If the method that uses throw is annotated with a @throws[...] annotation for the same exception type that’s being thrown, then don’t log a warning. This way exception warnings are propagated (or not) throughout the call graph in the same way as exceptions are.

To be clear, this is not my original idea, it is a pretty much straight up copy of Reanalyze from the OCaml ecosystem: reanalyze/EXCEPTION.md at master · rescript-association/reanalyze · GitHub . That document is short and very much worth reading.

The advantages of using the existing @throws annotation:

  1. No new keywords needed
  2. Codebases which use the annotation already will automatically work with this checker

The checker itself can be implemented in a few different places: as a Scala compiler warning, or perhaps an sbt plugin, or maybe even just a completely separate tool. In fact, reanalyze could maybe even be ported over to analyze Scala, since someone has written a Scala parser for OCaml: pfff/Parser_scala_recursive_descent.ml at develop · returntocorp/pfff · GitHub

How is that different from statically checked exceptions in Java?

1 Like

I think not substantially different. To me it seems like the reasons why people hate Java checked exceptions–being forced to deal with them in every method, and lack of composeability–are either considered virtues or mitigated by certain patterns in Rust-style Result<T,E> error handling, so perhaps what people really hated was the fact that it was baked in and could never be turned off? In which case it would probably make sense to implement it as either a separate tool, or as a compiler warning, say -Ywarn-exceptions.

Have you looked at how ZIO handles this issue; i.e. an explicitly typed error channel?

Yes, I consider that a form of Rust-style (Result<T,E>) error handling. I would like to, as I said earlier, bring this discussion back to specifically about checked exceptions.

1 Like