Pre-SIP: A New Type for Optionals and Error Handling (2)

The issue of handling missing / alternative values with Option and friends has been a source of disappointment for quite some time, especially if we look at the alternatives other modern languages provide. The community has been throwing libraries and proposals at it (including my own proposal to empower T | Null) but it never felt like we nailed it.
And despite the grass being greener on the neighbours side, IMHO no programming language managed to provide a fully unified and satisfactory solution so far.

This proposal is really refreshing: it’s ergonomic, holistic and innovative. In short, the true spirit of Scala ! And I’m sure it’ll serve as a lighthouse for other languages as has been the case with a several language features in the past.

While we’re at it, we have an opportunity to take some of the best productivity features and sugars programmers love that have proven their worth across popular languages. It’s great that the postfix ? operator has already been proposed, here are some other additions that come to mind:

Feature parity with Option

fold, get, contains, exists, … All the familiar methods we love from Option (and maybe others ? The Rust API could be a good source of inspiration) would make the feature more ergonomic and help with the transition.

?? AKA nullish / nil coalescing operator

A great companion to the ? operator. Languages which have the latter usually provide the former.

def isItGreat(input: String?): String = input ?? "Absolutely!"

Available in JS / TS, Swift, C# and Kotlin (as ?:). Functionally equivalent to getOrElse but shorter and probably clearer.

Aborting to the enclosing function by default

Could we have the best of both worlds ? maybe is great when one needs that flexibility, however the most common case will probably be Rust-like, i.e. aborting to the enclosing function. Would it be feasible / desirable to have that as the default (given that the return type matches) ? This obviously would spare us some ceremony and syntax overhead (especially for one-liner functions) but mostly would make the feature easier and more accessible to beginners.

I think this would get ambiguous too quickly, since we also allow code that omits the maybe but passes a CanErr parameter (in which case the abort would go to the caller). In terms of notation it’s not more verbose than Rust since Rust demands a wrapper for the result. So if we adopted Rust’s scheme we’d write:

def f = 
  ...
  Ok(result)

whereas in this proposal it is:

def f = maybe:
  ...
  result

It’s the same number of calls in the end.

2 Likes

So I note that JEP 401 is now in Java 28 as a preview feature. Is there anything in this feature that would likely preclude future integration with Java value classes?

3 Likes

Still digesting this, but from a usability standpoint, I’m mixed. Its clean, and I like the explicit T ? Err, which is less cluttered and clearer than Either (and the order doesn’t feel backwards). I worry that T? meaning T | null sends the message that ā€œnulls are ok nowā€, after years if steering users away from them—and Scala being relatively free of NPEs as a result. Maybe you save the Option wrap drama but you still need the match on result. Will the compiler warn you if you forget to match and blindly try to use result T?

4 Likes

That’s why this only makes sense when paired with Explicit Nulls. As I understand it, Explicit Nulls neuters the traditional problem, which implies that it is now possible to use null in productive ways.

2 Likes

But that’s why it also shouldn’t be used for any serious library at this point (or soon), because only a subset of the ecosystem uses explicit nulls. If we deprecate anything but explicit nulls, then sure, use ?. But otherwise it should stay niche.

Or, alternatively, it needs to make sense without explicit nulls. Introducing a new decoration T! would do the trick, because then T for a type not provable to be disjoint from null would act akin to T | Null, T! would act like T, and T? would turn T into T! or whatever the sentinel is (which might be a no-op). So all the key pieces would be isomorphic, more or less, just accessed differently.

1 Like

I agree this cannot go in before explicit nulls are standard. There’s hopefully going to be a SIP soon for explicit nulls.

Right now, the implementation requires -Yexplicit-nulls to be set for ? to be parsed without error.

7 Likes

Today’s SIP meeting voted to accept the proposal as SIP 84 into the experimental design phase. There’s PR 26956 with an implementation draft that makes it an experimental feature.

Once that PR is merged, people are invited to experiment with the feature and give feedback.

1 Like

Was the risk for ecosystem fragmentation discussed ?
And if so what was the result ?

While this concerned has been raised here, I have not seen it being addressed by anyone in favor of this proposal
(but I might of course have missed it)

2 Likes

It was not voted to be experimental yet, just accepted into the process and given a number.

Yes, my bad. There will have to be a review and another vote for that.

I think that to reduce the impact of ecosystem fragmentation, it would have to

  1. Have good interop with Option and Either
  2. Work with non-explicit nulls
  3. Be heavily supported in the standard library from the start

The reason for the first is obvious, hopefully–reduce the reason to pick Option and Either for new work because old libraries need those.

The second reduces fragmentation within the leading-edge ecosystem. I think there is actually nothing about T ? E that requires explicit nulls, because T as a subtype of Any which includes null just means that you are agnostic about dealing with them, while T? means that you declare you have an explicit awareness of the possibility. There’s less point because you go from T? to T without the non-nullity of T documented in types, but it ought to at least work.

The third one gives a reason to use it rather than to stick with the old way: you gain lovely capabilities that you don’t want to miss out on. For instance, map.at("key") would be a V? so you don’t have boxing of the success case. (Whether at or something else, I’m not sure; can’t be get because that’s Option already, unless compatibility is so good that it wouldn’t even matter. But I don’t think that’s practical.)

So this could still result in a style mismatch, but I think full-on fragmentation would be rare.

3 Likes

Seth suggested that this might be a better venue so I will copy my comment here about the nomenclature, regarding result T or E as the way to say T ? E:

What if we pronounce this ā€œMaybe T, otherwise Eā€ or somesuch? I think the result T or E naming is awkward. Firstly it’s not clear whether it should be parsed mentally as (result T) or E or result (T or E). Secondly, neither really matches the construct. In the first–why is E not a result? In the second–what does the result wrapping of T or E mean…can’t you un-result it and still have the underlying T or E? (No–because that’s not how it works.) And then the ? sigil suggests that the two constructs are related, but the spoken form does not.

Anyway, it’s very much a bike-shedding concern, but I think it is awkward to speak about as introduced here.

If we like ā€œresultā€, result T else E is probably better than ā€œorā€ because T else E doesn’t sound like its own thing nearly as much as T or E does.

Note that despite being bike shedding, how you say a thing is part of the API surface, so it deserves careful attention, at least for as long as we talk about code. I suppose if LLMs are writing all code nowadays, it doesn’t really matter what it’s called.

3 Likes

Quoting the SIP:

There is a potential issue with right-biased higher-kinded type inference.
Matching a type constructor F[_] with a type argument R ? E would infer
[X] =>> R ? X, whereas we would usually want to have [X] =>> X ? E inferred instead.

Right biased higher-kinded type inference was arguably a design mistake caused by overfitting to the Either type and blindly copying Haskell. In Haskell, right bias makes sense because type parameters are curried, but in Scala and most other languages it is unnatural.

One could work around the problem and special case ? to use left-bias instead. But this is not proposed as part of this SIP.

I think this issue must be resolved before this is shipped. Shipping this without a solution would be outright hostile to those of us who rely on typeclass based infrastructure libraries like cats.

6 Likes

+1 to finding a solution for higher-kinded type inference of ?. Regardless of whether the current bias was a mistake, it is how Scala works. Having ? not work properly with HKT inference will be a baffling Scala wart to developers that run into it.

3 Likes

This is a sentiment shared by more people than who interact with the contributors forum:

The three :backhand_index_pointing_up: users to my knowledge do not have contributors accounts (or at least their Discord profiles differ from their contributors forum profiles).

1 Like

I agree with Rex that introducing the word ā€œresultā€ is unnecessary and undesirable. We can just say ā€œmaybe T, unless Eā€, probably often shortened in conversation to ā€œT unless Eā€. Let’s not have both ā€œmaybe typeā€ and ā€œresult typeā€; more terminology muddies the waters, fewer new words keeps things clear and lean.

But note I’m suggesting ā€œunlessā€ over ā€œelseā€, because it conveys a stronger flavor of E being the unhappy path.

4 Likes

Scala noob here. I am wondering if it’s better that Null has a constructor that is not null. That way I can have a code base that has no nulls syntactically.

1 Like

I endorse ā€œunlessā€ unless ā€œelseā€, so that if something goes wrong, we fail over to ā€œelseā€.

there will be Err(()) == null, but do you mean something like Null?