Explicit Nulls: Option-like extensions for working with nullable T | Null

Scala Standard Library
22

I should chime in, as I have a large codebase (~45kloc) that has been using explicit nulls since it was introduced (through all its iterations and numerous bugs) and for many years (7 years now), my project is an FRP facade on top of java’s Swing, Qt, Skia, vulkan and web, which uses nulls for a ton of things with semantic meaning.

I second everything said by sjrd here, specially the understanding of principle of least power.

When I wanted optional parameters (and all my widgets have a 20+ parameter list of optional arguments sometimes with defaults), I’ve found that I rather have a

type Opt[A] = A | UnsetParam.type
object UnsetParam

with a small api to use, than using Option or | Null, since swing has specific meanings for null which are different from user unspecified. As mentioned, something like scala-unboxed-option would work here but since the entire api for this is like 10loc I didn’t bother.

In practice, you never want A | Null beyond the field that does store null, you always want to unwrap as soon as possible. I have, over the years used several extension methods to work with nullable types, and now a days, I basically only really use 2 or 3 extension methods:

  • nullFold which is exactly like Option’s fold but for A | Null (all inlined as you’d expect) and
  • unn which is the same as nn but without the check, since there’s not a single time where I’ve found the default nn to make sense, I am already doing the check somewhere else and there’s no situation where I want to discard the nullable part while doing a check that throws a worse NPE (the default JVM will produce a better NPE with an error description showing the dereference that failed).

As you can imagine, nullFold is called like that to be explicit and differentiate it from other folds that might be available due to imports.
The 3rd extension method that I’ll occasionally use is the probably awfully named ? that maps the A in the A | Null. I feel there’s no good name for this as map feels wrong. In fact this method is so awkward that it never feels nice to use, but sometimes (seldomly) I really want to transform T | Null to U | Null.

What I think would really improve usability is to add nullFold, fix nn and do something about vars, because the flow typing support is cute and I’ve effectively used it 0 times. That’s how useful it is as it’s currently implemented :person_shrugging:.
Other than this, you don’t need anything as you don’t need to propagate | Null anywhere, it won’t give you a realistic performance boost and it will make your codebase harder. Better to have a custom 0 value like I did above for the UnsetParam

3 Likes
23

  • A bound like T >: Null <: AnyRef | Null doesn’t interact well with flow typing or with .nn. For example, the inferred type argument for (x: T).nn would still be T (widening to upper bound is not always a good choice). It’s usually better to define T <: AnyRef and add | Null only when you know a null value is possible. This approach is also more compatible with most existing code (including code compiled without explicit nulls), since you don’t need to propagate A | Null everywhere.

  • We don’t plan to change the behavior of .nn. It’s intended as a strict boundary that prevents any null from flowing into safe code. That said, we may provide an “unchecked” variant in the future, similar to what we use internally in the compiler:

    extension [T](x: T | Null) transparent inline def uncheckedNN: T =
  x.asInstanceOf[T]

  • We agree that map, getOrElse, and fold can be useful. A reasonable option could be to provide them in a separate object, so that users can opt in via an import, without affecting code that doesn’t use explicit nulls.

As a side note: we’re also in the process of migrating the entire standard library to explicit nulls. This will give users better type information and make the library easier to maintain.

1 Like
24

I must protest this, as .nn is strictly worse than just not using it and getting a real NPE, which will give useful information. Compare:

scala> val s: String = null
val s: String = null            

scala> s.stripPrefix(">")
java.lang.NullPointerException: Cannot invoke "String.startsWith(String)" because "$this" is null
  at scala.collection.StringOps$.stripPrefix$extension(StringOps.scala:735)
  ... 67 elided
                                                                                                                                                                                                                                                                          
scala> s.nn.stripPrefix(">")
java.lang.NullPointerException: tried to cast away nullability, but value is null
  at scala.runtime.Scala3RunTime$.nnFail(Scala3RunTime.scala:28)
  ... 67 elided

When you are chaining operations on things that might return null, the JVM one will tell you where in the chain it failed, .nn will give you a headache.

1 Like
25

Would it be possible to implement .nn using some kind of macro so that the exception message would provide some information about the receiver of .nn, like some implementations of assert do?

26

Instantiatiing type parameter A as X | Null will stealthly bring null values to generic Scala code, which, more often than not, does not check for nullity. It’s a good way of dealing with Java API, but I think should be actively avoided, as it will cause any call x.toString to throw a NullPointerException. Therefore we should make it harder to use null values, not easier.

I think a much better solution is an opaque type with Option API extension methods. You can name it Opt or ?? for brevity and provide an implicit conversion T => Opt[T]. This solves all your issues with zero changes to the language. You can even use None as the empty value:

opaque type Opt[+T] >: None.type = T | None.type | BoxedOpt[T]

object Opt:
  def apply[T](value :T) :Opt[T] = value match {
    case null           => None
    case None           => new BoxedOpt(value)
    case _ :BoxedOpt[_] => new BoxedOpt(value)
    case _              => value
  }

private class BoxedOpt[+T](val value :T) extends Serializable

BoxedOpt addresses @sjrd ‘s qualms.