Make Null a subclass of AnyVal under -Yexplicit-nulls

Language Design
56

Thank you all very much for your input. I’ve been very busy over the past few months, but I did read and consider everything happening here.

We recently discussed this feedback internally, where I presented what I write in this post, below. We decided that we would go ahead with the change.

(From a process point of view, it will eventually have to come back for more potential discussion when we actually make a SIP for the whole explicit-nulls feature.)

Below, I summarize the major concerns that have been brought up, and address them.

Before diving into it, I’d like to acknowledge @alexandru’s point here:

The fact that (x: Any).toString()—in general, (x: T).someMethodOfAny()—is unsafe is a valid point. However, making Null not even a subtype of Any is definitely not an option for Scala, so that cannot be the solution. Therefore, this concern is orthogonal to the change proposed here. I will address it in a separate post/proposal.

Now, here we go. I will use headings summarizing categories of feedback/concerns, for easy visual parsing. Because that’s a very short form, it may sound cynical sometimes. Please bear with it.

If you feel you voiced a concern that was not appropriately taken into account among the categories below (and is not about (x: T).someMethodOfAny()), feel free to point it out.

We don’t really care about the diagram

Indeed. This was certainly my mistake. I put the diagram first as a justification, and in particular the difficulty of teaching it.

The diagram is but a symptom of a deeper issue, though. The Scala value system is fundamentally divided into values with and without identity (also: unforgeable versus forgeable). AnyRef and AnyVal are a very concrete embodiment of this dichotomy in the type system. The rules applying to AnyRef don’t apply to AnyVal, and conversely. They are the materialization of two different sets of semantics.

If Null cannot be put into either of those categories, it complicates the design. It means the compiler needs to be aware of Null in many places. The spec needs to treat Null specially. It is a liability to the type system core as much as to the surface, diagrammatic view.

If we can meaningfully put Null into either of those categories, it’s a win for the type system, the compiler, and teachability overall. If that does not cause meaningful drawbacks, we should do it, in an effort to simplify the language as a whole.

We won’t be able to say “T can be anything except Null”

True. Currently, we can do this with T <: AnyRef | AnyVal. This won’t be possible anymore.

What is such a bound good for, though? There have been two use cases put forward.

I want to safely call (x: T).toString()

This is definitely valid, and comes back to the point I mentioned before about (x: Any).toString(). I will address it fully in a separate post. For now, recall that safe alternatives exist: String.valueOf(x), s"$x", "" + x, etc. They’re not nice, but they’re safe.

I want to use null internally as a sentinel

This is the Queue[T] example mentioned above in the thread.

Why do we want the Null exclusion? From an API perspective, ruling out Null is clearly a liability. If we can offer the same service while allowing Null for the users of our Queue, it is a win. So it must be for the benefit of the implementation: we don’t want user values to be confused with the null we internally use as sentinel.

The underlying problem is that null is forgeable (like other AnyVals but unlike other AnyRefs). You can’t prevent someone else from materializing a value that is indistinguishable from the null that you want to use as sentinel.

There is an alternative to null as the choice of sentinel that solves that problem: a private object Sentinel.

Instead of


if (x == null) ...

you write


if (x eq Sentinel) ...

and you’re gold. Nobody outside your Queue can forge another value that will be eq Sentinel, so you don’t have to rule out Sentinel from T. And clearly you also don’t have to rule out Null. Everyone wins.

Now some will say: but performance! Comparing to Sentinel (a constant after JIT’ing) is not as fast as comparing to null. null is 0, and processors compare faster to 0 than to other values.

That is true, but is also a very small effect. The only reason one might care is if they are micro-optimizing a particularly hot path. That can happen, of course, but when it happens, you also definitely don’t want boxing! In that scenario, you should restrict T <: AnyRef in the first place. And therefore, whether Null <: AnyVal is irrelevant.

In conclusion, IMO there is no reason for ruling out Null but not other AnyVals.

It feels wrong / null is a reference

We can also say: “It feels right”, and “null is a value”. We can’t justify any of those point of views, though, so it’s kind of moot.

Is there any other language that does not call null a reference?

As a matter of fact, yes: JavaScript.

In JavaScript, the type Null is explicitly part of the primitive types, which are distinct from Objects.

But I don’t think this is a very interesting question in the context of Scala.

A related, more useful question is: does null fit the semantics of AnyRef or AnyVal?

We have already established that it fits the semantics of AnyVal but not of AnyRef. Chief among them is the ability to forge a value that is indistinguishable from null. See also: the above point about the sentinel, and why Sentinel is better than null (because it can’t be forged).

The JVM says null is a reference

Yes, it does. It also says a bunch of other things that Scala models differently:

  • int is a primitive and can’t be used in generic signatures,

  • void has no value (we have Unit and ()),

  • return is a statement, not an expression,

  • etc.

We must not confuse our source language model and the target we compile to. They have different rules. We can and should abstract away the underlying machine if it suits our source language model better (while giving access to the raw performance bits when necessary).

throw null (technicality)

When you write throw null, it throws a NullPointerException. That is true, and is specified. That’s not because null is the same as a NullPointerException. It’s because you can’t actually throw null, just the same that you can’t call null.foo(). The type system lets you (at least Scala without explicit nulls and the JVM type system), but at run-time it’s not a valid operation. So just as when you try to call null.foo(), the JVM throws an NullPointerException when you try to throw null.

The “idiomatic” way of performing throw null in Scala 3 with explicit nulls would be to write null.nn.

7 Likes