Make Null a subclass of AnyVal under -Yexplicit-nulls

Language Design
41

We have AnyKind already so Null <: AnyKind could also be possible with not sub classing Any. But then I guess be need to allow values still to be of type Null

42

I am with @sjrd here. AnyVal is the obvious superclass of Null. null is a value, but not a reference (in the explicit-nulls model). All values that are not references are under AnyVal. The only things that are under Any bit not AnyVal nor AnyRef are universal traits that can be implemented by both value and reference classes.

1 Like
43

that’s a strong opinion.

is there a language with proper nulls (i.e. null pointers or references, not null-like stuff like empty optionals) that doesn’t define null as pointer or reference?

java calls null a reference and a value:
Chapter 4. Types, Values, and Variables .

The null reference is the only possible value of an expression of null type.

c# calls null a reference and a value:

The null keyword is a literal that represents a null reference, one that doesn’t refer to any object. null is the default value of reference-type variables. Ordinary value types can’t be null, except for nullable value types.

c# has explicit nullability mode. i haven’t dug deep if the definitions change under explicit nullability mode, but the null keyword page doesn’t mention such special cases.

there are two options: either the scala beginner should be prepared to deal with nulls or not:

  • if not, then simply redraw the diagram to make nulls less prominent and don’t elaborate on null
  • if yes, then lying that null is not a reference is counterproductive

imho the people that are most likely to run into null-related exceptions are beginners, irrespective whether nullness is explicit or not, so presenting some weird leaky abstractions to them won’t make them less affected by nulls. as i’ve written previously:

expand to see previous common type of example of unintentionally running into nulls

from my point of view, simplifications are sometimes more detrimental than helpful. good example is monads. when i was trying to grasp what’s the deal with monads, i’ve read many tutorials and most often they used some weird (sometimes even childish) analogies that didn’t help at all.

2 Likes
44

Well, certainly in the regular case, null has multiple (diamond) inheritance with every reference class; Null is the bottom type of the hierarchy that has AnyRef as its top type.

With explicit nulls, whether or not you put null under AnyVal, you cut all those inheritance links. But it is an actual instantiable value, so you have to put it somewhere. You can match on it, so it has to be Matchable. The choices are really only (1) AnyVal or (2) its own thing directly under Matchable.

Then the question is that if you choose (1), do you swap the inheritance from AnyRef to AnyVal on explicit nulls, or is null always part of AnyVal and the only question is whether it also has diamond inheritance with AnyRef. So do you add a link, or only cut them?

Those are really the only two questions left. The issues about types lying will still be there; those are orthogonal. If String might be null, it should be typed as String | Null. Will it be, while genuinely always-instantiated string values are typed as String? The problem with the non-lie Null is that it is not a helpful truth. People rarely check nullness anyway, because the happy path is the one where you have ensured that the values have no nulls. But if that is intent, why not match the intent with the type?

45

i’m not an expert but i think you might have this wrong. in java java.lang.Object seems to be the root class, not the root type. boxed types like java.lang.Integer come under this root but primitives like int (meant to be represented in scala by AnyVal) have no common root. so in fact java.lang.Object =:= AnyRef and really your statement is an argument for removing Null from the AnyRef hierarchy and not an argument against putting it under AnyVal

46

You’re right that in Java int doesn’t inherit from java.lang.Object but that’s because it doesn’t participate in any subtyping scheme. If you say “root”, you imply subtyping.

Theoretically, java.lang.Object is AnyRef, but that’s paying lip service, because in fact Any is java.lang.Object, much like how Scala’s Int is getting boxed into java.lang.Integer, from Scala’s point of view, int existing as a backend optimization.

To be clear, I don’t really care about the hierarchy, what I care about are these 2 use-cases:

  1. The ability to easily say “these are any kind of object, but not null” (concrete usecase being the queue implementations of JCTools)
  2. Compiler banning any method calls on nullable references — If you make null an AnyVal you cannot ban methods such as toString or equals on nullable references (Liskov substitution principle), so as long as Any has this definition and as long as null <: Any, the “explicit nulls” feature doesn’t make null-usage safe, being half baked.

Kotlin solves these 2 cases for me. Java’s JSpecify does as well.

1 Like
47

Theoretically, java.lang.Object is AnyRef, but that’s paying lip service, because in fact Any is java.lang.Object, much like how Scala’s Int is getting boxed into java.lang.Integer, from Scala’s point of view, int existing as a backend optimization.

i don’t think i agree that Any =:= java.lang.Object simply because primitives are sometimes autoboxed (i assume that’s what you’re getting at). what is really happening is that Int <: AnyVal | AnyRef, ie. it is sometimes a value and sometimes an object. that doesn’t make all types really object types at all. and remember this is only true internally because of inadequacies in the jvm (inadequacies that people are trying to resolve). on a different backend the primitives might be true primitives and not have the same problems (this is mostly the case for scala.js i believe). i don’t reckon it’s a good idea to entrench jvm inadequencies in the general type system except where absolutely necessary (this mentality is why scala has AnyVal and the unified type system, not untethered primitives)

Compiler banning any method calls on nullable references — If you make null an AnyVal you cannot ban methods such as toString or equals on nullable references (Liskov substitution principle), so as long as Any has this definition and as long as null <: Any, the “explicit nulls” feature doesn’t make null-usage safe, being half baked.

agreed on point #1 but why do you feel that null has to not respond to methods? it feels like you are equating Null with Nothing (ie. an absence of any value). null is a real thing so has to come under Any. it has a data representation (0), a string representation (null in most languages), and can be equated. so to me it’s far more bizarre that it doesn’t respond to most normal methods (this feels like a mistake inherited from java). having these methods would not only be honest about what null is but it’d also be helpful in some cases. and banning all methods on null also wouldn’t really solve the problems people encounter. most methods are banned right now (it only supports eq and asInstanceOf and i’m not sure what else) and yet it seems pretty uncontroversial to say that there are issues with null that need to be resolved. null responding to methods when you expect it to explode instead would merely be a symptom of a larger problem, not the problem itself. and dragging it out of from Any and untethering it will just cause issues in the same way primitives in java cause issues (surely this is the ultimate source of the boxing issue: implementing primitives as untyped and separate from true types and thus unusable in generics)

what it sounds like you want is the ultimate goal of the ‘explicit nulls’ work. once nulls are properly accounted for in the type system and it’s honest about where nulls might appear then nulls should become simple to deal with and the problems should mostly disappear. whether it responds to methods is neither here nor there

btw, looks like kotlin doesn’t give you those 2 things:

>>> null.toString()
res0: kotlin.String = null
>>> null.equals(null)
res1: kotlin.Boolean = true)
>>> null.hashCode()
res2: kotlin.Int = 0
48

That’s because it has convenient extension methods defined in the standard library. Here’s how it looks like:

public inline fun Any?.hashCode(): Int = this?.hashCode() ?: 0

Those extension methods are not virtual methods defined on Any?. If you try accessing the actual methods, Kotlin doesn’t let you, because Any? is not Any and doesn’t have hashCode() defined. Good example though. See how null.hashCode() behaves in Scala and compare.

49

well, that’s kinda interesting, but i don’t see the relevance here. it shouldn’t matter where the method is implemented. the reality is in scala null blows up on most methods and you seem unhappy with it, but in kotlin it responds to more methods and you seem happy. so responding to methods can’t be the problem surely

the key issue in how it handles null seems to be the ‘Any? is not Any’ part. in kotlin it’s truthful about the nullability but scala isn’t (yet) truthful. once it is then the same benefits should come to scala. perhaps, if you don’t already, it would be worth turning on ‘explicit nulls’ to see how it works for you and report back any problems. i don’t know myself but it seems the right path

50

to talk about the original post for a moment, if i ignore the graph confusion part and focus on the core idea of Null <: AnyVal then i’m more and more convinced it’s a good idea

null is a real value (ie. stored in memory), represented by a 0 in 4 or 8 bytes on most systems. it is seen as a pointer but it’s not inherently a pointer (indeed it points to nothing). rather it’s a marker for the absence of a pointer. it is conflated with a pointer-to-data / object type by traditionally being stored in the same word of memory (essentially a classic union type, done for simplicity and performance, not for likeness of types). it’s true nature is as a unitary value and so nulls have a common immutable value, not individual instances (akin to 0 : Int or true : Boolean and not to AnyRef). as it’s unitary it’s minimal representation would be a 0 in a half bit (ie. unary digit / unit) if such a thing existed. no more space is required. it’s easily stored on the stack directly and has no associated heap space, again like an AnyVal and unlike an AnyRef. the idea of a boxed null is a meaningless concept given it’s uniqueness within the jvm as something that is essentially typeless. but i guess you could say it’ll never be boxed on any backend and be correct, so again like any strict value type. to me it is so very like Unit that they almost feel like the same thing: a single value for when there is no other useful value but we need something (maybe just remove Null and do val s : String | Unit = () instead? save us the hassle :grin:)

so overall it feels very value-like. it could easily fit directly under Any but it feels to me to match with AnyVal more

1 Like
51

But it does. We are talking about the subtyping hierarchy. Any in Scala has the methods of java.lang.Object, Any? in Kotlin does not. Not sure what you’re trying to argue for here to be honest. Even if you introduce extension methods in Scala, which isn’t possible currently BTW, or extra linting checks, what you’d still have is this:

val x: Int | Null = null
// upcast is perfectly legal due to subtyping relatioship
val y: Any = x 
y.hashCode()

Scala is unsound here for as long as Null <: Any, whereas Kotlin is not.

If you go down that path, then all types are imaginary, it’s all zeroes and ones and we should all get back to programming in C and just cast everything to void*.

null is a value, yes, with its own type that shouldn’t inherit from java.lang.Object, and in Scala it does.

Unit is not the one billion dollar mistake, null is, and the reason for that is because it can be a subtitute for types without being able to perform their function (Liskov substitution principle is violated).

You can wax poetic about values, and Unit and so on, but facts are facts and one fact is that -Yexplicit-nulls is currently flawed and will remain flawed until Null <: Any stops being true. And in the meantime, Kotlin has a better design and Java has working solutions, such as JSpecify, and even their JEP proposal and preview feature doesn’t fall into Scala’s trap.

And the current proposal basically does nothing to address any issues in actual programming, other than … making the type hierarchy look nicer I guess? :man_shrugging:

3 Likes
52

what if Null is kept as direct subtype of Any but java.lang.Object’s methods are removed from Any? these methods can be reintroduced in AnyVal and AnyRef types, but not Null. then calling java.lang.Object’s methods on Null wouldn’t be possible, which would be an upgrade from current situation. additionally, scala’s stdlib could define type NonNull as either AnyVal | AnyRef (assumming both types guarantee no nulls) or an additional trait with only AnyVal and AnyRef as subtypes, i.e. Null wouldn’t be a subtype of NonNull trait.

null values are needed for compatibility with foreign code, i.e. java, javascript, native ffi, etc. otherwise, scala code should be maximally null-free. making Null a subtype of AnyVal removes no-nulls guarantee from code that operates on AnyVal and makes defining NonNull type harder.

rust has nulls only in the form of raw pointers, but raw pointers can’t be dereferenced outside of unsafe blocks. scala doesn’t have the notion of dereferencing a raw pointer, so instead of prohibiting explicit dereferencing, scala can just remove all java.lang.Object’s methods from Null type.

4 Likes
53

Not sure what you’re trying to argue for here to be honest

you said you care about 2 use cases. i’ve been asking you how your use case #2 makes sense / matters at all. that’s basically it. i still don’t know why you think it matters (i don’t think it truly matters either way so i am making no argument there)

Any in Scala has the methods of java.lang.Object, Any? in Kotlin does not

Any doesn’t respond to all the methods from java.lang.Object, only a subset. the subset that presumably every value could respond to, not just refs. it doesn’t have any of the thread methods for instance. that’s because Any !=:= java.lang.Object

val x: Int | Null = null
// upcast is perfectly legal due to subtyping relatioship
val y: Any = x 
y.hashCode()

Scala is unsound here for as long as Null <: Any, whereas Kotlin is not.

this could and should be perfectly fine and it only isn’t because, bizarrely, null doesn’t respond to the methods the types say it should, thus breaking type guarantees. that’s the fundamental problem there. it has nothing to do with Null <: Any. going back to your use case #2, you seem to have shown that null not responding to methods is the direct issue, not the solution

If you go down that path, then all types are imaginary

i only said that to contrast it with Nothing (that has no value) and AnyRef (where the data isn’t the value itself)

Unit is not the one billion dollar mistake, null is, and the reason for that is because it can be a subtitute for types without being able to perform their function (Liskov substitution principle is violated).

it is the billion-dollar mistake precisely because it was a mere untyped and unchecked value conflated with values of a different type. whilst scala has mistakes of its own that need to be fixed, it’s a different situation

You can wax poetic about values

i gave the real qualities of null to support it’s placement under AnyVal. there’s no poetry anywhere. feel free to actually address the points instead

facts are facts and one fact is that [it] will remain flawed until Null <: Any stops being true

that’s not a fact. that’s an opinion. there are multiple possible ways that this could work well and kotlin’s way, whilst better than scala at present, isn’t the one true way

-Yexplicit-nulls is currently flawed

you might’ve written this somewhere else, so forgive me if you have, but perhaps you could explain how it’s flawed so that your issues can be addressed

And the current proposal basically does nothing to address any issues in actual programming, other than … making the type hierarchy look nicer I guess? :man_shrugging:

agreed. if that’s the sole purpose here then there’s probably little point. the important work is in the ‘explicit nulls’ project: moving null out from under AnyRef to literally anywhere else (resolves the ‘null not responding to methods it claims it does’ issue), and ensuring a value can’t be null unless it’s part of the type

and with that i shall bow out

54

NPE as a boxed null :smiley:

55

I believe it is important to be able to describe types which are not null, otherwise your code will need to become polluted with unnecessary null checks.