javascript is dynamically typed and there is not even a proposal for enforced nullness marking syntax. since most of the typing is erased when compiling scala to js, there is much more freedom in encoding whatever extra types that are needed and then it doesn’t affect the js engine anyway.
also, as an unimportant side note, string is a primitive in javascript. one can make a string object, but nobody does that. what to infer from that? surely, changing the position of string in scala’s types hierarchy, based on javascript weirdness, is not desired.
I kinda do like using explicit nulls to be able to use safely use null for its original purpose of an absent value. For example:
def doSomething(default: String | Null = null)
You could argue: Use Option!
But Option requires manually wrapping in Some.
You could argue: Don’t use null as a sentinel!
But String and Null are disjunct types so this is safe, and using null in this way is widely understood.
You could argue: But it’s not safe to call from Java!
Neither is Option[String], could be null at all levels.
To me, explicit nulls has two value proposals:
Help with null checking at the Java interop boundaries by hinting that the values may be null.
Make null available as a normally typed value in Scala.
To me, this proposal strengthens the second point. It’s just conceptually (and thus in terms of teachability) but I think conceptual simplicity is an important property of a type system.
I also wonder why people are so worried about AnyVal without null. I think using AnyVal as an explicit type anywhere seems less idiomatic to me than using null.
Using AnyVal | AnyRef as an “NotNull” seems to be very uncommon according to this GitHub search (like what, 5 hits?).
Just AnyVal seems to also be rare, often in some API wrappers trying to support primitives
I agree, we should work on that !
(I need to open a pre-sip about it, but my rough draft is “if in parameters, Int* is Seq[Int], then Int? should be Option[Int]”, and then doSomething() -> None, doSomething("hi") -> Some("hi"), doSomething(?myOption) -> myOption)
I think a programming language is at its best when it offers one “obviously” best way to do things
(decreases burden of choice on the users)
But currently, this is not the case:
T | Null if my type doesn’t contain null, to get speed and parameter convenience
Option[T] works all the time, and I get a lot of tools to handle it (map, filter, Option.when, etc)
On this end, I would frown upon String | 42 almost as much as String | Null !
(point being 42 is a normally typed value, and is even very well behaved, yet it is still not a good fit)
Regarding the performance downsides of Option@sjrd I think had a proposal/experiment to encode Option[T]asT | null, in cases where that is safe
I recently learned that this is what rust does, since references are never null, see std::option - Rust
Existing codebases that don’t want to depend on experimental features can’t use it today, because in non-experimental versions of Scala (i.e. without enabling -Yexplicit-nulls), Null is a subtype of AnyRef.
That’s too one dimensional. As a principle, it works against having broad features that overlap.
Designing this way often ends up with a giant list of narrow features, which also causes burden of choice to pick the right one.
At this point, are you arguing that null is bad, or that union types are bad? Because it seems like the latter?
At the end of the day, my argument kinda just is that the semantics of String|Null are good and useful (like any similar union type).
And null is widely understood as a “useless default” so its also “intuitive” as a “did not pass a value“ (intuitive in the sense that people have seen it used that way).
I have measured this for many of my usecases and the performance difference does not really matter.
For me String | Null is really just semantically more clear and syntactically more convenient to be able to use either (unrelated) type.
options are intergrated throughout the whole scala library and have many helper methods, while unions with null aren’t. null-oriented languages have things like Null coalescing operator - Wikipedia or Safe navigation operator - Wikipedia to make handling of nullable values more convenient - adding that to scala will complicate things for little benefit, as scala’s approach for handling nullness is inferior (convenience-wise) to specialized null handling syntax anyway. if scala authors admit that using null in ordinary scala code is good, then people will want more and more null-oriented convenience. that’s slippery slope.
I think it might look like that, but languages like Scala and Rust have done it by instead consolidating many useful features under simple mechanisms.
For example java’s static methods and the singleton pattern are both replaced by objects in Scala.
And in the particular case being discussed the situation is even more clear cut:
We already have a really good way of expressing “one instance of a value or nothing” (Option[T]), so if we make the other (T | null) better, it will only make the choice between the two harder.
I think I’m moreso arguing that using sentinels is kind of a bad idea, and that in some way null is the worse sentinel (since calling methods on it makes it crash)
I would say it is also widely understood as a “footgun” or even as “the billion dollar mistake”, which is why Scala has opted to go with Option instead.
And I think it is the kind of thing where intuition leads the wrong way:
It works for simple examples Int | NullString | Null, MyClass | Null, but leads to issues when you try to generalize:
def first[T](list: List[T]): T | Null = list.headOption.getOrElse(null)
def mapFirst[T](list: List[T], f: T => T): List[T] =
val head = first(list)
if head == null:
// "there is no first element, so list must be empty" (mistake)
List()
else:
f(head) :: list.tail
mapFirst(List(null, 0, 7), x => if x == null then 0 else x + 1)
// returns List() and not List(0, 0, 7) !
(And this is a risk for any sentinel value that is not private to the callsite)
Since explicit nulls is still experimental, would it be possible (under explicit nulls) to completely remove null from the type system? E.g., anywhere we get a Java value that could be null, the Scala compiler interprets (and automatically converts) it to Option[T]? Similarly, anywhere a Java method accepts a nullable type, the Scala compiler could expose it as Option[T] | T, and convert it to T or null behind the scenes before passing it to the Java method?
This would be a terrible idea, I think, because it would make interoperability extremely difficult.
Better would be a different type of option, Opn[T] which is opaque type Opn[T <: NotNull] <: Any = T | Null (you still need to know about Null and NotNull) that could be unwrapped via Option or other things. It could act basically identically to an unboxed Option.
Without a way to express null and pass it when desired to APIs that use null as special values, Scala would end up unable to consume and interact with the non-Scala ecosystem at least on the JVM. That seems like a very bad place to be; one of the saving graces of using Scala in the face of Python-does-everything is that the JVM-does-everything-too, and Scala can leverage that. Also, JS-does-very-much. But you can’t just lose the ability, and transparent conversion from Option doesn’t work because these things end up embedded in places where you don’t realize they are any longer. So in order to avoid deep traversal and copying, you need to be able to express them properly at the outset. However, Option[T] is not that expression: both Some and None are proper objects, and that is relied upon.
I was thinking null and None could be unified, e.g., you could just represent None as null at runtime, but I think this breaks backwards compatibility, so is not possible in Scala 3.
The issue is not only backwards compatibility: T | Null | Null and Option[Option[T]] are very different types
The first has two kinds of values: t: T and null
The second has three: Some(Some(t: T)), Some(None), None
Okay, this reads like you are just dismissing my arguments, and I find none of the dismissals convincing. I also don’t want to further hijack the thread into discussing if union types are bad, so let’s just disagree.
Edit: Okay, I feel like I came off badly here. I do think the arguments why String|Nullis bad are valid. When I say I am not convinced, what I mean is that those arguments don’t change my opinion that I find null to be useful and safe enough (with explicit nulls). But that’s just an opinion, and not one I am willing to fight strongly for
Actually there’s a straight-forward fix: define given[A] => Conversion[A, Option[A]] = Some(_) in the Option companion object. Is there any reason not to do that? I used to think we don’t have that because then you’d be able to call Option methods on all objects… but when it lives in the Option companion, it’s only available in contexts where an Option is required. And in the future, it’ll be even less problematic because you’ll need into to trigger an implicit conversion.
Actually we could even go one step further and declare given[A] => Conversion[A | Null, Option[A]] = Option(_).
Not that it matters because of the backwards compatibility issue, but I think you misunderstood my suggestion.
The idea would be that Option[T] = Some[T] | null, not Option[T] = T | null. Then the compiler would automatically convert the Option[T] into T | null when passing to a Java method that accepts T | null. To the Scala type system, a Java method foo(String bar) would look like foo(bar: Option[T] | T), so you don’t need to wrap everything in Some just to call a method.
String does not live outside of AnyVal and AnyRef. It has a settled place under AnyRef. That is its rightful place because on the JVM it is unforgeable (it has reference equality). While it is forgeable in JS (two strings with the same sequence of characters are indistinguishable), the least common denominator still puts it on the AnyRef side.
Null is forgeable on all platforms, and it does not have a place yet.
Nope. That would be using null as a sentinel. If I did propose such a thing, my previous arguments would be quite hypocritical What I proposed, a long time ago, is another encoding in which we use private values as sentinels (an infinite tower of Some(...(Some(None))...). That way, it is always safe, even for Ts that are nullables. Exactly as I explained previously.
For reference:
It would have been possible, if backward compatibility were not a concern, to encode None as null. However, it would still not be possible to encode Some(x) as x. The reason is the same as for the tower of Some^n(None) above: you would not be able to distinguish None from Some(None). This is true even if you never show null to user-space code.
If you can’t represent Some(x) as x, you still can’t use Option[T] as an interop type for Java’s nullable T. You need to manipulate T | None instead (standing for the nullable T). Then None is the new null. You’re back to square one in terms of safety, just with a different name.
This is not viable for other reasons. It would not work along a class hierarchy where bar: X in a superclass, and X is instantiated to Option[Y] in a subclass. It’s also not working when the nullable type appears as a type argument to a generic type. This is a common trap: it’s tempting to show Java things as “something else” to Scala, but it eventually never works because of the above points, among others.
Not necessarily !
I should have been clearer about:
I should have said “in cases where we know T is not nullable at compiletime”
That is the approach that Rust follows (null pointer optimization):
See Option Represnentation (already linked above)
It is therefore sound, when T is one of these types, to transmute a value t of type T to type Option<T> (producing the value Some(t)) and to transmute a value Some(t) of type Option<T> to type T (producing the value t).
[…] Rust further guarantees the following:
transmute::<_, Option<T>>([0u8; size_of::<T>()]) is sound and produces Option::<T>::None
transmute::<_, [u8; size_of::<T>()]>(Option::<T>::None) is sound and produces [0u8; size_of::<T>()]
This is only guaranteed for the Ts in the table (essentially non-null pointers and non-zero integers), not for arbitrary T!
(I don’t think we should implement this in Scala, as it sounds like a big undertaking for potentially very slim performance gains, but it is cool tech!)
I think it’s doable, but non-trivial to implement. You would need to track things at two levels: one for the type checker, and one for the code generator. You expose all T | null values as Option[T] to the type checker, but the code generator knows the runtime types, and can inject the appropriate conversion logic. E.g. if you’re doing a read or a write, you inject a small inline conversion. This might be questionable in terms of performance at the moment, though I think once value classes become available in the JVM it could end up being very cheap. If you don’t know the exact input type because it’s e.g. an erased generic parameter, you just put in an inline check that does the conversion, e.g. the byte code equivalent of:
switch (obj) {
case null -> null;
case scala.None -> null;
case scala.Some s -> s.value;
default -> obj;
}
I might be missing some important edge cases I guess, but the general idea is to track both runtime representation and the type-checker/user-facing type in tandem. I think it could even be done in a backward compatible manner (at least in terms of binary compatibility). It might even be possible to unify scala.Option, java.util.Optional, and T | null this way.
Whether or not it’s worth the complexity in the compiler, or even a good idea otherwise, is another issue.
Yes, you are–and this is the entire problem: there are edge cases.
Suppose you have Option[String | Null]. This can be inhabited by Some("eel") and Some(null) and None. All have different meanings.
But your default equivalent collapses Some(null) into Some(null).get, which is null. Same as None.
This is the fundamental problem with this encoding: if null is allowed at all, and Option is general, you must allow Some(null) and not collapse it with None. Yes, it is an edge case. But you cannot avoid worrying about that edge case. You must solve it. A “solution” that has broken edges is broken.
Worse yet, if this is the actual encoding, Some(None) is indistinguishable from None. So you can’t nest Option, either.
There are ways around this, e.g. with Seb’s tower of Some(…Some(None)…) sentinels. But null is not a way around it because you need a different sentinel for every depth.
What I proposed, a long time ago, is another encoding in which we use private values as sentinels (an infinite tower of