Making union types even more useful

[morgen-peschke]

@smarter

No, that’s not what I meant. case x: A => is meaningless if A is a type variable upper-bounded by Any, but the context of the original question in this thread is something with A and B both being classes, in that class case x: A => works of course.

Yep, my misunderstanding was that there was some detail in the encoding of union types that would allow extracting the participating types in any situation where (a: A) match { case _: A => } would work.

[dwalend]

my misunderstanding was that there was some detail in the encoding of union types that would allow extracting the participating types in any situation where (a: A) match { case _: A => } would work.

Is there any help from ClassTag or it’s kin? (Did ClassTag make it into Scala3?)

[morgen-peschke]

I’m not sure, I tried using TypeTest in this Scastie, but I haven’t branched out to the ClassTag and co.

It doesn’t look like ClassTag made it in, though the docs for TypeTest are floating, so I might have missed something.

[sjrd]

ClassTags are there, of course, but they are only useful to create Arrays of genetic types. If you’re trying to use a ClassTag for anything else, it’s wrong and it will blow up when you least expect it. There’s a reason we created TypeTest and Typeable.

[morgen-peschke]

@sjrd

There’s a reason we created TypeTest and Typeable.

Might be worth looking into why TypeTest’s page isn’t accessible in the docs (I had to come into the url directly from Google to access it), and possibly update it to explain this? Google returns almost nothing immediately useful on the subject (Searching in the Dotty context returns more results, however most is on the older side and the danger you’re referring to isn’t apparent in the docs).

[bjornregnell]

They’re different things but they have a method in common

In the thread about multi-level enums there is now a post by @michel-steuwer with an interesting use case for unions with shared members:

@michel-steuwer

The second example from the original post (the Hierarchy of Reals/Rationals/Integers/Naturals) is more tricky using Union Types as the top-level type has a shared member.

[mbloms]

@bjornregnell

Maybe I should give null a try - but I have gotten that bad null-smell into my bones so it will perhaps be like an acquired taste for me nowadays.

Trust me, nobody dislikes null more than me, but you have to consider why null is bad. It’s not the mere existence of null which is the problem, it’s the fact that it inhibits every other type. In the presence of -Yexplicit-nulls this is not a problem since null has been lifted out to its own type.

@bjornregnell

I would be really interested in examples of such headaches? (So far it seems to play out well, e.g. by using .collect { case p: Pitch => ...} on structures to skip Empty values and sometimes .toOption when that is needed for integration with standard library methods etc…)

The only benefit you get from your Empty class over null is that it allows you to call toString. I don’t have any specific pitfalls in mind, it’s just that Option and Null has better support.

Specifically, the flow typing support for Explicit Null, makes a lot of what you’re trying to do easier. For example you can do this works without casts:

Scastie - An interactive playground for Scala.

If you need to be able to call toString etc, without checking for null, then Option is better supported and more importantly will be more recognizable for people reading your code.

Of course, Union Types in general would probably benefit a lot from wider support of flow typing, but I imagine implementing that is easier said than done.

[bjornregnell]

Many thanks for your input and nice examples!! Esp. interesting that the proposed “Enhancement 1” above would require, or at least benefit from, flow typing. It would be a cool feature in a future Scala 3 version.

I enabled -Yexplicit-nulls in my build and I got 26 errors related to java integration with javax.sound.midi . So I should start digging into all those null-unsafe places… It is a nice bonus that Scala 3 now gives me a list of all those UncheckedNull.

@mbloms

The only benefit you get from your Empty class over null is that it allows you to call toString

One goodie with the toString of Empty is that my library, intended to be used as a “chord laboritory” with the REPL as main user interface, is that structures are prettyprinted nicely in the REPL. The type signatures also read nice with Empty without type param. And if I introduce a similar concept of Silence then it is also motivated from a domain modelling perspective.

[rcano]

Has the story around explicit null improved? back in version 0_28 (before the milestones and this RC), it was super broken, like, big time. Flow typing would randomly not work for trivial if (a != null) ..., extending a java class like JPanel would simply make the compiler crash, java methods would return Array[Thing | Null] | Null, and using .nn would only remove the outer layer, etc etc

Edit: As of RC1, extending JPanel with explicit null on doesn’t crash the compiler, it still fails to compile with a weird error that seems to be it’s failing to parse the members of JPanel.

[CiaraOBrien]

Using nn still only removes the outer layer last I checked. The rest has generally improved significantly.

[rcano]

Yes, .nn should only remove the outer layer, what I meant to say and poorly articulated is that, the times of such collection coming from java is huge, and there’s no default .nn variant to cast it away, like a .nnn or something. To make things worse, while the outer array itself will be UncheckedNull, letting you do things immediately with it, the inner type wouldn’t (IIRC).

[odersky]

It’s still a -Y option so should be considered as experimental and some more changes are in the pipeline:

A New UnsafeNulls Language Feature for Explicit Nulls

[odersky]

Update: The changes to nullability have now been merged.

A New UnsafeNulls Language Feature for Explicit Nulls

[megri]

While on the topic of explicit nulls…

I remember, while reading through one of the original drafts I believe, there being some discussion about flow types and how they were needed to strip away the nullability in a scenario like def len(str: String|Null) = if str == null then -1 else str.length.

I find the concept really intriguing in a less specific setting, like for union types. Interestingly, it seems the mechanism is halfway there with pattern matching: an exhaustive warning for Y is emitted if the type X|Y is only matched on the X branch. However, despite the compiler telling us that the Y branch is unchecked, it still infers the union for a “naked” match:

(xy: X|Y) match
case x: X =>
case y     => // y is of type X|Y even though the compiler complains specifically about Y if this case is removed.

While the above example not being very useful on its own, the required foundation for these “as-you-go” refinement types seems to be in place (singleton types + intersections/unions). It does also solve the problem of stripping away part of a union in an elegant manner, which I remember toying with at some point.

[odersky]

I agree in principle. Now the only hurdles are algorithmic complexity, compilation speed, and whether the community at large cares enough about union types (which might depend on the outcome of the experiment, so it’s a chicken and egg problem). Somebody could go all in with union types and flow typing to try that experiment.

[smarter]

@megri

However, despite the compiler telling us that the Y branch is unchecked, it still infers the union for a “naked” match

This happens because exhaustivity checking is run at a later phase than typechecking. Trying to move some of the logic from exhaustivity checking into the typer to give more precise types to the default case branch would be an interesting thing to try but no on has attempted that experiment so far.

[jimka2001]

What you are asking for makes sense, and I think any user of the system would expect the behaviour and would be surprised about this error. After all x is definitely a member of A | B. I.e., no element of the type lacks the method.

I realize that from an implementation perspective it might be difficult.

To generate a set of methods or members applicable to A|B is probably hard. But that’s not what’s being asked for here. Instead, what is being asked is whether a user named method/member is applicable.

[RubenVerg]

It could be implemented as

def f(x: A | B) = x.y

==

def f(x: A | B) = x match {
  case x_A: A => x_A.y
  case x_B: B => x_B.y
}

and the compiler knows if A and B have y

[jimka2001]

I really respect that the algorithmic complexity and compilation speed are of utmost importance for community acceptance. However are union types really a bit of a stab in the back. Do they give just enough abstraction to help the people developing dotty itself, but not really an abstraction level to the mortal programmer to make them elegantly usable?

I admit my perspective is naive. When Scala 3 is used in-force, we’ll see whether user’s love them or find them almost usable.

[murraytodd]

I have to weigh in here: I would consider the missing pattern matching exhaustivity to be a bug. As others have said, if union types are a real (new) aspect of Scala 3 language, they should be properly checked in pattern matching. (My strong opinion comes from the fact that I’m having to work around the lack of union types in my own work these days, and am eager to be able to write cleaner, more succinct code.)