Making union types even more useful

Explicit nulls can be annoying to work with directly, especially in combination with strict equality, but they are viable now. Here’s an unboxed nullable-option type I (re-)implemented while porting radix-tree to dotty, for example:

package radixtree

object Opts {

  opaque type Opt[A] = A | Null

  object Opt {
    def apply[A](a: A): Opt[A] = a
    def empty[A]: Opt[A] = null
    def fromOption[A](a: Option[A]): Opt[A] = a match
      case Some(x) => x
      case _       => null
  }

  private given OptCanEqualNull[A]: CanEqual[Opt[A], Null] = CanEqual.derived

  extension [A](ref: Opt[A]) {
    def isDefined: Boolean = ref != null
    def isEmpty:   Boolean = ref == null

    def get: A = if ref == null then throw new NoSuchElementException("Opt.empty.get") else ref

    def ref: A | Null = ref

    def toOption: Option[A] = if ref == null then None else Some(ref)
  }

  extension[A](a: Opt[A]) def same(b: Opt[A]): Boolean =
    if a.isDefined && b.isDefined then a.get.equals(b.get)
    else a.isDefined == b.isDefined

  extension [A, B](ref: Opt[A])
    def map(f: A => B): Opt[B] = if ref == null then null else Opt.apply(f(ref))
    def flatMap(f: A => Opt[B]): Opt[B] = if ref == null then null else f(ref)

}

(Strangely, I had to ditch the custom implementation of toString that the original Opt had because the underlying toString implementation somehow manages to penetrate the opaqueness of Opt[A] and consistently get selected as the receiver for all Opt[A].toString calls, leading to NPEs.)

2 Likes

All opaque types are upper-bounded by Any, and toString is defined on Any, so the extension won’t ever be selected.

2 Likes

My bad, forgot type test on line 15 and 19.
It should have been:

Yes, we can use the familiar HasX here

trait HasPitchClasses: 
  def pitchClasses: Set[PitchClass]

but still kind of artificial from a domain perspective as Chord and Scale are different things. Creating these HasX feels a bit boilerplaty, but the contract + docs-in-one-place-argument is of course strong.

But what about Set[PitchClass] in the Filter union? I could of course wrap it in

case class PitchClasses(pitchClasses: Set[PitchClass]) extends HasPitchClasses

but then I get some confusion of when to use Set[PitchClass] and PitchClasses from the library user’s point of view, and I wouldn’t want to open Pandoras box of implicit conversion here (see other exciting thread :slight_smile: ).

And we have Empty in the union, which I use in more places than just as “is a HasPitchClass thing” so, I want to avoid using inheritance there.

So all in all I found the

type Filter = Empty | Set[PitchClass] | Scale | Chord

to be so nice and clean :heart:

1 Like

Thanks for the example. I’ll see how something similar works out in my example code. But several times when I have started on opaque types I have found that I wanted a custom toString and went back to a case class. And also, often I find it cleaner with just a value of pure emptiness without type parameters and stuff (c.f. Nil for lists). And Int | Empty reads so nice IMHO.

They’re different things but they have a method in common so it makes sense to have a common trait (and as I said it’s the only place where one can document what this method does on both of them).

Nothing prevents you from using a union but also having a base trait for things which have a method in common.

1 Like

Nothing is a subtype of anything

Thanks for pointing that mistake out – I should have seen that! I’ll update my example above.

But your proposal still does not compile using RC2-nigtly:

scala> extension [T <: Matchable](x: T | Empty)                                                                                
     |   def isEmpty: Boolean = x match
     |     case Empty => true
     |     case _ => false
     |   
     |   def nonEmpty: Boolean = !isEmpty
     |   
     |   def get: T = x match
     |     // no type coercion needed, Nothing is a subtype of anything.
     |     case Empty => throw new NoSuchElementException("Empty.get")
     |     case x: T => x
     |   
     |   def getOrElse(default: T): T = x match
     |     case Empty => default 
     |     case x: T => x
     | 
     |   def toOption: Option[T] = x match
     |     case Empty => None
     |     case x: T => Some(x)
     | end extension
11 |    case x: T => x
   |         ^^^^
   |         the type test for T cannot be checked at runtime
15 |    case x: T => x
   |         ^^^^
   |         the type test for T cannot be checked at runtime
19 |    case x: T => Some(x)
   |         ^^^^
   |         the type test for T cannot be checked at runtime

In this particular case, the docs for Scale and Chord would be different, as the docs should explain that steps are accumulated to form the pitch classes. But I see your point. Maybe I should stop wishing for matchmaking of non-relatives :slight_smile:

But these are only warnings right? Either way. If asInstanceOf is part of your solution it’s almost always a warning sign that what you’re doing maybe isn’t such a good idea.

If you use Null instead of Empty, there is a special compiler rule that makes this work. The reason is that the erasure of X|Empty is Object while the erasure of X|Null is X. This makes Null both safer and more efficient in this case. Of course, you can’t call toString on null, but if your concerned about that I promise you that you will be better off using Option.

A custom Empty class is only going to give you a headache.

1 Like

I think it could be handy if you have do deal with third-party code which you cannot modify. I.e. adding a base trait does not work. There it could be simpler to use a union type instead of e.g. create n adapters with a base trait (thought with dotty this will become much simpler with the export clause :slight_smile:). I guess those are rather rare cases and using a pattern match in such a situation is also OK, i.e. not too much boilerplate. Anyway… I guess we will see in a few years, when unions are used more often, if it is justified to modify the language in order to get rid of the boiler plate.

3 Likes

Ah, yes - I had fatal warnings on… Thanks for the explanations of how the types are erased! Maybe I should give null a try - but I have gotten that bad null-smell into my bones so it will perhaps be like an acquired taste for me nowadays.

I would be really interested in examples of such headaches? (So far it seems to play out well, e.g. by using .collect { case p: Pitch => ...} on structures to skip Empty values and sometimes .toOption when that is needed for integration with standard library methods etc…)

I took a swing at creating a typeclass style version of this, to see if I could use that to get around some of the unpleasantness.

The first attempt was to check if, given T[_] we could summon T[A|B] if T[A] and T[B] both existed. Sadly, this doesn’t seem to be the case. This was a bit of a moonshot, as I could see cases where that wouldn’t be able to be done automatically (e.g. Numeric[A|B] is a bit harder to auto-derive than Show[A|B]).

The second attempt was to do this manually, which hit a bit of a weird roadblock:

trait Show[A] {
  extension (a: A) def show: String
}
object Show {
  given [A, B](using AS: Show[A], BS: Show[B]): Show[A | B] with {
    // Error: `as` expected
    extension (ab: A|B) def show: String = ab match {
      case a: A => a.show
      case b: B => b.show
    }
  }
}

This was a bit confusing, as I didn’t see any relevant as mentions in the given docs, so I’m not sure if this is a bug, or if I’m simply trying something that isn’t possible.

Scastie

Sorry for the edits, I’ve got a squirming 2 year old that’s trying to “help” :man_facepalming:

1 Like

Your scastie is running on Scala 3.0.0-M1 for some reason, if you hit Reset in the build settings and click Scala 3, you get 3.0.0-RC1.

This cannot possible work, type erasure means that type test will always succeed since it will just be erased to Object (and the compiler will let you know with a warning).

1 Like

Weird :man_shrugging: , thanks for spotting that.

After refreshing the build, the errors have moved to the call sites, and are almost beautifully obscure:

ambiguous implicit arguments of type Show[(Oops | Ouch)] found for an implicit parameter of method printError.
I found:

    Show.given_Show_A[A, B](
      /* ambiguous: both object given_Show_Oops in object Oops and object given_Show_Ouch in object Ouch match type Show[A] */
        summon[Show[A]]
    , 
      /* ambiguous: both object given_Show_Oops in object Oops and object given_Show_Ouch in object Ouch match type Show[B] */
        summon[Show[B]]
    )

But both object given_Show_Oops in object Oops and object given_Show_Ouch in object Ouch match type Show[A].

Scastie

If I’m understanding this correctly, this means that union types are completely unusable with typeclasses of any sort. Can you confirm this? If true, this would seem to be a complete deal-breaker for union types, due to the number of libraries which use one or more typeclasses.

Union types get erased at runtime to the least upper bound of the participants, so it would be impossible to tell one member of a union apart from another by type alone at runtime. However, this is what TypeTests are for, as they allow you to determine type from other information, such as the classic “just give each one a string with its original type name on it”, or something else that would enable you to tell them apart. They’re also a typeclass, so a library user could define their own instance to perform runtime type tests on the types they want to use, which could include unions.

I get that, I’d misinterpreted this previous comment:

I didn’t catch that, not only was the automatic expansion of ab.foo removed, what it was expanded to is also no longer legal.

The upshot is this makes union types very difficult to abstract over. While this is disappointing, I haven’t exactly gotten use to them extensively yet, so it’s just a missed opportunity rather than something that’s actively causing me trouble.

I gave Typeable a try, and unfortunately came up empty again. This time it didn’t work with either a given instance or a direct method call with the types passed separately. I was able to get it to work if I explicitly specified both types to the direct method, but that’s not terribly useful for what I have in mind (tracking failure states).

The error almost looks like it’s erasing A|B to Object|Object:

no implicit argument of type Show[Object] was found for parameter x$2 of method printError2.
I found:

    Show.given_Show_A[A, B](/* missing */summon[Show[? <: Object]], ???, ???, ???)

But no implicit values were found that match type Show[? <: Object].

Scastie

No, that’s not what I meant. case x: A => is meaningless if A is a type variable upper-bounded by Any, but the context of the original question in this thread is something with A and B both being classes, in that class case x: A => works of course.

Yep, my misunderstanding was that there was some detail in the encoding of union types that would allow extracting the participating types in any situation where (a: A) match { case _: A => } would work.

my misunderstanding was that there was some detail in the encoding of union types that would allow extracting the participating types in any situation where (a: A) match { case _: A => } would work.

Is there any help from ClassTag or it’s kin? (Did ClassTag make it into Scala3?)

2 Likes

I’m not sure, I tried using TypeTest in this Scastie, but I haven’t branched out to the ClassTag and co.

It doesn’t look like ClassTag made it in, though the docs for TypeTest are floating, so I might have missed something.