Should opaque type aliases nest?

While playing around with Dotty, I wanted an error type which could operate with accumulation or fail-fast semantics as needed. I attempted to implement this as a set of opaque type alias over Either, and decided to see if I could add the less common behavior as a layer over the more common behavior and forward as much of the functionality down-layer as possible.

What I found is that it looks like opaque type aliases can’t nest. I’ve created a smaller example of the behavior here:

package testing 

// This is the behavior that we'll attempt to forward, and
// end up unable to do so.
trait Semigroup[A] {
  def combine(lhs: A, rhs: A): A
}
object Semigroup {
  given [A: Semigroup](lhs: A) {
    def combine(rhs: A): A = summon[Semigroup[A]].combine(lhs, rhs)
  }
  given Semigroup[Int] {
    def combine (lhs: Int, rhs: Int): Int = lhs + rhs
  }
}

type ErrorMsg = String

// This is the default behavior
opaque type Accumulate[A] = Either[ErrorMsg, A]
object Accumulate {
  def[A](aa: Accumulate[A]) unwrap: Either[ErrorMsg, A] = aa
  
  def present[A](a: A): Accumulate[A] = Right(a)
  def missing[A](e: ErrorMsg): Accumulate[A] = Left(e)
  def extract[A,B](ma: Accumulate[A], fe: ErrorMsg => B, fa: A => B): B = ma.fold(fe, fa)
  
  given[A](given SA: Semigroup[A]): Semigroup[Accumulate[A]] {
    import Semigroup.given
    
    def combine(lhs: Accumulate[A], rhs: Accumulate[A]): Accumulate[A] = (lhs, rhs) match {
      case (Right(l), Right(r)) => Right(l combine r)
      case (Left(l),  Left(r))  => Left(s"$l|$r")
      case (l @ Left(_), _)     => l
      case (_, r @ Left(_))     => r
    }
  }
}

// This is the optional behavior I'd expect to be able to layer
// on because the compiler sees Accumulate and ShortCircuit as
// unrelated types
opaque type ShortCircuit[A] = Accumulate[A]
object ShortCircuit {
  import Accumulate.unwrap
  
  def present[A](a: A): ShortCircuit[A] = Accumulate.present(a)
  def missing[A](e: ErrorMsg): ShortCircuit[A] = Accumulate.missing(e)
  def extract[A,B](ma: ShortCircuit[A], fe: ErrorMsg => B, fa: A => B): B = Accumulate.extract(ma, fe, fa)
  
  given[A](given SA: Semigroup[A]): Semigroup[ShortCircuit[A]] {
    import Semigroup.given
    
    def combine(lhs: ShortCircuit[A], rhs: ShortCircuit[A]): ShortCircuit[A] = {
      println(s"Combining $lhs & $rhs")
      (lhs.unwrap, rhs.unwrap) match {
        case (Right(l),    Right(r)   ) => Accumulate.present(l combine r)
        case (l @ Left(_), _          ) => l
        case (_,           r          ) => r
      }
    }
  }
}

@main def main(): Unit = {
  // Tangentally: it seems like the companion objects of opaque types
  // are not part of the implicit scope. Is this intentional?
  
  //import Accumulate.given
  import ShortCircuit.given

  import Semigroup.given
  
  val formatI: Int => String = i => s"Value: $i"
  val formatE: ErrorMsg => String = e => s"Error: $e"
  
  def outputA(a: Accumulate[Int]): Unit = {
    println(Accumulate.extract(a, formatE, formatI))
  }
  def outputS(s: ShortCircuit[Int]): Unit = {
    println(ShortCircuit.extract(s, formatE, formatI))
  }
  
  val aI: Accumulate[Int] = Accumulate.present(1)
  val aE: Accumulate[Int] = Accumulate.missing[Int]("Oh no!")
  val sI: ShortCircuit[Int] = ShortCircuit.present(1)
  val sE: ShortCircuit[Int] = ShortCircuit.missing[Int]("Oh no!")
  
  println("==== Accumulating ====")
  outputA(aI)
  outputA(aE)
  outputA(aI combine aI)
  outputA(aI combine aE)
  outputA(aE combine aI)
  outputA(aE combine aE)
  
  println("==== Short Circuiting ====")
  outputS(sI)
  outputS(sE)
  outputS(sI combine sI)
  outputS(sI combine sE)
  outputS(sE combine sI)
  outputS(sE combine sE)
}

Expected output:

==== Accumulating ====
Value: 1
Error: Oh no!
Value: 2
Error: Oh no!
Error: Oh no!
Error: Oh no!|Oh no!
==== Short Circuiting ====
Value: 1
Error: Oh no!
Combining Right(1) & Right(1)
Value: 2
Combining Right(1) & Left(Oh no!)
Error: Oh no!
Combining Left(Oh no!) & Right(1)
Error: Oh no!
Combining Left(Oh no!) & Left(Oh no!)
Error: Oh no!

The actual output varies:

  • If the import for Accumulate.given is present, they all exhibit accumulating behavior.
  • If the import for ShortCircuit.given is present, they all exhibit short-circuit behavior.
  • If both are present, the compiler complains about ambiguous implicits.
  • If neither are present, the compiler complains about missing implicits.
    Apparently, opaque type aliases don’t have implicit scope?

As the docs are quiet about what happens when you try to nest opaque types, I wanted to raise two questions:

  1. Is this the intended behavior?
  2. If so, should it be?
2 Likes

Does it still happen if main and the opaque types are in different packages?

Yep, that’s the case in the original code, they’re in the same package here to make it easy to copy-paste out and play around with.

This actually fails nicer than the bigger version, as that just recurses between the extension method and the implementation until it blows the stack :slight_smile:

Looks like a bug to me.

It would be good to minimize this further and state what the failing expectations are.

1 Like

I’ve trimmed it down a bit, and I think I can see what’s going on.

Code

trait Test[A] {
  def test(a: A): String
}
object Test {
  given [A: Test](a: A) {
    def test: String = summon[Test[A]].test(a)
  }
  
  given Test[String] = s => s"$s got: String"
}

package aliased {
  opaque type Foo[A] = A
  object Foo {
    given[A](given N: Test[A]): Test[Foo[A]] = a => s"${N.test(a)} in Foo"
    given[A](a: A) with
      def foo: Foo[A] = a
  }

  opaque type Bar[A] = Foo[A]
  object Bar {
    given[A](given N: Test[A]): Test[Bar[A]] = a => s"${N.test(a)} in Bar"
    given[A](a: Foo[A]) with
      def bar: Bar[A] = a
  }
  
  opaque type Baz[A] = Foo[A]
  object Baz {
    given[A](given N: Test[Foo[A]]): Test[Baz[A]] = a => s"${N.test(a)} in Baz"
    given[A](a: Foo[A]) with
      def baz: Baz[A] = a
  }
}

@main def main(): Unit = {
  import Test.given
  import aliased.Foo.given
  import aliased.Bar.given
  import aliased.Baz.given
  
  println("expect 'String in Foo'              ".foo.test)
  println("expect 'String in Foo in Foo'       ".foo.foo.test)
  println("expect 'String in Foo in Bar'       ".foo.bar.test)
  println("expect 'String in Foo in Bar in Foo'".foo.bar.foo.test)
  println("expect 'String in Foo in Baz'       ".foo.baz.test)
  println("expect 'String in Foo in Baz in Foo'".foo.baz.foo.test)
}

Run In Scalastie

Output

expect 'String in Foo'               got: String in Foo
expect 'String in Foo in Foo'        got: String in Foo in Foo
expect 'String in Foo in Bar'        got: String in Bar
expect 'String in Foo in Bar in Foo' got: String in Bar in Foo
expect 'String in Foo in Baz'        got: String in Foo in Baz
expect 'String in Foo in Baz in Foo' got: String in Foo in Baz in Foo

Thoughts

Baz acts exactly as I expected Bar to behave, and the difference appears to come down to lines 22 and 29:

22:    given[A](given N: Test[A]): Test[Bar[A]] = a => s"${N.test(a)} in Bar"
29:    given[A](given N: Test[Foo[A]]): Test[Baz[A]] = a => s"${N.test(a)} in Baz"

What appears to be happening is Bar is able to unwrap the Foo, despite not being in Foo's companion object. Rereading the docs indicates the scope in which these aliases are transparent isn’t restricted to their companion objects, but instead their enclosing scope.

This nicely explains the bad behavior in my original code, as the aliases were top-level definitions.

Is this a hard technical limitation, or could this be changed?

1 Like

It’s intentional. For example it allows defining an opaque type member of a class which is not opaque to the other members of a class. It also allows defining an opaque type that knows about the right-hand side of another opaque type without needing more wrapping (e.g. in https://github.com/lampepfl/dotty/blob/master/compiler/src/dotty/tools/dotc/core/Flags.scala where Flag knows about FlagSet = Long).

1 Like

Fair enough. This does make top-level alias definitions an easy way to shoot yourself in the foot - especially if they’re generic. Normally, putting A where you’d need a Foo[A] would be a compilation error, but in this case it took a couple of weeks before the logic bomb exploded, and I only found it because I tweaked my tests slightly.

Would it be reasonable to require an explicit cast within scope, but outside the companion object? This would keep the flexibility, but significantly reduce the risk accidental errors. To give an idea of the impact of this, the example from the docs would be changed to this:

object Logarithms {
  opaque type Logarithm = Double

  object Logarithm {
    def apply(d: Double): Logarithm = math.log(d)
    def safe(d: Double): Option[Logarithm] = if (d > 0.0) Some(math.log(d)) else None
  }

  given logarithmOps: (x: Logarithm) extended with {
    def toDouble: Double = math.exp(x: Double)
    def + (y: Logarithm): Logarithm = Logarithm(math.exp(x: Double) + math.exp(y: Double))
    def * (y: Logarithm): Logarithm = Logarithm(x: Double + y: Double)
  }
}

Alternately, would it make sense to allow specifying the scope of the opaqueness? Again modifying the example from the docs, this could look something like this:

object Logarithms {
  opaque[Logarithms] type Logarithm = Double

  object Logarithm {
    def apply(d: Double): Logarithm = math.log(d)
    def safe(d: Double): Option[Logarithm] = if (d > 0.0) Some(math.log(d)) else None
  }

  given logarithmOps: (x: Logarithm) extended with {
    def toDouble: Double = math.exp(x)
    def + (y: Logarithm): Logarithm = Logarithm(math.exp(x) + math.exp(y))
    def * (y: Logarithm): Logarithm = Logarithm(x + y)
  }
}

But would also allow creating opaque type aliases which don’t need the enclosing object:

opaque type Logarithm = Double

object Logarithm {
  def apply(d: Double): Logarithm = math.log(d)
  def safe(d: Double): Option[Logarithm] = if (d > 0.0) Some(math.log(d)) else None

  given logarithmOps: (x: Logarithm) extended with {
    def toDouble: Double = math.exp(x)
    def + (y: Logarithm): Logarithm = Logarithm(math.exp(x) + math.exp(y))
    def * (y: Logarithm): Logarithm = Logarithm(x + y)
  }
}

If you define an opaque type at the top-level you just have to keep in mind it’s not opaque in the whole file, that’s not a big deal since you can just put different top-level definitions in different files and opaque types won’t be visible between them.

I don’t have any opinion on the modifications you propose, but I can tell you that they probably would not be easy to implement given the current implementation of opaque types in dotty.

Are you sure about that? I can’t test this at the moment, but my code was behaving an awful lot like I’d accidentally gifted my aliases with package wide transparency.

Specifically, an implicit for one was resolving as the implicit for the other, which was defined in a separate file.

This test is intended to show that top-level opaques are visible only in their own file. So, I don’t think they can leak.

2 Likes