Proposal for Opaque Type Aliases

Scala Improvement Process
85

I might say warning instead of error, but this is an interesting point. I have no idea whether it’s actually feasible – since Any is kind of the universal get-out-of-jail-free card, I don’t know whether it would be straightforward in the compiler to prevent this upcast, and from a type-theory perspective it seems weird to have a type that isn’t just an ordinary subtype of Any.

But it does seem like it goes against the intent of opaque types to allow this upcast silently – requiring an explicit : Any if you really want to do the upcast seems like it might nudge people away from this particular bad idea…

1 Like
86

I think that this is a good idea but note that it won’t be perfect: you are going to have to choose between false positives and false negatives. Consider a method toAny:

def toAny[T](t: T): Any = t: Any

Should defining toAny trigger errors? I would say “no” because it’s well-formed and should be valid in a language with Any.

Should calling it on opaque types trigger errors? I would say “no” because, from the call site’s point of view, it consumes the opaque type without upcasting it and just happens to return an Any.

1 Like
87

Rather than blocking pattern matches involving opaque types completely and preventing stylistically-useful code, what about restricting them down a bit: maybe opaque types should only be allowed to appear in pattern matches on types that are made up of unions of themselves and other things with different erasures (including the degenerate union of just the opaque type itself)?

For example, suppose you had the following opaque types:

opaque type UserId = UUID
opaque type ProductId = UUID
opaque type EmailAddress = String
object EmailAddress {
  def unapply(e: EmailAddress): Option[String] = Some(e)
}

Then you could write:

(???: Int | UserId | EmailAddress) match {
  case i: Int => "Int"
  case u: UserId => "UserId"
  case e: EmailAddress => "EmailAddress"
}

val EmailAddress(str) = (???: EmailAddress)

(???: EmailAddress) match {
  case EmailAddress(str) if str.endsWith(".com") => ".com address"
  case _ => "other address"
}

but not:

(???: UserId | ProductId) match {
  case u: UserId => "UserId"
  case p: ProductId => "ProductId"
} // error: same erasure

(???: EmailAddress | String) match {
  case e: EmailAddress => "Email"
  case _ => "Something else"
} // still error: same erasure

(???: Any) match {
  case EmailAddress(str) => s"Email: $str"
  case _ => "Something else"
} // error: match performs unsafe cast

This would not get rid of the possibility to “forget” that a runtime value has existed as an opaque type by upcasting to Any and then matching on the erasure, but it should prevent something from being silently and unsafely cast to be an opaque type (which is likely to cause intended invariants to be violated) by upcasting something with the same erasure to Any and then matching on the opaque type.

Edit just to make it absolutely clear: my pitch is that when any opaque type appears in the pattern for any case, you do an analysis of the type being matched on, not that you somehow do some analysis of all the types appearing in all the cases, which I think would be very tough to get right.

88

This allows you to pass in the erasure of the opaque type and violate its invariants. For example, let opaque type Nat = Int. Its own methods will forbid negative numbers, but pattern matching would allow you to have a negative natural number.

1 Like
89

Ah no, I’m saying that you wouldn’t be able to do a match where the left-hand side has the same erasure as an opaque type appearing in any of the casees. For example, in your case, if Nat appeared in any of the cases and the type of the left-hand side was Int or anything with Int as its erasure other than Nat, there would be an error.

90

I’m a little late to the discussion, but I find the syntax of the feature quite confusing.

AFAIU, the major motivation behind the feature is improving performance as explained in the original SIP document. I’m wondering though, shouldn’t the performance boost be implemented by the compiler behind the scenes and not exposed in the source code?

Consider a much less radical syntactic change:

opaque class Logarithm extends Double {
  def toDouble: Double = math.exp(this)
  def +(other: Logarithm): Logarithm = Logarithm(toDouble + other.toDouble)
  def *(other: Logarithm): Logarithm = +(other)
}

object Logarithm {
  def apply(d: Double): Logarithm = math.log(d)
  def safe(d: Double): Option[Logarithm] =
      if (d > 0.0) Some(apply(d)) else None
}

Or perhaps the class signature should be this:

class Logarithm opacifies Double

This syntax remains very close to existing constructs which are familiar and are less confusing. It also doesn’t expose the underlying implementation of this feature – whether it’s being done with extension methods or something else.

There’s also quite a bit of ambiguity with the proposed syntax. Let’s take the “permissions” example from the dotty documentation:

object Access {
  opaque type Permissions = Int
  opaque type PermissionChoice = Int
  opaque type Permission <: Permissions & PermissionChoice = Int

  def (x: Permissions) & (y: Permissions): Permissions = x | y
  // ambiguity 1
  def (x: PermissionChoice) | (y: PermissionChoice): PermissionChoice = x | y 
  def (x: Permissions).is(y: Permissions) = (x & y) == y
  def (x: Permissions).isOneOf(y: PermissionChoice) = (x & y) != 0

  val NoPermission: Permission = 0
  val ReadOnly: Permission = 1
  val WriteOnly: Permission = 2
  // ambiguity 2
  val ReadWrite: Permissions = ReadOnly | WriteOnly
  val ReadOrWrite: PermissionChoice = ReadOnly | WriteOnly
}

In the first case (“ambiguity 1”), it is unclear whether the pipe in x | y is the Int operation or a recursive call to the same method.

In the second case, | is an ambiguous method for ReadOnly (of type Permission), and that ambiguity is somehow resolved by the explicit type of the val to which the operation is assigned. I believe that this is a precedent that does not exist with any other feature in the language.

I would’ve imagined the permissions use-case to be coded this way:

opaque class Permission extends Int {
  def |(other: Permission): Permission = Permission(super | other)
  def isOneOf(permission: Permission): Boolean = (this & permission) != 0
}

object Permission {
  val NoPermission = Permission(0)
  val WriteOnly = Permission(2)
  val ReadOnly = Permission(4)
  val ReadWrite = ReadOnly | WriteOnly // 6
}

It’s probably worth mentioning that I didn’t fully understand what the original example was trying to model. There is no difference between “and” and “or” when talking about combination of permissions; ReadWrite == ReadAndWrite == ReadOrWrite.

91

The very first proposal for this feature used something similar to what you propose. However, it was rejected precisely because there was no way to provide class-like semantics (notably, correct instance tests) while gaining the performance guarantees and the other semantics that we wanted. It turns out that correct semantics are precisely the ones that a type alias already has, and therefore the concept of opaque type alias was much more appropriate. An opaque type alias really is just a type alias that we made opaque. It is nothing like an independent class.

92

Could you please elaborate on that?

Obviously an “opaque” class has different semantics than a regular one:

  1. It can extend final classes.
  2. It extends classes; traits can only be mixed in.
  3. It extends everything from the super class privately.

But case class also has different semantics than regular ones. I still find this idea more closely related to classes than to type aliases with extension methods, and more importantly – easier to grasp this way.

93

Ah wait, I think I understand what you are referring to - that this is in fact not possible to assert:

obj.isInstanceOf[Logarithm]

Which is the same for type aliases.

Thing is, this is really confusing to think of them as aliases. The whole idea with type aliases is that one can replace them in the source code with the original type and the program will not change. This is not true for opaques. In most cases, it would be impossible to retain the program’s semantics by replacing them with the original type; in fact, instance checking is the only place where they share this similarity with aliases.

How about thinking of them as classes but adding a compiler warning whenever they are used in instance checking?

obj.isInstanceOf[Logarithm] // warning
case log: Logarithm =>  // warning
94

IIRC: Anything requiring a runtime distinction will break. classOf[T] and instanceOf[T] are some examples, but not all. I think there are were quirks with AnyValue types and boxing as well.

Opaque types are not classes – classes/traits exist in the bytecode. Opaque type aliases are purely on the compiler side, just like type aliases.

I suppose the purpose of the Opaque vairant is explicitly for this not to be true.

I do like the simplicity of a ‘class like’ syntax. I don’t know if there is a way to get such syntax without implying incorrect semantics and strange exceptions to existing rules.

95

Bytecode representation is an implementation detail; semantics shouldn’t be decided according to it.

Opaques are not really classes nor aliases. They are unique. The question is whether this merit an entirely new syntax or some conjunction of existing constructs with a small amount of new constructs. If we go for the second option, then the question is which is the most fitting existing construct.

Thinking of opaques as aliases is confusing to me:

type AliasString = String
opaque type OpaqueString = String

// conversion to
val obj: AliasString = "hey" // compiles
val obj: OpaqueString = "hey"  // doesn't compile

// conversion from
val alias: AliasString = ???
val op: OpaqueString = ???

val obj: String = alias // compiles
val obj: String = op  // doesn't compile

// invocation
alias.toLowerString() // compiles
op.toLowerString() // doesn't compile

Aliases and opaques are for the most part quite different.

Using extension methods also seem like an ad-hoc ability rather than something that is an integral part of opaques and their intended use (opaques without additional methods have no use).

Perhaps we can just drop the class, this and super?

opaque Permission extends Int { i =>
  def |(other: Permission): Permission = Permission(i | other)
  def isOneOf(permission: Permission): Boolean = (i & permission) != 0
}
1 Like
96

Nope, as mentioned before, outside of their definition they behave exactly like abstract type aliases do. You can already make opaques like this:

object Permission {
  type Permission
  def apply(i: Int): Permission = i.asInstanceOf[Permission]
  private[this] def unwrap(p: Permission) = p.asInstanceOf[Int]
  
  def (i: Permission) | (other: Permission): Permission = Permission(unwrap(i) | unwrap(other))
  def (i: Permission) isOneOf (permission: Permission): Boolean = (unwrap(i) & unwrap(permission)) != 0
}

And various tagging libraries like GitHub - estatico/scala-newtype: NewTypes for Scala with no runtime overhead let you do the same with less boilerplate.

3 Likes
97

More so, the one specific feature of opaque types have is that they’re “opaque” outside of their definition, but are transparent inside their containing class:

object outer {
  object inner {
    opaque type T = Int
    implicitly[T =:= Int] // found, T = Int here
  }
  implicitly[inner.T =:= Int] // NOT found! T != Int here
}

However, this is also not different from how typealiases behave – it’s just upcasting:

trait Opaque {
  type T
}

object outer {
  val inner: Opaque = new Opaque {
    override type T = Int
    implicitly[T =:= Int] // found, T = Int here
  }
  implicitly[inner.T =:= Int]  // NOT found! T != Int here
}

The type ascription on val inner: Opaque is what erases the knowledge of T=Int and forces inner.T to be treated like a unique unknown type – yet inside the definition of inner the knowledge is preserved. What we can’t do in Scala though is declare an object with an upcasted type, so short of stuffing lazy vals into a package object we can’t declare such upcasted opaques at top-level. Even without opaques dotty makes this pattern better by allowing to declare top-level vals – and perform upcasting, opaque types themselves just make this, rather obscure, pattern actually tolerable to execute be inventing special syntax for it.

4 Likes
98

This is very interesting, I haven’t though of that. What is then the purpose of the new feature if it’s already possible to have opaques? Either:

  1. This implementation is still not as efficient as could be? (which is the major motivation for the feature / design pattern).
  2. There’s too much boilerplate, in which case it seem to me that the new syntax is about the same.

Type members and type aliases may use the same keyword (type), but they are not the same; they behave differently and are used for different purposes.

Indeed type members seem more closely related to opaques, but the current implementation – much like the currently proposed feature – does not really acknowledge opaques as a thing of its own but rather as a design pattern achieved by combining other features (extension methods).

If we were to introduce a new syntax just for this design pattern, why not go all the way and include everything that the design pattern requires?

99

It’s also interesting to note how this library chose to encode “new type” in its newer version – via a wrapper case class with normal methods. The underlying implementation that uses type members and extension methods is invisible.

100

Replace “bytecode” with “runtime” and my point still stands.

As for whether semantics should have anything to do with it, they MUST. The entire reason for this proposal’s existence is due to a performance concern. Every use case can be handled today using a simple wrapper class, like class Logarithm(d: Double) { ... } – other than the performance concern from wrapping the value.

In order to ensure that this feature never requires boxing, and is purely a compiler side fiction, there must not be a runtime representation. This contrasts with extends AnyVal which has runtime representation, boxing, etc. One can not have an open ended discussion of semantics here: the requirements define the semantics. Syntax is of course always open to bike shedding, but semantics in this case are not.

An opaque type is not a class. It can’t be an ‘opaque class’ – classes by definition have distinct identity, and support runtime type inspection.

These are compiler-only types, with no runtime representation. There is an existing feature for such things.

Don’t think of them as aliases. Perhaps Opaque Type Alias is the wrong name. How about Opaque Type? I’m not sure what to name of the superset of Type Alias and Opaque Type – both are user provided types with no runtime representation.

I have never thought of them as aliases, just compiler-only types with differing visibility rules.

What I do find confusing syntactically is how one has to create a companion object for the type to define its member definitions, but its hard to argue with a simple extension of what exists today that mostly just tweaks visibility rules.

3 Likes
101

Again, it seems that opaques share similarities and differences with all of the other components (classes, type aliases and type members), but they are neither. They have their own unique semantics.

I believe that associating them with type would be confusing, as we can already see here the confusion between type aliases and type members, which are different but are mistaken as similar since they share the same keyword. Let’s not add yet another meaning into that soup.

As @kai shown, they could actually be achieved with Scala 2 (given that the performance is the same as what is proposed), so it’s only a matter of syntactic sugar. If that’s what this all about, I suggest making their syntax tailored specifically to their use case:

opaque Permission(i: Int) {
  def |(other: Permission): Permission = Permission(i | other)
  def isOneOf(permission: Permission): Boolean = (i & permission) != 0
}

object Permission {
  // provided by default and can be overridden with a different implementation
  def apply(i: Int): Permission = i
  // not provided by default
  def apply(name: String): Permission = ???

  val WriteOnly = Permission(2)
  val ReadOnly = Permission(4)
}

import Permission._

val readWrite = WriteOnly | ReadOnly // compiles
val readWrite: Int = WriteOnly | ReadOnly // doesn't compile

ReadOnly.isOneOf(readWrite) // compiles
ReadOnly.isOneOf(6) // doesn't compiles
ReadOnly.toDouble // doesn't compile

2 == WriteOnly // compiles and returns `true`
obj.isInstnaceOf[Permission] // doesn't compile
obj match { case p: Permission => ??? }  // doesn't compile
102

With sugar like that, and the other changes going into Dotty (‘new’ not needed when calling a constructor when not ambiguous, top level definitions), I think the companion object can be dropped.

An ‘opaque’ holds its methods in static scope, so holding values there is just more sugar:

opaque Permission extends Int {
  require(this >= 0)  // example of input validation
  def this(name: String) = ??? // alternate construction
  def |(other: Permission): Permission = Permission(this | other)
  def isOneOf(permission: Permission): Boolean = (this & permission) != 0

  // note, these values are static, just like the defs above.  These can not reference `this`
  val Write = Permission(2)
  val Read = Permission(4)
  val Execute = Permission(8)
}


import Permission._

val rw = Write | Read // compiles
val rw: Int = Write | Read // doesn't compile
val rwx = Write | Read | Execute

Read.isOneOf(rw) // compiles
Read.isOneOf(6) // doesn't compiles
Read.toDouble // doesn't compile

2 == Write // -- does not compile the author can add `def toInt: Int` or `def isInt(other: Int): Boolean`  if this is a real use case. Casting from an opaque to Any is not safe.
2 == (Write: Any) // compiles, returns true
obj.isInstnaceOf[Permission] // doesn't compile
obj match { case p: Permission => ??? }  // doesn't compile
103

I can’t disagree more. The entire point here is to introduce a new type that points to some underlying one. The keyword type is very precisely correct and clear…

5 Likes
104

I think that might be a bit confusing; how come a def has access to this but val doesn’t? I understand why that is but that seems like an irregularity comparing to classes and traits which share a similar syntactic structure.

I actually now think that it shouldn’t extend the wrapped type, as this gives the notion of a class, which it’s not. Also, the implementation there should be Permission(super | other); otherwise, this is endless recursive invocation.

Having the wrapped value as a “member” is better in multiple ways, I believe:

  1. No clear association with classes and inheritance.
  2. Actually represents the concept better - a type that wraps another type.
  3. Makes it clear that this type has only one immutable member (no other state).
  4. Reduces the need for this and super, as the wrapped type is a parameter with a name.

I guess I don’t have a strong opinion on that one. I’ll leave that to the discussion of multiversal equality.