Proposal for Opaque Type Aliases

eyalroth · February 17, 2020, 8:24pm

I think we’re going around in circles with definitions. Let’s try something different.

Putting performance considerations aside, give me one example where opaques would be used in a way that wrappers cannot.

smarter · February 17, 2020, 8:57pm

To provide safer facades to platform APIs (for JS APIs see Write facade types for JavaScript APIs - Scala.js, something similar should apply to native): if you want to restrict the values that can be passed to a particular JS function, you can’t use a wrapper because the erased type signature of the facade method has to match the type signature of the JS function, but you could use an opaque type.

eyalroth · February 17, 2020, 11:31pm

I’m not very fluent in JavaScript, TypeScript or Scala.js. There is nothing in this documentation that explains what are “facade types”; it is only said they are similar in spirit to TS type definitions (and then a link to TS documentation which doesn’t discuss type definitions).

What I can tell is that this documentation is about interoperability of Scala.js with JS libraries, which seems to suffer from a lot of irregularities and have different semantics than JVM Scala.

Would it be wise to design this new feature based on a use case relevant only to a different platform that already has so many irregularities?

On another note, the mentioning of JS and TS got me searching.

I found out that JavaScript already has opaque types via Flow (static type checker tool), and they are described similarly to this proposal.

On the other hand, TypeScript sees the topic differently. There is a long awaited feature for “nominal typing”; i.e, distinguishing between type aliases of the same underlying type without encapsulation of the underlying type’s methods. There are ways to emulate this feature though, so it’s unclear when (or if) it will be implemented, but it seems that opaques are out of the picture.

It seems that “nominal types” are the same as my suggested “strict types” from earlier, which address the problem more directly. It yet again begs the question of what is the purpose of opaque types, and what can they provide that wrappers cannot, except for better performance.

nafg · February 18, 2020, 1:11am

Even if it were true that there would be no JVM example of a case where opaque types can be used and wrappers cannot, and the JVM were the only platform, that does not mean there is no justification for opaque types. You asked for an example of one thing and then you inferred something else from that.

I guess there are 3 types of motivations for a new feature (that have come into the discussion) that helps with a goal that can already be achieved. (1) If it’s more performant, (2) if there are cases that cannot be done the previous way, and (3) if the new way produces more understandable code. I guess you already eliminated (1) and (3) so you need a case of (2) but it should be for the JVM?

mdedetrich · February 18, 2020, 9:48am

Whether it has to do with JS or JVM is irrelevant. opaque types allow you to specify that some type is the exact same as the other type on the underlying platform while allowing restrictions on the constructor, i.e. if you want to make a PositiveInt type on Scala.js you can use an opaque type to do this, i.e.

object JsTypes {
  opaque type PositiveInt = scalajs.types.Double //This isn't the actual Scala.js double type, using it for illustration
  object PositiveInt {
    def fromDouble(value: scalajs.types.Double) = ???
    def toDouble: scalajs.types.Double = ???   
  }
}

Since opaque types strictly are not a wrapper/adaptor and rather they just form a new type at runtime, you can guarantee that at runtime (i.e. in your compiled Javascript code) all instances of PositiveInt will actually appear as a Javascript Number type with zero performance cost.

We also have the same irregularities even on the JVM, its just that most people don’t care or we just type alias the Java implementation (with all of its problems). If for example you are writing performance sensitive code and you need a fast Option type, you basically have to manually implement opaque types in current Scala with its workarounds (currently right now Option always boxes), see GitHub - sjrd/scala-unboxed-option: A type-parametric unboxed Option type for Scala as an example. You can go even crazier and make another Option type that is represented as value in some | null as an opaque type.

In these cases you cannot represent it as an AnyVal because it isn’t an AnyVal. Its not wrapping anything and you deliberately don’t want it to wrap anything.

Opaque types are also really handy for Scala native for the same reasons posted as above.

eyalroth · February 18, 2020, 12:17pm

I am not disputing the need for opaques nor do I ask for a better justification. I am times and times again trying to view them differently only so their syntax will be clearer (in my opinion, obviously).

I am by no means eliminated motivation #1 (performance), but argue that it shouldn’t affect syntax.

Yes, I am wondering what cases qualify as motivation #2. As of the Scala.js example, I didn’t infer it at all; quite the opposite, I expressed unfamiliarity with the topic and hoped for a better and more specific explanation. I indeed noted that if this example is platform-specific, then I’d question its significance in the overall design of opaques’ syntax.

That leaves us with motivation #3, which is what I’m trying to focus on – more readable code.

I am not disputing their usefulness, but rather their syntax / encoding.

Let’s get more concrete and compare between the syntax alternatives.

Opaque as a type modifier

This is the currently proposed syntax:

object Access {
  opaque type Permission = Int
  implicit class ops(p: Permission) {
    def |(other: Permission): Permission = p | other
  }
  val NoPermission: Permission = 0
}

Cons:

Enclosing object with a different name is required.
Need to be familiar with extensions in order to add functions.
Ambiguity in method invocation (is p | other recursive?).
Need to be familiar with additional scope rules / semantics.

Opaque as a type entity

This is an alternative syntax in which opaques are type entities – much like class and object are. It doesn’t mean they are classes, but that they are on-par with classes in terms of syntax.

opaque Permission(i: Int) {
  def |(other: Permission): Permission = i | other.i
}
object Permission {
  val NoPermission =  Permission(0)
}

I believe this syntax overcomes all of the cons of the previous syntax. It feels familiar as it uses the “entity” syntax where an encapsulated data type is associated with methods. It can also leverage existing access modifiers (private) to control the level of encapsulation; it doesn’t need new scope mechanisms.

Nominal types

This syntax is not an alternative to the previous two, but a complementary one. It better addresses the need to distinguish between type aliases of the same underlying type without encapsulating them.

nominal type Password = String
nominal type GUID = String

These are not intended to be extended with more functions – that’s what opaques are for. Indeed the semantics of those need to be further worked on:

import Password

val password: Password = "god123"

password + password // ok
password.toLowerCase() // what's the return type?
password + "x" // allowed? what's the return type?

rgwilton · February 18, 2020, 1:18pm

I agree that

   implicit class ops(p: Permission) {
    def |(other: Permission): Permission = p | other
  }

seems confusing, and naively looks like a recursive function that will never terminate.

kai · February 18, 2020, 2:43pm

Putting performance considerations aside, give me one example where opaques would be used in a way that wrappers cannot.

Opaques can have lower and upper bounds. They metaphorically punch new holes in typespace where there were no types before, they can add arbitrary subtypes or supertypes to any, even final, classes and participate in variance in a way that wrappers cannot. e.g.

type MagnificientString = MagnificientString.T
object MagnificientString {
  opaque type T <: String = String
  def apply(s: String): T = s"magnificient-$s"
}

val mags: List[MagnificientString] = List("sky", "star", "snow").map(MagnificientString(_))
// list mags is list string
val ordinaries: List[String] = mags

def unmag(l: List[MagnificientString]): List[String] = l.map(_.stripPrefix("magnificient-"))
unmag(mags)
// the opposite is not the case
unmag(ordinaries) // Found: (ordinaries : List[String]) Required: List[MagnificientString]

eyalroth · February 18, 2020, 5:33pm

This is a great example, thank you. I’m not sure whether this is something we want, but it’s definitely a new possibility. Regardless, wouldn’t you agree that this encoding is preferable?

opaque MagnificientString[T <: String](s: T)
object MagnificientString {
  def apply(s: String): MagnificientString = MagnificientString(s"magnificient-$s")
}

kai · February 18, 2020, 6:45pm

Well, no, it implies both a direct constructor (might not want it) and direct destructor (same) and you can’t exactly name write it like:

opaque Mag[S <: String] private (private val s: S)

Because, well, s is not a val. And the parens are not a constructor either. And the overall structure doesn’t actually tell that:

Mag is actually parameterless, you declare it as Mag[S <: String], but refer to it as just Mag (val mags: List[Mag]). This is counter-intuitive
Mag itself conforms to <: String, not its parameter. It should be Mag extends String at least.

Overall as a way to write the above, this is very counter-intuitive.

On the other hand, having to ‘re-export’ opaques as in

type Mag = Mag.T
object Mag {
  opaque type T <: String = String
}

Is odd, opaque, non self-explaining, entirely unobvious to anyone and laborious boilerplate, I’d rather have the decision to delete companions for opaques be reverted so that more obvious syntax can be used.

morgen-peschke · February 18, 2020, 8:42pm

kai:

Well, no, it implies both a direct constructor (might not want it) and direct destructor (same) and you can’t exactly name write it like:
opaque Mag[S <: String] private (private val s: S)

How about something like this:

opaque MagnificientString[T <: String] = T {
  def unwrap: T = this
}
object MagnificientString {
  def apply(s: String): MagnificientString = MagnificientString(s"magnificient-$s")
}

This avoids implying construction, or the existence of a member which doesn’t actually exist.

eyalroth:

Nominal types

This syntax is not an alternative to the previous two, but a complementary one. It better addresses the need to distinguish between type aliases of the same underlying type without encapsulating them.
nominal type Password = String
nominal type GUID = String

If the intent is that the compiler treat them as different for things like implicit lookup as well, would newtype encapsulate this idea?

eyalroth · February 18, 2020, 8:46pm

kai:

Well, no, it implies both a direct constructor (might not want it) and direct destructor (same) and you can’t exactly name write it like:
opaque Mag[S <: String] private (private val s: S)

But that’s the syntax with classes – and it works. As for private val that’s not necessary because the syntax is equivalent to that of class (members are private by default) and not case class. That also means no “destructor” (I assume you mean unapply).

This is a fair point.

Why? we can think of opaque as being the type of its “member”. If the member is S, and S is subtype of String, then the opaque is a subtype of String.

I agree, and I’m glad we see things the same way. I think this use-case is quite new and unexplored, and perhaps deserves its own unique syntax. Or maybe it can be achieved with the nominal type syntax:

nominal type Mag <: String = String
// or maybe even just
nominal type Mag = String

We need to explore nominal types semantics more for that, though.

eyalroth · February 18, 2020, 9:14pm

I’m not sure the implied constructor is a problem:

val mag: Mag = "hey"
// but that is also ok:
val mag: Mag = Mag("hey")
// or
val mag = Mag("hey")

I do however think that the = is irregular and confusing in this location, as it doesn’t exist for other “entities”.

And there’s still the problem of the square brackets, which imply that the opaque should be declared Mag[_], but in fact it’s Mag.

Maybe something like this?

opaque Mag(s: Mag.S) { ... }
object Mag {
  type S <: String
}

Though I don’t like abstract type syntax inside objects in the first place (discussed earlier).

Maybe use trait inheritance?

trait MagT {
  type S <: String
}
opqaue Mag(s: Mag.S) extends MagT {
  type S = String
}
// or
trait MagT[S <: String] {
  def s: S
}
opqaue Mag(s: String) extends MagT[String] {

Though both of these look bizzare.

I am not sure what newtype is, and I haven’t thought about nominal types and their interaction with implicit lookup; again, I believe their semantics need to be further explored.

Katrix · February 19, 2020, 1:25am

Wasn’t new type Foo = String one of the alternative proposals for the syntax in the original SIP

Jasper-M · February 19, 2020, 10:20am

eyalroth:

I think this use-case is quite new and unexplored, and perhaps deserves its own unique syntax. Or maybe it can be achieved with the nominal type syntax:
nominal type Mag <: String = String
// or maybe even just
nominal type Mag = String

It would be weird to create separate syntax for it, because it fits perfectly with the intended semantics of opaque types. And it fits perfectly with how you expect a bounded type alias to behave.

Moreover I don’t think a special “nominal types” feature makes sense in Scala, because Scala already has nominal types (classes, traits …). It makes more sense for typescript which has a purely structural type system.

eyalroth · February 19, 2020, 3:15pm

It doesn’t fit the (intended) semantics of opaques. Opaques encapsulate their enclosed type by default, while these do not. Also, opaques are intended to be regularly extended (with additional methods that is), while these do not.

I don’t know how I would expect a bound alias to behave given that it’s impossible to express it in Scala 2. I see bound types as abstract definitions (of types), and type aliases as concrete ones. Combining them doesn’t make sense to me.

I’m not convinced that bound type aliases are describable. It seems like an overly complex feature that very few will understand and use regularly, perhaps a bit like forSome.

Then perhaps they deserve a different name. The shortest description I can think for them is “non interchangeable type aliases”, but obviously noninterchangeable is not a very good keyword.

Jasper-M · February 19, 2020, 3:45pm

Opaques hide the underlying type from the rest of the program. An “unbounded” opaque type has bounds >: Nothing <: Any, just like an unbounded type member does. You can narrow down the bounds but the underlying type stays hidden. Unless of course you do this:

scala> object Opaque { opaque type T >: String <: String = String }
// defined object Opaque

scala> implicitly[Opaque.T =:= String]
val res13: Opaque.T =:= String = generalized constraint

Perhaps the compiler could warn here that this doesn’t really make sense as an opaque type…

A type member can be a concrete alias in one context and a bounded abstract type in another. And the semantics are the same as those of an opaque type.

scala> trait Opaque {
     |   type T <: CharSequence
     |   def apply(s: String): T
     | }
// defined trait Opaque

scala> val Opaque: Opaque = new Opaque {
     |   type T = String
     |   def apply(s: String): T = s
     | }
val Opaque: Opaque = anon$1@297db6ad

scala> val t = Opaque("foo")
val t: Opaque.T = foo

scala> implicitly[t.type <:< String]
1 |implicitly[t.type <:< String]
  |                             ^
  |Cannot prove that (t : Opaque.T) <:< String

scala> implicitly[t.type <:< CharSequence]                                      
val res11: t.type <:< CharSequence = generalized constraint

eyalroth · February 19, 2020, 4:29pm

That’s exactly the difference in semantics between opaques and “non interchangeable type aliases”; the former do not hide the underlying type from the rest of the program.

Sure, that is the general idea of abstract definitions vs concrete ones. A definition may be either abstract, specifying a a contract that needs to be implemented in one of several possible ways; or concrete, specifying such an implementation (and potentially adhering to an abstract contract).

A definition cannot be both abstract and concrete; being concrete, it already has a specific implementation, so there is no meaning to add an abstract contract.

This is why this syntax is confusing:

opaque type Mag <: String = String

It contains both an abstract definition and a concrete one. How can a type be both concrete and abstract?

eyalroth · February 19, 2020, 4:52pm

One more thing I realized with this example is that MagnificientString does not actually hide the underlying type:

MagnificientString("sky").toLowerCase() // compiles

This really is a bizarre case with strange semantics. I’m not sure it makes sense.

jeremyrsmith · February 19, 2020, 6:09pm

I think the confusion here is that an opaque type alias has a definition (the right-hand side of the =) which is hidden, but can also have bounds which aren’t hidden (the <: String bit). You’ve specified that even though outside of MagnificentString we can’t know that T is exactly String (the definition is hidden), we do know that it’s always a subtype of String (the bound isn’t hidden), and therefore you can use the toLowerCase() method. This isn’t a particularly real-world use case for opaque type aliases (though I can imagine some use case where having a transparent bound but an opaque definition might be useful)