Synthesize constructor for opaque types

Language Design
123

It’s rather boilerplate heavy, as (at least as I understand it) macros can’t currently produce definitions and you have to re-wrap the type after each call, but you can use them in a way that provides access to the methods of the underlying type:

object types {
  object Name {
    opaque type Type <: String = String
    def apply(s: String): Type = s
  }
  type Name = Name.Type
  
  object Email {
    opaque type Type <: String = String
    def apply(s: String): Type = s
  }
  type Email = Email.Type
}
object using {
  import types.{Name, Email}

  @main
  def test (): Unit = {  
    def foo(str: String): Unit = {
      println(str)
    }
    def bar(n: Name, e: Email): Unit = {
      println(s"$n @ $e")
    }

    val name = Name("JDoe")
    val email = Email("[email protected]")
    
    //foo(name)
    //  Found:    (name : types.Name.Type)
    //  Required: String

    //foo(email)
    //  Found:    (email : types.Email.Type)
    //  Required: String

    foo(name)
    foo(email)

    bar(name, email)

    println(name.toLowerCase())
    
    //bar(name.toLowerCase(), email)
    //  Found:    String
    //  Required: types.Name
    
    bar(Name(name.toLowerCase()), email)
    
    //bar(email, name)
    //  Found:    (email : types.Email.Type)
    //  Required: types.Name
    //  Found:    (name : types.Name.Type)
    //  Required: types.Email
  }
}
124

You can extract the boiler plate into a trait::

  trait OpaqueOf[A] {
    opaque type Type <: A = A
    def apply(s: A): Type = s
  }
  object Name extends OpaqueOf[String]
  type Name = Name.Type
  object Email extends OpaqueOf[String]
  type Email = Email.Type

And it can be extended further. One could add a validating function to go from A to the opaque type, for example. I think that isn’t the biggest issue here. The criticism, as I understand it, is that you can’t do this:

object types {
  object Duration {
    opaque type Type <: Double = Double
    def apply(n: Double): Type = n
  }
  type Duration = Duration.type
}

object using {
  import types.Duration

  @main
  def test(): Unit = {
    val d1 = Duration(5.0)
    val d2 = Duration(2.0)
    val d3: Duration = d1 + d2  // should compile but doesn't
    println(s"d1: $d1 d2: $d2 total: $d3")

    val d4 = 10.0 + d1 // should not compile but does
  }
}
125

My understanding is that this is less than ideal because it will force primitives to box. I’d normally use this for stuff that’s already objects, so it’s less of a problem for me.

With a minor tweak, it does work. The missing step is that the result needs to be re-wrapped because the result of _: Duration + _: Duration is a Double - which makes sense because operation on the value may make it no longer valid, like there’s no way to guarantee that (_: Email).take(5) is still a valid Email:

object types {
  object Duration {
    opaque type Type <: Double = Double
    def apply(n: Double): Type = n
  }
  type Duration = Duration.Type
}

object using {
  import types.Duration

  @main
  def test(): Unit = {
    val d1 = Duration(5.0)
    val d2 = Duration(2.0)
    val d3: Duration = Duration(d1 + d2)
    println(s"d1: $d1 d2: $d2 total: $d3")

    val d4: Double = 10.0 + d1
    val d5: Duration = Duration(10.0d + d1)
  }
}

The d4 compiles because d1 known to be a Double subclass, which is usually fine for these sort of wrappers. Usually what you’d want to avoid is mixing subclasses, or passing a Double where you want to be sure that it’s a valid Duration.

Edit: sorry for being scatter-brained, here’s the Scastie link

1 Like
126

This can be reduced to

object types {
  opaque type Name <: String = String
  def Name(s: String): Name = s

  opaque type Email <: String = String
  def Email(s: String): Email = s
}
1 Like
127

It’s boilerplate heavy if you wish for better performing value-classes; it’s irrelevant if you wish for non-interchangeable type aliases.

128

I can’t think of any evidence or reasoning to support that point of view. How can you objectively declare how much boilerplate is relevant for each (hair-splitting) case? There’s nothing inherent about a non-interchangeable type alias that means any amount of boilerplate is irrelevant, and nothing inherent about value-classes to mandate the opposite. Maybe you can explain your reasoning a bit more?

To me: boilerplate is boilerplate. I don’t care about the categorisation of the domain I’m applying it to. I just think “well this could be automated but isn’t” and then I think about stuff like {frequency,ease} of {creating,modification} relative to value it provides.

1 Like
129

Don’t you think this is a wild exaggeration? Opaque type aliases give you abstraction without overhead, which is exactly what they set out to do. You are free to define an API by defining methods that operate on the opaque type, where you also specify type signatures.

What do you mean by non-interchangeable type alias? How would they work? What is the semantics?

If I define an opaque type:

object foobar:
  opaque type Foo = Bar

Do you want Foo to unconditionally simply have all methods of Bar with all occurrences of Bar changed to Foo?

I’m genuinely trying to understand what it is you want, because I don’t see what it is you want to do that can’t be done in the scope of the current proposal.

3 Likes
130

What I meant is that the boilerplate of the current proposal (of opaques) has nothing to do with non-interchangeable type alias, as the current proposal does not seem to address that particular feature/use case/motivation. Obviously, if and when that feature will be addressed, boilerplate would be something to take into account.

I do not. I would not use (the current design of) opaques for the purpose described in the original SIP (non-interchaneable type aliases), for the same reason I never used value-classes for that purpose nor mere wrapper classes. There is a very basic assumption that I believe is incorrect in the original SIP:

One appropriate solution to the above problem is to create case classes which wrap String .

That is, In my opinion, a workaround. Not an appropriate solution. And if for some reason I considered that an appropriate solution, I would still use mere wrapper classes and not opaques, as I would prefer their familiarity and simpleness over opaques, which only give a performance benefit I do not care for (and believe that most should not care for it either in most cases).

Authors often introduce type aliases to differentiate many values that share a very common type. […] However, since type aliases are replaced by their underlying type, these values are considered interchangeable.

I would have hoped that this SIP would address exactly those questions – how would they work and what would be the semantics – and try to come up with a design that focuses on providing a solution to the core motivation presented. Instead, it focused on providing a solution to a minor problem (performance) to one of the workarounds for the core motivation (wrapper classes).

131

I think that’s what’s been done. Opaque types allow you to define a type alias which is interchangeable in the scope they are defined, and to be non-interchangeable outside of that scope. It’s a very general and versatile solution with far broader use cases than value classes, which is why so many people have been arguing against weakening them to better performing value classes a la Haskell’s newtype by synthesizing constructors for them by default. (which is the whole point of this thread)

As I just said, opaque types are much more general than better performing value classes.

After reading your reply, I’m still non the wiser as to what you think is missing. It’s very hard to figure out a solution with well defined semantics without understanding what the problem you want to solve is.

So:

What is the problem that you want to solve? Opaque types give you non-interchangeable type aliases. Do you want to somehow expose all methods from the underlying type? What is it that I can’t do in the current proposal that you want to be able to do?

1 Like
132

I’m not trying to be mean, but this rhetoric has been repeated over and over in the previous discussions on Opaques. I then asked explicitly for examples of use cases which cannot already be achieved by wrapper classes (if we ignore performance), and there were two examples only – one is specific to Scala.js, and the other was rather unexplored (it raised a lot of questions and was not discussed at length).

I am not saying that these use cases are not valid and are not desired, nor do I object for a better performing implementation of wrapper classes (with caveats). I am saying that all of these do not satisfy the original motivation of the SIP, and those are all new and unrelated motivations, and that I still wish for a design that actually addresses the original motivation.

Yes, I want to expose all the methods from the underlying type. That’s how type aliases behave and why they are being used – to give a more meaningful name to a type and retain all of its functionality.

The SIP acknowledges this, by starting the motivation with:

Authors often introduce type aliases to differentiate many values that share a very common type.

And then immediately points out the problem with type aliases:

In some cases, these authors may believe that using type aliases […] means that if they later mix these values up, the compiler will catch their error.

What I want is to give meaningful names to generic types, while using them exactly the same way I’d use the underlying type (so far that’s what type aliases are for), only that the compiler will prevent me from accidentally mixing between them if they have the same underlying type, just like it would prevent me from mixing any two normal types.

133

I don’t think you’re mean. I hope you don’t think I come across as mean :wink:

It’s hard to give a comprehensive list of use cases. I think the one @kai gave you was pretty telling.
It’s also possible to define a type consisting of Natural numbers:

object o:
  opaque type Nat <: Int = Int
  def natOption(i: Int): Option[Nat] = if i >= 0 then Some(i) else None

This exposes that Nat is an Int, making it possible to pass it to methods requiring Int and to use any of the methods from Int. Most importantly, you can’t pass any Int to a function that wants a Nat.

What you want, if I understand you correctly, is the ability to use all the Int methods on a Nat without exposing the subtyping relation so that a Nat can’t be implicitly upcasted to an Int.

I’ve actually been playing with an idea to make it possible to expose methods from the underlying type as methods on the opaque type. The problem with this is that it’s easier said than done to just expose all methods.

Consider the Nat case:
What is the type of (x: Nat) + (y: Nat)? The sum of two natural numbers are natural right? So Nat would be a good choice. Except that we forgot to consider overflow so now 2147483647: Nat + 1: Nat = -2147483648: Nat. Oops

How about (x: Nat) - (y: Nat)? This could be negative. Should we expose Int as return type? Or not expose the method at all?

It doesn’t always make sense to expose all methods unconditionally. It makes even less sense to replace the underlying type with the opaque type in all method’s type signatures. So either way, exposing methods should be an active choice, and if fancy type signatures are needed, they should be overridden explicitly.

It should also be noted that some of this is partly possible already:

  • You can use extension methods to manually provide forwarding methods for the ones you want. Sure, it’s boilerplaty, but it’s doable. All the methods on IArray are extension methods.

  • Also, if you have a method that return this.type then the opaque type is preserved:

scala> abstract class Box[+T](val x: T):                                                                                                                                                       
     |   def doStuff(): this.type
     | 
// defined class Box

scala> object o:                                                                                                                                                                                                                            
     |   opaque type MyBox[T] <: Box[T] = Box[T]
     |   def MyBox[T](x: Box[T]): MyBox[T] = x
     | 
// defined object o

scala> lazy val abox: Box[Int] = ???                                                                                                                                                                 
lazy val abox: Box[Int]

scala> lazy val mybox = o.MyBox(abox)                                                                                                                                                                                                       
lazy val mybox: o.MyBox[Int]

scala> :type mybox.doStuff()
o.MyBox[Int]

What I’m trying to say with this really long post is that what you want is absolutely doable, (with the caveat that there are a lot of edge cases to consider) and the current proposal doesn’t really need to change in any significant way to allow it.

Would it be enough if you could just write something like this:

opaque type ID = Int
transparent extension (self: ID)
  export self._

?

3 Likes
134

There’s some issues with that on a type system like Java’s, which is inherently a subset of Scala’s type system for the most part. For example, let’s say we define Name to be String. I think it’s uncontroversial that if a is Name, then a.concat(b) will compile if b is Name but not if b is String.

But what about a.contains(b)? That will not compile if b is Name, since the parameter is CharSequence, not String.

And, conversely, s"$a" will not compile, because toString() will return Name, not String, unless it is special-cased.

We could define a rule that if a method is defined by an ancestor trait/class, then the “conversion” to the opaque type should not happen, but I can easily see that causing issues as well. And it would not cover F-bounded types, or any equivalent replacement of them.

Even if restricted to AnyVal types, it’s not easy. What type should be returned by == and != in opaque Switch = Boolean? And if opaque Age = Int, then a / 2 will not compile, or produce a warning and return the wrong type, if a is Age but a / 2.0 will because Int defined a /(x: Double): Double. And, on the other hand, a >> b will compile if both a and b are Age, but it really shouldn’t.

So I understand why you’d like to have automatic method forwarding but I think the complexity does not warrant it.

1 Like
135

Not at all! It’s just that I have a tendency of coming off as arrogant and disrespectful, so I want to make sure to let you know that I very much appreciate your attempt at understanding what it is that I want.

I think you’ve made a pretty good point against Natural numbers as a use case for what I want :slight_smile:

However, I think I’ve made a mistake when you suggested this solution earlier with Username. I tried to see if that solution will prevent me from mixing two types – say, Username and Password:

opaque type Username <: String = String
def Username(s: String): Username = s
opaque type Password <: String = String
def Password(s: String): Password = s

def authenticate(username: Username, password: Password): Boolean = true

@main def foo() = {
  val username = Username("admin")
  val password = Password("12345678")
  println(s"Authenticating '$username'...")
  val isAuthenticated = authenticate(password, username) // compiles :(
  println(s"Result: $isAuthenticated")
}

(Scastie)

But now I see that this compiles only because the opaques are defined at the top level. If they are defined within an object – and even if imported from that object – it will indeed detect the incorrect mixing of argument types.

On a second thought, maybe the implicit (up)cast to the underlying type is not a terrible idea. In fact, perhaps even an implicit (down)cast from the underlying type to the opaque is desirable as well:

val username: Username = "admin" // down cast
val password = Password("12345678")
val credentials: String = s"$username:$password" // up cast

I guess now is the time when I join the “less boilerplate” club; i.e, I’d much prefer this:

object o {
  softkeyword opaque type Username = String
  softkeyword opaque type Password = String
}

Than this:

object o {
  opaque type Username <: String = String
  def Username(s: String): Username = s
  opaque type Password <: String = String
  def Password(s: String): Password = s
}

Again, I really appreciate your help :slight_smile:

2 Likes
136

It’s hard to convey tone in text, I have the same issue. English being a second language doesn’t help either :man_shrugging:

Yes, you have to be careful about that. Opaque types behave like normal type aliases in the scope they are defined, that’s kind of the whole idea. I would also avoid putting opaque types at the top level at all if I were you, they don’t behave like one would expect:
http://dotty.epfl.ch/docs/reference/other-new-features/opaques-details.html#toplevel-opaque-types
It’s actually something which should maybe be addressed, because it’s very surprising to mortals.

This is exactly what the current implementation gives you. :slight_smile:

Would you mind explaining what you mean here? Downcasts should never ever be implicit. What is possible in the current implementation is to flip the subtyping relation:

object login where
  opaque type Password >: String = String

This will prevent you from leaking a Password to something that expects a String, and it will let you implicitly upcast a String to a Password. Note: it’s not a downcast! I defined Password as a supertype of String.

Of course, since `Password` is no longer a `String`, you can't call any of the methods on `String`, you have to provide an API yourself: 
object login where
  opaque type Password >: String = String
  extension (self: Password)
    def ++(that: Password): Password = self ++ that
    def trim: Password = self.trim

It’s important to understand however, that all methods that are defined in Any can always be called. That includes toString, so I can still do this, there are no downcasts involved:

scala> val password: Password = "Passw0rd"
val password: login.Password = Passw0rd

scala> s"Users password is $password"
val res0: String = Users password is Passw0rd

I again also want to stress that you don’t need to have a subtyping relation at all, I don’t even think it was part of the SIP to begin with. If you write opaque type Password = String then Password is completely unrelated to String outside the scope of definition, and you again have to define the API yourself.

What you of course can do, if you want an implicit “downcast” available, is to just define an implicit conversion:

137

Note that this is only possible if Name is defined to be a subtype of String. Since toString is a method on Any, which is a supertype of all types, including opaque types. The return type of toString must conform to String, so it can only be Name if Name is a subtype of String.

I think the only reasonable solution is that exporting methods should always be an active decision, and if there is some kind of shorthand to export some methods, then the type signatures should really be unchanged unless explicitly overridden. Having the compiler guess what you mean will only lead to confusion.

I have a draft for a proposal which explores this here if anybody wants to chip in, it’s far from done though, and I really should be spending my time on other things:

138

As I’ve remarked on that Transparent members for Opaque types proposal, I think there’s a much simpler solution here. It seems that the one problem not being presently solved by opaque types is addressing primitive obsession without explicitly forwarding each method.

If that is the case, then just providing OpaqueString, OpaqueInt et cetera on a library takes care of it. It can also address some gotchas I mentioned, such as contains on String taking CharSequence instead of String, arithmetic operators on number types having overloads for other number types, and things like >>(n: Int): Int on Int having to keep n as Int but replace the return type with the opaque one.

Someone ought to write a library for Scala 3 with the most common use cases for opaque types.

139

I am bit dumbfounded. Shouldn’t this not compile?
So it seems you can’t have a generic way of creating newtypes, based on the similar examples above of creating new types.

140

That’s because == is not “type safe” in Scala 2, and Scala 3 wants to be backwards compatible. However in Scala 3 you can enable strict equality and then you will get the result you expect: Scastie - An interactive playground for Scala

An alternative that is safe by default is defining a CanEqual instance for your opaque types (Scastie - An interactive playground for Scala)

trait NewType[Wrapped] {
  opaque type Type = Wrapped
  def apply(w: Wrapped): Type = w
  extension (t: Type) def unwrap: Wrapped = t
  given CanEqual[Type, Type] = CanEqual.derived
}

edit: Actually if you enable strict equality you also need a CanEqual instance in order to be able to compare Name to Name and Email to Email. So perhaps the best definition of NewType is:

trait NewType[Wrapped] {
  opaque type Type = Wrapped
  def apply(w: Wrapped): Type = w
  extension (t: Type) def unwrap: Wrapped = t
  given (using CanEqual[Wrapped, Wrapped]): CanEqual[Type, Type] = CanEqual.derived
}

That way your opaque type can only be compared to itself if the wrapped type can also be compared to itself.

5 Likes
141

Hello, Throwable, I didn’t see you there :unamused:

142

The second example is a neat solution. Works for both worksheet and sbt projects now.