The very first proposal for this feature used something similar to what you propose. However, it was rejected precisely because there was no way to provide class-like semantics (notably, correct instance tests) while gaining the performance guarantees and the other semantics that we wanted. It turns out that correct semantics are precisely the ones that a type alias already has, and therefore the concept of opaque type alias was much more appropriate. An opaque type alias really is just a type alias that we made opaque. It is nothing like an independent class.
Could you please elaborate on that?
Obviously an “opaque” class has different semantics than a regular one:
- It can extend final classes.
- It extends classes; traits can only be mixed in.
- It extends everything from the super class privately.
But case class also has different semantics than regular ones. I still find this idea more closely related to classes than to type aliases with extension methods, and more importantly – easier to grasp this way.
Ah wait, I think I understand what you are referring to - that this is in fact not possible to assert:
obj.isInstanceOf[Logarithm]
Which is the same for type aliases.
Thing is, this is really confusing to think of them as aliases. The whole idea with type aliases is that one can replace them in the source code with the original type and the program will not change. This is not true for opaques. In most cases, it would be impossible to retain the program’s semantics by replacing them with the original type; in fact, instance checking is the only place where they share this similarity with aliases.
How about thinking of them as classes but adding a compiler warning whenever they are used in instance checking?
obj.isInstanceOf[Logarithm] // warning
case log: Logarithm => // warning
IIRC: Anything requiring a runtime distinction will break. classOf[T] and instanceOf[T] are some examples, but not all. I think there are were quirks with AnyValue types and boxing as well.
Opaque types are not classes – classes/traits exist in the bytecode. Opaque type aliases are purely on the compiler side, just like type aliases.
I suppose the purpose of the Opaque vairant is explicitly for this not to be true.
I do like the simplicity of a ‘class like’ syntax. I don’t know if there is a way to get such syntax without implying incorrect semantics and strange exceptions to existing rules.
Bytecode representation is an implementation detail; semantics shouldn’t be decided according to it.
Opaques are not really classes nor aliases. They are unique. The question is whether this merit an entirely new syntax or some conjunction of existing constructs with a small amount of new constructs. If we go for the second option, then the question is which is the most fitting existing construct.
Thinking of opaques as aliases is confusing to me:
type AliasString = String
opaque type OpaqueString = String
// conversion to
val obj: AliasString = "hey" // compiles
val obj: OpaqueString = "hey" // doesn't compile
// conversion from
val alias: AliasString = ???
val op: OpaqueString = ???
val obj: String = alias // compiles
val obj: String = op // doesn't compile
// invocation
alias.toLowerString() // compiles
op.toLowerString() // doesn't compile
Aliases and opaques are for the most part quite different.
Using extension methods also seem like an ad-hoc ability rather than something that is an integral part of opaques and their intended use (opaques without additional methods have no use).
Perhaps we can just drop the class, this and super?
opaque Permission extends Int { i =>
def |(other: Permission): Permission = Permission(i | other)
def isOneOf(permission: Permission): Boolean = (i & permission) != 0
}
1 Like
Nope, as mentioned before, outside of their definition they behave exactly like abstract type aliases do. You can already make opaques like this:
object Permission {
type Permission
def apply(i: Int): Permission = i.asInstanceOf[Permission]
private[this] def unwrap(p: Permission) = p.asInstanceOf[Int]
def (i: Permission) | (other: Permission): Permission = Permission(unwrap(i) | unwrap(other))
def (i: Permission) isOneOf (permission: Permission): Boolean = (unwrap(i) & unwrap(permission)) != 0
}
And various tagging libraries like GitHub - estatico/scala-newtype: NewTypes for Scala with no runtime overhead let you do the same with less boilerplate.
3 Likes
More so, the one specific feature of opaque types have is that they’re “opaque” outside of their definition, but are transparent inside their containing class:
object outer {
object inner {
opaque type T = Int
implicitly[T =:= Int] // found, T = Int here
}
implicitly[inner.T =:= Int] // NOT found! T != Int here
}
However, this is also not different from how typealiases behave – it’s just upcasting:
trait Opaque {
type T
}
object outer {
val inner: Opaque = new Opaque {
override type T = Int
implicitly[T =:= Int] // found, T = Int here
}
implicitly[inner.T =:= Int] // NOT found! T != Int here
}
The type ascription on val inner: Opaque is what erases the knowledge of T=Int and forces inner.T to be treated like a unique unknown type – yet inside the definition of inner the knowledge is preserved. What we can’t do in Scala though is declare an object with an upcasted type, so short of stuffing lazy vals into a package object we can’t declare such upcasted opaques at top-level. Even without opaques dotty makes this pattern better by allowing to declare top-level vals – and perform upcasting, opaque types themselves just make this, rather obscure, pattern actually tolerable to execute be inventing special syntax for it.
4 Likes
This is very interesting, I haven’t though of that. What is then the purpose of the new feature if it’s already possible to have opaques? Either:
- This implementation is still not as efficient as could be? (which is the major motivation for the feature / design pattern).
- There’s too much boilerplate, in which case it seem to me that the new syntax is about the same.
Type members and type aliases may use the same keyword (type), but they are not the same; they behave differently and are used for different purposes.
Indeed type members seem more closely related to opaques, but the current implementation – much like the currently proposed feature – does not really acknowledge opaques as a thing of its own but rather as a design pattern achieved by combining other features (extension methods).
If we were to introduce a new syntax just for this design pattern, why not go all the way and include everything that the design pattern requires?
It’s also interesting to note how this library chose to encode “new type” in its newer version – via a wrapper case class with normal methods. The underlying implementation that uses type members and extension methods is invisible.
Replace “bytecode” with “runtime” and my point still stands.
As for whether semantics should have anything to do with it, they MUST. The entire reason for this proposal’s existence is due to a performance concern. Every use case can be handled today using a simple wrapper class, like class Logarithm(d: Double) { ... } – other than the performance concern from wrapping the value.
In order to ensure that this feature never requires boxing, and is purely a compiler side fiction, there must not be a runtime representation. This contrasts with extends AnyVal which has runtime representation, boxing, etc. One can not have an open ended discussion of semantics here: the requirements define the semantics. Syntax is of course always open to bike shedding, but semantics in this case are not.
An opaque type is not a class. It can’t be an ‘opaque class’ – classes by definition have distinct identity, and support runtime type inspection.
These are compiler-only types, with no runtime representation. There is an existing feature for such things.
Don’t think of them as aliases. Perhaps Opaque Type Alias is the wrong name. How about Opaque Type? I’m not sure what to name of the superset of Type Alias and Opaque Type – both are user provided types with no runtime representation.
I have never thought of them as aliases, just compiler-only types with differing visibility rules.
What I do find confusing syntactically is how one has to create a companion object for the type to define its member definitions, but its hard to argue with a simple extension of what exists today that mostly just tweaks visibility rules.
3 Likes
Again, it seems that opaques share similarities and differences with all of the other components (classes, type aliases and type members), but they are neither. They have their own unique semantics.
I believe that associating them with type would be confusing, as we can already see here the confusion between type aliases and type members, which are different but are mistaken as similar since they share the same keyword. Let’s not add yet another meaning into that soup.
As @kai shown, they could actually be achieved with Scala 2 (given that the performance is the same as what is proposed), so it’s only a matter of syntactic sugar. If that’s what this all about, I suggest making their syntax tailored specifically to their use case:
opaque Permission(i: Int) {
def |(other: Permission): Permission = Permission(i | other)
def isOneOf(permission: Permission): Boolean = (i & permission) != 0
}
object Permission {
// provided by default and can be overridden with a different implementation
def apply(i: Int): Permission = i
// not provided by default
def apply(name: String): Permission = ???
val WriteOnly = Permission(2)
val ReadOnly = Permission(4)
}
import Permission._
val readWrite = WriteOnly | ReadOnly // compiles
val readWrite: Int = WriteOnly | ReadOnly // doesn't compile
ReadOnly.isOneOf(readWrite) // compiles
ReadOnly.isOneOf(6) // doesn't compiles
ReadOnly.toDouble // doesn't compile
2 == WriteOnly // compiles and returns `true`
obj.isInstnaceOf[Permission] // doesn't compile
obj match { case p: Permission => ??? } // doesn't compile
With sugar like that, and the other changes going into Dotty (‘new’ not needed when calling a constructor when not ambiguous, top level definitions), I think the companion object can be dropped.
An ‘opaque’ holds its methods in static scope, so holding values there is just more sugar:
opaque Permission extends Int {
require(this >= 0) // example of input validation
def this(name: String) = ??? // alternate construction
def |(other: Permission): Permission = Permission(this | other)
def isOneOf(permission: Permission): Boolean = (this & permission) != 0
// note, these values are static, just like the defs above. These can not reference `this`
val Write = Permission(2)
val Read = Permission(4)
val Execute = Permission(8)
}
import Permission._
val rw = Write | Read // compiles
val rw: Int = Write | Read // doesn't compile
val rwx = Write | Read | Execute
Read.isOneOf(rw) // compiles
Read.isOneOf(6) // doesn't compiles
Read.toDouble // doesn't compile
2 == Write // -- does not compile the author can add `def toInt: Int` or `def isInt(other: Int): Boolean` if this is a real use case. Casting from an opaque to Any is not safe.
2 == (Write: Any) // compiles, returns true
obj.isInstnaceOf[Permission] // doesn't compile
obj match { case p: Permission => ??? } // doesn't compile
I can’t disagree more. The entire point here is to introduce a new type that points to some underlying one. The keyword type is very precisely correct and clear…
5 Likes
I think that might be a bit confusing; how come a def has access to this but val doesn’t? I understand why that is but that seems like an irregularity comparing to classes and traits which share a similar syntactic structure.
I actually now think that it shouldn’t extend the wrapped type, as this gives the notion of a class, which it’s not. Also, the implementation there should be Permission(super | other); otherwise, this is endless recursive invocation.
Having the wrapped value as a “member” is better in multiple ways, I believe:
- No clear association with classes and inheritance.
- Actually represents the concept better - a type that wraps another type.
- Makes it clear that this type has only one immutable member (no other state).
- Reduces the need for
thisandsuper, as the wrapped type is a parameter with a name.
I guess I don’t have a strong opinion on that one. I’ll leave that to the discussion of multiversal equality.
It is very clear when you only consider this feature. When you consider the other features as well – type aliases and type members – this is becoming confusing, as they are all different things that behave differently and for different purposes.
Why not call everything a type? Why do we have classes and traits? Maybe we should just encode everything with zeros and ones, that is what this is all about after all.
This is yet another case of polysemy in the language (the other being implicit). Conflation of meanings, even if they share some similarities, is confusing and unproductive. We are not short of words, so why not use them?
An opaque type alias has the compile-time typing rules of an abstract type member, and the run-time semantics of a type alias.
Once you know that, everything is clear, and opaque type alias is a perfect name for this hybrid.
3 Likes
@sjrd has the precise definition, but to amplify that: the distinction between types and classes is central to Scala. It’s one of the topics I usually teach to folks the first day. They’re fundamentally different concepts.
This isn’t squishing everything into the word type – that word means something precise, specific and important, and it’s being used correctly here.
2 Likes
It might be clear for someone who’s working on the internals and is very proficient in Scala; it is not so clear for most developers. Just look a few comments above and see the confusion between type aliases and abstract type members.
And no, it doesn’t have the compile-time rules of an abstract type member. An abstract type member is just that – abstract – and only comes into play in conjunction with inheritance (anonymous or not). With opaques there is no inheritance; they are wrappers.
I don’t dispute that classes and types are different, but rather the other way around. My point is that we acknowledge the different between them and give them different words, and we should exactly the same for abstract type members, type aliases and opaques.
This attempt of linking these different concepts with the same keyword seem to me tantamount to trying and removing class from the language and replacing it with type. It wouldn’t be so hard to do as the two are mostly used with different constructs so there won’t be much (compiler) ambiguity. Obviously, that is a bad idea, and I pointed it out to try and illustrate why I think using type for opaques is a bad idea as well.
Anyhow, I don’t horribly mind it if the syntax looked like that:
opaque type Permission(i: Int) {
def this(name: String) = ??? // alternate construction
def |(other: Permission): Permission = Permission(this | other)
def isOneOf(permission: Permission): Boolean = (this & permission) != 0
}
object Permission {
def apply(yetAnotherConstructor: Double): Permission = ???
val Write = Permission(2)
}
this could be replaced by something else. Having a fake constructor as in your example is confusing – the parameter can not be modified on input, it can not be public, val, var, etc… it shares nothing with an ordinary constructor list other than naming the variable.
this was the first thing that came to mind.
This construct shares more with object than traits or classes, so I am not concerned with objections about dissimilarity with trait or class semantics.
Yes, extends is not the best word. It was an existing keyword, I did not want to make a new one. I guess for or as may be more appropriate
opaque Permission as Int ?
No. this is of type Int not type Permission, so no recursion. That may be confusing, maybe a different name should be used, like inner or wrapped.
To be honest, I dislike the ‘fake member’ as much as I dislike ‘this’. I am not convinced whether a user supplied name is better than a fixed one either way. Both have advantages and disadvantages.
The scope is far larger than multiversal equality. See the discussion in this thread in the past. I would not want an Akka like send def send(Any: a) = ... to accept opaque types. Wrap it in a case class, or explicitly unwrap to int. Its not type safe to upcast these. I wouldn’t even want them to work in something like `s"{someOpaque}" unless that opaque explicitly has its own toString.
In short… I don’t think they are true subtypes of Any, because the language relies on runtime information for dispatch in many places (esp. pattern matching and OO polymorphism), and without runtime information they can not conform to the contracts of Any.
Thus, these values are not true subtypes of Any. You have to unpack one to get an Any value out, but can create a List[Permission] without doing so. Unfortunately, there is no escape from lack of safety, unless you forbid treating the type parameter in List[+A] covariantly if it is inhabited by an opaque type. There are probably other holes too, especially with union and intersection types. I don’t know what it would look like to try and patch them all or if its possible.
But, if the blatantly obvious passing of an opaque value to its ‘wrapped’ type is blocked, why not any other casting?
val x: Logarithm = ???
// why block this:
def square(d: Double) = d * d
square(x) // can't do it!
// what about this ?
def sneaky(d: AnyVal) = d match { case Double => d * d }
sneaky(d)
// or this?
def sneakyAny(d: Any) = d match { case Double => d * d }
sneakyAny(d)
I don’t understand why its not ok to supply a Logarithm for a Double parameter, but it is OK to substitute it for an Any parameter.
This is a compiler-only type, a user should have to unwrap or convert it before leaving where the compiler can track its type. IMO all the above should be invalid. Its not safe to implicitly erase an opaque type.
1 Like
I actually think the entire point is that they are not wrappers.
They are more like a disguise or costume. Hide the true runtme nature of the type from the compiler, and provide a different set of methods visible to the compiler.
5 Likes