I agree that pattern matching against an opaque type pattern should give at least an unchecked warning: https://github.com/lampepfl/dotty/pull/10664
Would it be possible to make it illegal? People that want to do a type coercion could always use asInstanceOf
. Maybe wouldn’t be so bad to have some precedence for illegal type testing? People defining opaque types could always define TypeTest
instances if they want type tests to be possible.
Ignoring unchecked warnings is already unsafe, so I don’t think that going further for one special case is useful, perhaps there’s a debate to be had on whether unchecked warnings should be errors by default but that’s straying off topic.
Yeah that’s fascinating. I’ve had plenty of experiences where avoiding boxing made a significant perf improvement (very rigorously validated by JMH), and I’ve also had experiences where I’d a bunch of work that is undoubtedly a theoretic improvement only the see the results either not change or even get worse sometimes. Part of me thinks domain vs generic is a heuristic for determining with boxing will make a difference but I wouldn’t put much faith in that, even myself. The JVM still perplexes me to this day. Anyway…
I did not close any PR or issue, or shut down any discussion, just offered my opinion. Others are free to disagree.
100% and I want to emphasise that I haven’t seen you personally close a PR or shut down a discussion. As far as I can see, no one at all has shut down any discussion on this topic yet. To clarify, the reason I mentioned that at all is that it’s a pattern I’ve seen a few times where a discussion concludes but, for right or wrong, a majority (or maybe just a very large group) are still unconvinced, and the documentation and/or the community fail to sway new users who come across the same common use case. In those situations, PRs and discussions do end up just getting shut down because in the eyes of maintainers, the ship has sailed, the debate has been had and there’s no value in repeating it. To me this issue is (was?) starting to look like it would go down that path which is why I want to highlight it now so that even if we don’t modify the implementation, we beef up the documentation to transparently best address those common use cases as best we can, even if a significant amount of people disagree that opaque types are relevant to those use cases.
Fair enough. I think it’s safe to say that I already derailed this topic to the limit already.
Before I get a grip on myself and stop spamming I just want to say thanks to @odersky and @smarter for taking the time to address this despite everything on your plate with the Scala 3 launch! As a newcomer on this forum, that feels great!
Some new info gained from experiments is here: https://github.com/lampepfl/dotty/issues/10662#issuecomment-739852480
This is very interesting. Am I correct in assuming this deals with edge cases where the newtype-like opaque type usage is unsafe? (I’m missing the tl;dr)
Yes, that’s right. I’ll try to summarize:
EDIT: To clarify, the solution makes it safe to emulate Haskell’s newtype
using opaque type
, but only if no upper type bound is exposed, like in Haskell which has no subtype polymorphism at all, only parametric polymorphism.
Summary
Problem
The semantics of opaque type
is not the same as newtype
in Haskell. opaque type
is actually much more general. The main reason opaque type
doesn’t do encapsulation in the same way newtype
do is that since Scala allows you to pattern match on litterally Any
thing, you can expose the underlying type representation (intentionally or by mistake) using pattern matching.
Quoted example (minimized)
For this reason, users using the pattern of defining a constructor/unapply
method will experience some gotcha moments when opaque types don’t behave like case classes would have.*
Solution
This is solved in two steps:
- Disallow opaque types in typed patterns (like the one above).
- Restrict
Any
so that only subtypes of a new traitMatchable
can be used as the scrutinee in pattern matching.
(1) prevents conversions like String -> Name
by warning on patterns like:
(str: String) match {case n: Name => n}
Because Name
can’t appear in a typed pattern in the case
clause anymore.
(2) prevents conversions like Name -> String
by warning on patterns like:
(n: Name) match {case str: String => str}
Because Name
isn’t a subtype of Matchable
so it can’t be pattern matched on at all.
Demo: (minimized)
scala> object n :
| opaque type Name = String
| object Name :
| def apply(str: String): Name = str
| def unapply(n: Name): Some[String] = Some(n)
// defined object n
scala> import n._
scala> "hi there" match {case n: Name => n}
1 |"hi there" match {case n: Name => n}
| ^^^^^^^
| the type test for n.Name cannot be checked at runtime
val res0: String & n.Name = hi there
scala> Name("Franz Kafka") match {case str: String => str}
1 |Name("Franz Kafka") match {case str: String => str}
| ^^^^^^
| pattern selector should be an instance of Matchable,
| but it has unmatchable type n.Name instead
val res1: n.Name & String = Franz Kafka
See this for more:
Note that this isn’t in master yet, and warnings won’t be turned on in 3.0.
*This is also why case classes should still be encouraged and preferred when their semantics is needed! Some kind of sugar like case opaque type
could maybe be added in the future when the semantics of opaque type
are better understood in practice and the feature has matured more. Preferably these unsafe patterns should be compiler errors rather than just warnings before such sugar is added.
Matchable isn’t enough to make all usages of opaque types safe as I mentioned in Add `Matchable` trait - #3 by smarter, the problem is that users are free to define a visible upper-bound for their opaque type which is itself a subtype of Matchable.
Yes, it is enough to emulate Haskells newtype
though (unless I’m mistaken?) since that doesn’t support type bounds anyway.
I have the feeling that this discussion, like the previous ones on the topic, has failed to conclude one major question – What is the main motivation behind opaques?
The original SIP explains the motivation as so:
Authors often introduce type aliases to differentiate many values that share a very common type (e.g.
String
,Int
,Double
,Boolean
, etc.). In some cases, these authors may believe that using type aliases such asId
andPassword
means that if they later mix these values up, the compiler will catch their error. However, since type aliases are replaced by their underlying type (e.g.String
), these values are considered interchangeable (i.e. type aliases are not appropriate for differentiating variousString
values).
The current design of opaque types does not achieve that in the slightest, IMHO, and for one simple reason – opaques do not provide access to the methods of the underlying type (like type aliases do).
If better performing value-classes/wrappers is what people want, then there’s nothing wrong with the current design, but please let’s at least have the motivation for opaques cleared up and updated.
It’s rather boilerplate heavy, as (at least as I understand it) macros can’t currently produce definitions and you have to re-wrap the type after each call, but you can use them in a way that provides access to the methods of the underlying type:
object types {
object Name {
opaque type Type <: String = String
def apply(s: String): Type = s
}
type Name = Name.Type
object Email {
opaque type Type <: String = String
def apply(s: String): Type = s
}
type Email = Email.Type
}
object using {
import types.{Name, Email}
@main
def test (): Unit = {
def foo(str: String): Unit = {
println(str)
}
def bar(n: Name, e: Email): Unit = {
println(s"$n @ $e")
}
val name = Name("JDoe")
val email = Email("[email protected]")
//foo(name)
// Found: (name : types.Name.Type)
// Required: String
//foo(email)
// Found: (email : types.Email.Type)
// Required: String
foo(name)
foo(email)
bar(name, email)
println(name.toLowerCase())
//bar(name.toLowerCase(), email)
// Found: String
// Required: types.Name
bar(Name(name.toLowerCase()), email)
//bar(email, name)
// Found: (email : types.Email.Type)
// Required: types.Name
// Found: (name : types.Name.Type)
// Required: types.Email
}
}
You can extract the boiler plate into a trait::
trait OpaqueOf[A] {
opaque type Type <: A = A
def apply(s: A): Type = s
}
object Name extends OpaqueOf[String]
type Name = Name.Type
object Email extends OpaqueOf[String]
type Email = Email.Type
And it can be extended further. One could add a validating function to go from A
to the opaque type, for example. I think that isn’t the biggest issue here. The criticism, as I understand it, is that you can’t do this:
object types {
object Duration {
opaque type Type <: Double = Double
def apply(n: Double): Type = n
}
type Duration = Duration.type
}
object using {
import types.Duration
@main
def test(): Unit = {
val d1 = Duration(5.0)
val d2 = Duration(2.0)
val d3: Duration = d1 + d2 // should compile but doesn't
println(s"d1: $d1 d2: $d2 total: $d3")
val d4 = 10.0 + d1 // should not compile but does
}
}
My understanding is that this is less than ideal because it will force primitives to box. I’d normally use this for stuff that’s already objects, so it’s less of a problem for me.
With a minor tweak, it does work. The missing step is that the result needs to be re-wrapped because the result of _: Duration + _: Duration
is a Double
- which makes sense because operation on the value may make it no longer valid, like there’s no way to guarantee that (_: Email).take(5)
is still a valid Email
:
object types {
object Duration {
opaque type Type <: Double = Double
def apply(n: Double): Type = n
}
type Duration = Duration.Type
}
object using {
import types.Duration
@main
def test(): Unit = {
val d1 = Duration(5.0)
val d2 = Duration(2.0)
val d3: Duration = Duration(d1 + d2)
println(s"d1: $d1 d2: $d2 total: $d3")
val d4: Double = 10.0 + d1
val d5: Duration = Duration(10.0d + d1)
}
}
The d4
compiles because d1
known to be a Double
subclass, which is usually fine for these sort of wrappers. Usually what you’d want to avoid is mixing subclasses, or passing a Double
where you want to be sure that it’s a valid Duration
.
Edit: sorry for being scatter-brained, here’s the Scastie link
This can be reduced to
object types {
opaque type Name <: String = String
def Name(s: String): Name = s
opaque type Email <: String = String
def Email(s: String): Email = s
}
It’s boilerplate heavy if you wish for better performing value-classes; it’s irrelevant if you wish for non-interchangeable type aliases.
It’s boilerplate heavy if you wish for better performing value-classes; it’s irrelevant if you wish for non-interchangeable type aliases.
I can’t think of any evidence or reasoning to support that point of view. How can you objectively declare how much boilerplate is relevant for each (hair-splitting) case? There’s nothing inherent about a non-interchangeable type alias that means any amount of boilerplate is irrelevant, and nothing inherent about value-classes to mandate the opposite. Maybe you can explain your reasoning a bit more?
To me: boilerplate is boilerplate. I don’t care about the categorisation of the domain I’m applying it to. I just think “well this could be automated but isn’t” and then I think about stuff like {frequency,ease} of {creating,modification} relative to value it provides.
The current design of opaque types does not achieve that in the slightest, IMHO, and for one simple reason – opaques do not provide access to the methods of the underlying type (like type aliases do).
Don’t you think this is a wild exaggeration? Opaque type aliases give you abstraction without overhead, which is exactly what they set out to do. You are free to define an API by defining methods that operate on the opaque type, where you also specify type signatures.
it’s irrelevant if you wish for non-interchangeable type aliases.
What do you mean by non-interchangeable type alias? How would they work? What is the semantics?
If I define an opaque type:
object foobar:
opaque type Foo = Bar
Do you want Foo
to unconditionally simply have all methods of Bar
with all occurrences of Bar
changed to Foo
?
I’m genuinely trying to understand what it is you want, because I don’t see what it is you want to do that can’t be done in the scope of the current proposal.
There’s nothing inherent about a non-interchangeable type alias that means any amount of boilerplate is irrelevant
What I meant is that the boilerplate of the current proposal (of opaques) has nothing to do with non-interchangeable type alias, as the current proposal does not seem to address that particular feature/use case/motivation. Obviously, if and when that feature will be addressed, boilerplate would be something to take into account.
Don’t you think this is a wild exaggeration?
I do not. I would not use (the current design of) opaques for the purpose described in the original SIP (non-interchaneable type aliases), for the same reason I never used value-classes for that purpose nor mere wrapper classes. There is a very basic assumption that I believe is incorrect in the original SIP:
One appropriate solution to the above problem is to create case classes which wrap
String
.
That is, In my opinion, a workaround. Not an appropriate solution. And if for some reason I considered that an appropriate solution, I would still use mere wrapper classes and not opaques, as I would prefer their familiarity and simpleness over opaques, which only give a performance benefit I do not care for (and believe that most should not care for it either in most cases).
What do you mean by non-interchangeable type alias? How would they work? What is the semantics?
Authors often introduce type aliases to differentiate many values that share a very common type. […] However, since type aliases are replaced by their underlying type, these values are considered interchangeable.
I would have hoped that this SIP would address exactly those questions – how would they work and what would be the semantics – and try to come up with a design that focuses on providing a solution to the core motivation presented. Instead, it focused on providing a solution to a minor problem (performance) to one of the workarounds for the core motivation (wrapper classes).
I would have hoped that this SIP would address exactly those questions – how would they work and what would be the semantics – and try to come up with a design that focuses on providing a solution to the core motivation presented.
I think that’s what’s been done. Opaque types allow you to define a type alias which is interchangeable in the scope they are defined, and to be non-interchangeable outside of that scope. It’s a very general and versatile solution with far broader use cases than value classes, which is why so many people have been arguing against weakening them to better performing value classes a la Haskell’s newtype
by synthesizing constructors for them by default. (which is the whole point of this thread)
Instead, it focused on providing a solution to a minor problem (performance) to one of the workarounds for the core motivation (wrapper classes).
As I just said, opaque types are much more general than better performing value classes.
After reading your reply, I’m still non the wiser as to what you think is missing. It’s very hard to figure out a solution with well defined semantics without understanding what the problem you want to solve is.
So:
What is the problem that you want to solve? Opaque types give you non-interchangeable type aliases. Do you want to somehow expose all methods from the underlying type? What is it that I can’t do in the current proposal that you want to be able to do?