From the moment you introduce asInstanceOf you could do
val x: Top = Logarithm(10)
try {
val y = x.asInstanceOf[Double]
...
} catch {
case e: ClassCastException => ...
}
Though granted, it’s a lot harder to fall in that loophole by accident. But you can still write the unsound code.
@Jasper-M Sure. That’s the purpose of asInstanceOf. It’s the universal escape hatch, which comes with no guarantees.
I agree this is not great, but this kind of pattern-matching can already defeat attempts at parametric design by allowing matching on post-erasure types:
def weird[A](xs: Array[A]): Array[Object] =
xs match {
case ys: Array[Object] => ys
}
weird(Array("arrays", "are", "invariant"))
// Array[Object] = Array("arrays", "are", "invariant")
def weirder[A](a: A): AnyRef = a match {
case y: Object => y
}
val x: Int = 3
weirder(x) // AnyRef = 3
From my point of view, if a caller wants a type that has a representation that can be matched at runtime, they should be using case classes or similar, not opaque types.
Is it possible to provide an explicit =:= between the underlying type and value type?
=:= will bring the ability of substitute, which is useful when you want to lift a type class.
You could have an opaque type and provide pattern matching on it via an extractor that converts it to something pattern-matchable.
opaque type Maybe[A] = Option[A]
object Maybe {
def unapply[A](ma: Maybe[A]): Some[Option[A]] =
Some(ma)
}
// then
val ma: Maybe[A] = ???
ma match {
case Maybe(Some(a)) => ???
case Maybe(None) => ???
}
Of course, it would be nice if extractors statically known to return Some (such as the one above) would preserve exhaustivity checking (see #10502).
Whether you can soundly substitute is a big sticking point, and it affects all sorts of things other than typeclasses; this was the major problem with “unboxed value classes” that was fixed so nicely by the followup opaque types proposal.
So returning to unboxed value classes would restore those problems as well, which go beyond merely whether the compiled code happens to box them.
You can do this with abstract types, too, in current Scala. It would be nice to be able to ban that, too.
That you can optionally supply =:= or <:< instances in the type companion providing unidirectional or bidirectional substitution as part of the interface is one of the great features of opaque types as currently proposed; it’s a sound way to declaratively expose part or all of the equality. (The whole substitute design space is supported, indeed.) So you can choose exactly how much of Logarithm = Double can be seen in the interface, and the language for doing so is simply defining methods.
I sure wouldn’t mind banning arbitrary type patterns and [ai]sInstanceOf calls, though.
If parameteric Top is really being proposed instead of opaque types can we see exactly how it can replace?
We would do something like:
class Logarithm(private val exponent: Double) extends Top {
def *(that: Logarithm): Logarithm = new Logarithm(exponent + that.exponent)
}
object Logarithm {
def fromDouble(d: Double): Option[Logarithm] =
if (d > 0.0) Some(new Logarithm(math.log(d))
else None
}
Is this what we are saying? And somehow extending Top means new Logarithm does not allocate? That seems a little weird to me (new not allocating and also maybe it precludes classes with two parameter constructors extending Top, which is also weird if it is a true top type).
I am a bit concerned about this because Top seems so ambitious it might never happen, or take very long, while working opaque types is something many in the community would really like to use now.
I think if this is used as a reason not to do opaque types it would be very nice to see a clear alternative proposal of how to implement zero-cost opaque types using Top.
Perhaps for Scala 2, the syntax extends Top can be used to create opaque types, analogous to extends AnyVal, but without yet truly truly introducing the full glory of Top as the true top type. The idea would be to have all the features of the opaque types proposal, but with the same syntax that Dotty would have for defining them.
There are probably some serious drawbacks with this that I am blind to.
+1 on parametric Top (at least for Dotty).
If A meant A <: Top, that pattern matching must be forbidden to ensure parametricity. A probably safe and overly conservative rule would forbid matching on scrutinees whose type mention any parametric type variable. Here a type match on Array is safe, but drawing good rules for full Scala in general seems nontrivial.
@oscar IIUC, the idea here is “value classes that truly never box” — most details are in doc linked earlier, and indeed new does not box: scala.github.com/_sips/sips/2017-09-20-opaque-types.md at f61ec0dd31c9d650dc867dcb5e74506e7c26f7be · non/scala.github.com · GitHub
@scottcarey extends Top and opaque types don’t seem to have quite the same semantics… or do they?
EDIT: more importantly, this proposal without a parametric Top would give you classes with questionable semantics — you still have all methods from Any, but they have inappropriate semantics, the ones given by the implementation, something a true parametric Top avoids. Though I suspect the earlier opaque types, and the encoding from @S11001001’s blog post, share this downside.
In all these proposals, it seems polymorphism over primitives must still box.
How does one implement asInstanceOf on primitives without at least one level of boxing?
I guess in all these proposals, if A is a type variable, a is a primitive (wrapped in a “newtype” or not) and a: A , a must have been boxed anyway, but reference types need no extra boxing. (In miniboxing A <: AnyVal is erased to long, but it seems miniboxing is not going to be merged).
The only difference (maybe) is that if you use some suitable specialization (Scala 2’s, miniboxing, Dotty-linker) and have parametric Top, the optimizer has a easier time removing boxing. Casts to AnyRef would still box, and the results of reference equality would be affected, but those results shouldn’t be guaranteed under parametric Top (not sure of the current actual guarantees).
Here’s a possible way to help migration to the new scheme:
[T] means [T <: Top]
Add a deprecated implicit conversion in Predef
@deprecated def topToAny(x: Top): Any = x.asInstanceOf[Any]
special case pattern matching, so a scrutinee bounded by Top is converted to Any, using the same deprecated conversion.
That should cover the majority of problematic cases. There will be a few that remain, but overall it seems doable.
@blaisorblade asInstanceOf is by definition unsafe. It would simply succeed if the underlying type is compatible with the representation type. So, no boxing is needed.
So assuming then that
class Foo(x: Double) extends Top
does not box, then what is the difference between:
implicit class Foo(x: Double) extends Top
implicit class Foo2(x: Double) extends AnyVal
We now have two ways to do method enrichment? That seems like a shame. I guess you should never do that after Top. Is that right?
Secondly, I find it ugly that now we would have a class that never allocates. Previously in scala class was something that could (in principle) be an allocation. Even with extends AnyVal in a generic context we get allocation. All class declarations correspond to some kind of JVM class entity.
To me, the opaque type proposal is much cleaner here because we already have type, and it never boxes or allocates (as far as I know). With opaque we are just putting additional constraints on a type but otherwise the rules apply. Similarly, we keep the notion that a class in scala corresponds to something that can be a JVM class.
I should add that while substitute is sufficiently powerful, it’s not necessarily convenient.
This example from the other thread which runs as written with opaque types:
def mdl(m: Map[Double, Logarithm]): Map[Logarithm, Double] = m
Suppose alternatively a =:= is provided and GADT-style =:= is made to work in patmat, it must be written
// supposing Logarithm.repr: Double =:= Logarithm
def mdl(m: Map[Double, Logarithm]): Map[Logarithm, Double] =
Logarithm.repr match {
case Refl() => m
}
Supposing a =:= is provided but GADT-style is not made to work:
def mdl(m: Map[Double, Logarithm]): Map[Logarithm, Double] = {
type F[A] = Map[Double, A] =:= Map[A, Double]
Logarithm.repr.substitute[F](implicitly)(m)
}
Scala users might have some difficulty coming up with the correct F for various scenarios.
If we started from scratch that would still be my preferred scheme, but it’s probably too hard to migrate to it. So in what follows I propose to leave Any as it is and introduce a new, fully parametric type Top.
I’m not so sure … I think the complications of adding a new Top type
to Scala are at least as great as changing the semantics of Any and
adding AnyObj to match Dotty.
It’d be interesting to make the change and try it with a few
representative projects and see how much breaks.
Cheers,
Miles
I’m probably not making myself clear — simply stated, the JVM does not allow checkcast on primitives, so they have to be boxed once. Or do I miss something? That boxing is not due to newtypes but is still relevant.
In particular, saying that “newtypes never box” is as misleading as “primitives never box”. And since really never boxing is the goal, all the newtype proposals are necessary but not sufficient without specialization.
I view extends Top as a better version of extends AnyVal — there’s a tradeoff in expressivity, but I’d never use AnyVal. extends Top or opaque types don’t let you change equals, but if you want the features of either, you need to switch to typeclass-based equality (not multiversal equality).
Type itself doesn’t, but you still box in generic contexts unless you specialize:
type T = Int // defined type alias T
def foo[U](u: U) = u //foo: [U](u: U)U
foo[T](1) //res0: T = 1 //boxing HERE
A better discussion of what boxes when is in the README for GitHub - hobwekiva/newtypes (@jvican that might help for revisions of the SIP) — I doubt you can improve on the generated code much.
I think that might be a valid and important usability concern about the concrete syntax. Maybe this should be written type Foo(a: Int) extends Top, newtype Foo(a: Int), or yet something else. In all cases, Foo would extend Top but not Any/AnyVal/AnyRef.
As long as it’s not forgotten about, this should be discussed separately so that we get the right semantics before bikeshedding the details (Pre-SIP: Unboxed wrapper types - #52 by jvican). And I’m not implying at all concrete syntax doesn’t matter, it’s just harder to discuss.
EDIT: @jvican is the plan to sketch out this proposal before choosing? It can be fine if Martin leaves details as exercises to the reader, but we still need to see the worked out version. I understand parts of the result doesn’t look very different from your earlier document on “newtype classes” ( extends Newtype), but that’s not clear. Also, details on Top are spread in a discussion over the Dotty repo that didn’t quite reach a conclusion.
Top: They are different types but the backend will optimize them as the same underlying type after type erasure.I’m not so sure … I think the complications of adding a new Top type
to Scala are at least as great as changing the semantics of Any and
adding AnyObj to match Dotty.
In that case I’d much prefer to keep the original proposal and keep Any as the true top type!
It’d be interesting to make the change and try it with a few
representative projects and see how much breaks.
If you get around to do this, I’d be really interested in the results!
That’s a good way to put it.
I’m probably not making myself clear — simply stated, the JVM does not allow checkcast on primitives, so they have to be boxed once. Or do I miss something? That boxing is not due to newtypes but is still relevant.
Ah yes, of course. The question is only whether newtypes do any additional boxing.