I have now discovered a different solution which keeps the most important parametricity guarantees without running into a lot of complexity and migration headaches. It is described in this PR. See also the associated issue that contains a more complete discussion (if you are interested, you should read the whole comments thread since there was quite a bit of development between the starting thoughts and where we ended up). In short, we need to add a trait Matchable that is inherited by AnyVal and AnyRef and that encapsulates the behavior of getClass, isInstanceOf and pattern matching.
The associated warnings that ensure parametricity can be rolled out over the next minor releases of Scala 3. But here’s the catch: This works only if we lay the groundwork now by adding the Matchable trait. If we do not do that we will likely have to wait several years for the next major version of the language. The problem is that we are now very close to finalizing Scala 3. So for this to go forward we’d need quick and thorough feedback from the community whether this is perceived as an important step forward or whether you think it is irrelevant or a dead end.
Discussions should take place here instead of the PR.
The dotty community build is quite small compared to Scala 2.
I think this change should be first applied to the Scala 2.13 branch (just for experimentation) and run a community build on it, before allowing Scala 3.0 to adopt it.
The current PR enables the warning for -source 3.1 or source 3.1-migration and later.
This makes cross-compilation between 2.13 and 3.1 difficult, I guess one can always use @unchecked instead, but then it’s worth asking whether we really need this new trait or we could instead warn by default on downcasting (or at least downcasting from Any).
The advantage of Matchable is that it prevents us from passing an opaque type, but this only works when the opaque type is defined as opaque type Foo = X, it doesn’t work if it’s defined as opaque type Foo <: SomeTrait = X, which will be upper-bounded by Matchable meaning that downcasting type tests would still be unsound.
The PR doesn’t mention universal traits: so far we could write trait Foo extends Any, can we now write trait Foo extends Matchable too?
I think this will always be true of restricting the languages top type. In the short view, the same thing could be achieved by warnings, a weaker top type on the other hand is a long term solution.
A Matchable trait also seems less like a hack than @unchecked warnings to me.
I wrote about this before here:
Disallowing (via compiler warnings or errors) downcasting to an opaque type makes a lot of sense regardless what it’s upper bound is. This is already the case as of PR #10664.
The issue is the other direction: Someone using something Boxish as a Box or a Foo as an X.
I don’t think there’s a good solution to this, thought I’d be happy if someone proved me wrong. It would be a problem regardless, and it’s good enough that you’re “safe” as long as you don’t have a subtype of Matchable as the upper bound on the opaque type.
I think you’re referring to my comment, which was a longer reasoning about why it might eventually be desirable to decouple Java’s Object from AnyRef some time in the future. This is not close to being considered. If anyone else is confused I’ll delete the comment.
As I mentioned in the issue, it’d be useful if classes stopped extending Matchable by default. In any case, I think there really should be a way of opting out of Matchable when defining concrete types.
This would notably allow safely using such non-Matchable types as upper bounds of opaque types. It would also allow signaling that a type is not supposed to have meaningful equals/hashCode and should be treated parametrically, which is great if in the future we want to replace it with an opaque type or something with a different runtime representation.
The latest development is that Matchable is just a marker trait allowing pattern matching.
It would still be really nice, if we could remove many method from Any, especially ==, !=, ## and maybe even toString (equals and hashCode are not as problematic, since they are explicit “Java-isms” and so stick out more).
But I have no idea how to achieve that, let alone safely
As mentioned in the associated issue, this is much harder to do than what one might think at first glance. Removing equals or toString from List for example, would be a breaking change, and those are defined in terms of calls to those methods on Any. A possible migration strategy would be to replace these methods with extension methods, which is what I suggest in this comment: https://github.com/lampepfl/dotty/issues/10662#issuecomment-740316060
I don’t mean to push my luck here. I think having Matchable is a significant improvement beyond what I hoped to see in 3.0. As @LPTK says however, having classes and traits without them would be useful in many cases from a principle of least privilege perspective.
A feasibly solution would be to have AnyVal, AnyRef and Matchable all extend Any directly, and then have classes and traits extend AnyRef & Matchable by default.
This would be compatible with the vast majority of existing code, while at the same time making it possible to explicitly extend AnyRef without extending Matchable.
@LPTK has raised the question whether Matchable is the best name, especially going forward when the trait will be about much more than just pattern matching.
Not commenting on the design in general but I would find it very weird to have such an extremely specific type (Matchable, Inspectable, …) at the top of the type hierarchy which is otherwise populated with extremely general types such as Any, AnyRef, AnyVal. So I would find AnyClass—originaly proposed by @arturopala—much more fitting.
In a sense, your findings match the reality. It is indeed very strange, that already at the (almost) very top, you find something you can inspect, match on, compute hash code of and so on. But that’s just how the authors created Java and JVM, that’s just the reality we have to live with.
Do you see what I mean?
AnyClass wouldn’t be a good name, because people will confuse it with AnyRef (aka Object) and it’s supposed to be not only for AnyRefs, but also for AnyVals – AnyVals aren’t classes.
But in Scala they behave like they “conceptually” are—and in many instances they actually are by the way, boxing/unboxing is an implementation detail. And being able to pattern match (especially the kind of pattern matching that this feature wants to prevent) basically means checking whether some value is an instance of some class.
@LPTK, @sideeffffect if any of the other methods are factored out of Any in the future, I really don’t see why they must be in the same trait as getClass and isInstanceOf. There could be uses for a class which has isInstanceOf, but not equals or vice versa.
But even if the other methods were moved into the same trait, the name actually isn’t that weird:
x equals y can be thought of: “does x match y?”
hashCode is heavily related to equals
toString gives you a string representation that “matches” the value
I don’t think it would’ve been unreasonable name wise. But it makes much more sense to me to make Equals and Showable their own traits.
AnyClass is weird to me for multiple reasons:
traits aren’t classes
singleton objects aren’t classes
values in general aren’t classes, they are class instances = objects
