Pre-SIP: deprecate asInstanceOf, introduce unsafeAsInstanceOf

[som-snytt]

Proposed illustrative implementation from the related thread:

def asInstanceOf[A]: A =

There has always been (since forever) a drumbeat about asInstanceOf. If education and documentation have failed, why isn’t it behind a SIP-18 import?

[Stephane]

There is little effective documentation for asInstanceOf.

1. Most people won’t look at the spec (which is wrong anyways).
2. My version of IntelliJ, which is up to date and working otherwise, does not provide any documentation for asInstanceOf, and I can imagine that many other Scala developers don’t have immediate access to it, and might not even be aware that it exists, given how hard it is to find online.
3. Googling ‘scala asinstanceof’ does not return any official documentation in the top results.

I can imagine that a dedicated documentation page, with several examples and pitfalls, could be quite helpful to many people.

[robstoll]

tried to improve the docs: make it clearer that asInstanceOf is unsafe and the outcome is undefined by robstoll · Pull Request #10696 · scala/scala · GitHub

[bishabosha]

I also checked and Metals does not provide a hover (in scala 2.13 and 3.3.1 where I tested) for asInstanceOf either

[som-snytt]

maybe call it unsafeAsKnives[A].

Really it’s unsafeWhenNotInstanceOf. When it’s instanceof, it’s perfectly fine.

[lihaoyi]

I agree that fleshing out the docs, fixing the scaladoc in IDEs, etc. is worthwhile. IntelliJ doesn’t properly jump to definition or show Scaladoc too IIRC (not sure if that changed in some recent version). We should fix that. We can add pages on scala-lang.org to help with SEO. I just don’t think changing the name is the right answer here.

Though I wouldn’t be opposed to calling it .__AS_INSTANCE_OF_DO_NOT_USE_OR_YOU_WILL_BE_FIRED[T]!

[nafg]

Well that’s the thing with safety. If someone crosses a highway safely that doesn’t mean that crossing highways is a safe thing to do.

[MateuszKowalewski]

I agree with the other comments that regarding naming the ship has likely sailed.

But if I had to name it today I’d call it unsafelyAssumeType[?]. Because the “as” in the current name points indeed a little bit in the direction of a regular “conversion” method. Usually asSomething Methods are perfectly safe; and actually perform some kind of conversion. But asInstanceOf does not! You just tell the compiler that “you know better”, and it should assume you’re right, no questions asked (and especially no conversion inserted).

[som-snytt]

assInstanceOf. Just like “assume” makes an “ass” of “u” and “me”, it makes an instance of an ass. (Usually we can’t tell unless a good friend and co-worker points it out to us.)

[alvae]

There’s been many interesting points made and I’d like to address some of those. So I’ll write multiple posts only so that people can more easily agree or disagree with each of them separately.

@bishabosha

in practice do people not read this? can we make it better

I can only speak for myself, but yes. I did read this documentation and FWIW I understood what it does. It helps that people coming with experience of a language that has UB at every corner won’t be shocked by the lack of guarantees.

IMO, as for many other elements of the documentation, the way to make it better is:

1. Make sure that one of the top 3 Google results when you type “Scala asInstanceOf” on a fresh browser without cookies leads to an official source of documentation.
2. Have a dedicated page for each method. On scala-lang.org you need to click on the method to see the additional notes. I would MUCH prefer a layout like this.
3. Better explain why one would use the method and when one should not use it.

Specifically on asInstanceOf, having everything written in a single paragraph is slightly discouraging. Personally it makes me feel like the method is harder to understand that it actually is. I would prefer if it had been displayed that way:

Note that the success of a cast at runtime is modulo Scala’s erasure semantics. For example:

• 1.asInstanceOf[String] throws a ClassCastException at run-time because the Int and String are distinct after erasure; but
• List(1).asInstanceOf[List[String]] does not because List[Int] and List[String] are not distinct after erasure.

From there, if you’re not a C++ developer who got used to believe everything not explicitly documented has UB, it would be good to read something along those lines:

Do not rely on thrown exception to determine whether or not a conversion via asInstanceOf was successful. Behavior is undefined In cases where the language isn’t able to catch an invalid conversion because of erasure (as shown above).

And finally I would add some notes about ways to work around the erasure thing.

[alvae]

@robstoll

IMO it’s part of language evolvement to improve/fix mistakes which were made

I can’t agree more. There are things that are nearly impossible to change and then there are the names of dangerous methods that require extra care in reviews and maintenance.

Further, changes of any kind are possible when there’s a smooth migration path. That’s obviously the case here. The compiler can emit a deprecation note that says "asInstanceOf is deprecated and will be soon replaced by “asUnsafeInstanceOf (or any color you like for the bike shed)”.

This kind of deprecation has precedent in other languages. I would not surprised if Scala veterans could name examples in Scala too.

@robstoll

Given that the asInstanceOf is very close to isInstanceOf (in terms of letters) and I guess most developers believe they know how isInstanceOf behaves they probably don’t look at the documentation of asInstanceOf assuming a similar behaviour.

That’s a very good argument.

If I can go off on a little tangent, I would add that I personally believe Scala lacks a lightweight way to safely narrow a type. Pattern matching certainly works, but even if I don’t go crazy with newlines that’s still 3 lines in braceless syntax.

val x = y match
  case z: T => Some(z)
  case _ => None

@MateuszKowalewski

I understand that asInstanceOf is meant as an “escape hatch”. But I still think it should warn on obviously failing casts.

I very much agree. compile-time safety is many times better than its run-time counterpart.

Note that I would also emit a warning in the following situation:

val x = Int
val p = x.isInstanceOf[String]
// conversion from Int to String always fails

IMO there’s no point writing predicates that will always fail. So if the compiler is able to make this determination, it should issue a warning.

@MateuszKowalewski

But if I had to name it today I’d call it unsafelyAssumeType[?]. Because the “as” in the current name points indeed a little bit in the direction of a regular “conversion” method.

That’s an excellent point!

[alvae]

@lihaoyi

Everyone already knows casts are unsafe. They are unsafe in Java. They have been unsafe since C introduced them many decades ago.

I disagree.

One issue to consider is that “unsafe” means different things to different people. My personal definition, which comes from C++, is that unsafe means “prone to UB”. In my experience, that’s typically not what unsafe means for JVM enthusiasts, which is one of the main selling points of using a virtual machine in the first place!

Hence, I would argue that asInstanceOf is more unsafe than most “unsafe” (as described by most Scala developers) operations. Crucially, it is unsafe in a way that makes bugs harder to debug than most other bugs one can write in Scala.

To perhaps better illustrate, consider this example in Swift:

let x: Any = 42
let y = x as! String
let z = unsafeBitCast(x, to: String.self)

If you ask a Swift developer, they will say that the cast initializing y is safe whereas the one initializing z isn’t. That’s because the first has defined behavior. It will invariably trap at run-time. The second will go through and then perhaps summon nasal demons, which are orders of magnitude harder to debug.

[lihaoyi]

The example you gave in Swift behaves the same in Scala too. Casting to a String traps on failure, bit-bashing using sun.misc.Unsafe does not.

There is definitely trickiness around whether things trap in the presence of erasure, but I’d argue that’s not a Scala-specific thing. I’ve certainly hit “cast succeeds, code blows up somewhere downstream” bugs in Java as well. And for the new gradual type systems on dynamic runtimes, such as Typescript or MyPy, cast-succeeds-code-blows-up-downstream is the norm, as all static types are erased on those platforms.

I don’t disagree that the behavior in the original post is confusing, and that Scala has its own trickiness due to “additional” erasure of type constructs that do not exist in Java. But fundamentally I feel like this behavior around casts is really standard. I’m not actually familiar with any platform where casts are guaranteed to fail loudly; maybe C# with its reified generics? I’m unfortunately not familiar enough with Swift to discuss it in detail

[Sporarum]

@alvae

@MateuszKowalewski

I understand that asInstanceOf is meant as an “escape hatch”. But I still think it should warn on obviously failing casts.

I very much agree. compile-time safety is many times better than its run-time counterpart.

Thinking more on this, I actually disagree in this case.
I would go even further, I think 1.asInstanceOf[String] should not throw an error at runtime !

The reason is, if we do some thing in some cases, it creates an expectation that it does it always, that the compiler has your back.
And in this case, it very much does not !

We can see this effect even in this thread:

@robstoll

there you go, another confusion, it inserts runtime checks

I am not sure even having a message like
“This conversion will always fail, did you mean … IMPORTANT: Note that this error message is only present for some misuse of asInstanceOf, DO NOT rely on it”
will actually diminish that effect

[Sporarum]

@alvae

If I can go off on a little tangent, I would add that I personally believe Scala lacks a lightweight way to safely narrow a type. Pattern matching certainly works, but even if I don’t go crazy with newlines that’s still 3 lines in braceless syntax.

val x = y match
  case z: T => Some(z)
  case _ => None

Maybe we should add something like this to the standard library (name TBD):

extension (x: Any)
  def castOption[T]: Option[T] = x match
    case y: T => Some(y)
    case _ => None

(And maybe a version that is TypeTest friendly)

I think Options make a lot of sense here, as they are basically a boolean + a return value

[alvae]

@lihaoyi

The example you gave in Swift behaves the same in Scala too. Casting to a String traps on failure

You can make the example demonstrating a case would not trap in Scala if you want.

let x: Any = [1]
let z = x as! [String] // will trap in Swift, won't throw in Scala

My larger point is that there is a difference between an “unsafe” operation that will consistently throw if its preconditions are violated (e.g., division by 0) and one that could cause UB (e.g., asInstanceOf).

@lihaoyi

I’m not actually familiar with any platform where casts are guaranteed to fail loudly

Swift will fail loudly for any conversion that cannot be guaranteed correct at runtime. That’s because as! (and as?, which is the feature returning the optional that I wish Scala had) are safe operations.

I don’t know if such safety guarantees can be offered in Scala but that’s another discussion. The point I’m making is that AFAICT most operations in Scala are safe (under my definition of safety, i.e., they can’t cause UB) but asInstanceOf isn’t. To me, that justifies adding “unsafe” somewhere in its name.

[robstoll]

created an issue for intellij:
https://youtrack.jetbrains.com/issue/SCL-22167/documentation-missing-for-methods-defined-in-Any

and one for metals:

not able to show the code documentation of asInstanceOf and other synthetic methods

[MateuszKowalewski]

@lihaoyi

[…] but I’d argue that’s not a Scala-specific thing. I’ve certainly hit “cast succeeds, code blows up somewhere downstream” bugs in Java as well […]

Who cares about what Java does?

Scala is not Java, and a Scala dev should not be required to know anything about Java!!! That’s a longstanding misunderstanding in the Scala community: Scala is not just some syntax sugar atop Java. Such a point of view is actively harmful to Scala as a language. (This issue is also something that shows in some other corners like the poor Scala documentation and tutorials which almost always assume that you’re an expert on all Java idiosyncrasies. This needs to stop finally!)

Java is a different language. It’s “unsafe” all the way down, it’s casting and NPE hell, and actually the best you could do in case you wanted to arrive at some sane design is to do the exact opposite of almost everything Java does.

Same goes for any other dynamic language. (Java is at its core a dynamic language, just rewatch the advertisements from the 90’s in case you missed that fact; being a dynamic language is core to the problem with Java!). So regarding Python or TS:

Some tacked on unsound pseudo type-system doesn’t change anything. These languages are unsafe and unsound at their core. Imho the result is even bigger trash as not having a static type system in the first place, as in such case you at least not lulled into believing that the “types” could safe your ass.

So both examples are not the yardstick for anything here.

@lihaoyi

But fundamentally I feel like this behavior around casts is really standard.

Where?

Maybe in the mindset of the 70’s of last century…

Repeating all the C nonsense over and over, even 60 years later, is just bananas imho.

The new “standard” are safe conversions with “as”. It’s the year 2024.

It’s actually a kind of joke that of all things I need to point to MS and something out of Apple, two companies that aren’t know for their high technical standards, more the contrary, but both do "the right thing"™ nowadays.

E.g. C#:

using System.Collections.Generic;					

public class Program
{
	public static void Main()
	{
		int[] ints = [10, 20, 30];
		IEnumerable<int> numbers = ints;
		
		var TheAnser = (int)"42";
		var CastedNumbers = (IEnumerable<string>)ints;
	}
}

This code will fail at compile time with:

Compilation error (line 10, col 18): Cannot convert type 'string' to 'int'
Compilation error (line 11, col 23): Cannot convert type 'int[]' to 'System.Collections.Generic.IEnumerable<string>'

And that aren’t even the “safe conversions”! These are casts…

@Sporarum

I would go even further, I think 1.asInstanceOf[String] should not throw an error at runtime !

So it shouldn’t compile? (Like in C#?)

Because not throwing an error here at runtime is not an option. This needs to fail.

The only alternative were nasal daemons… Nobody wants that.

That’s nothing about Scala. That’s runtime behavior of the VM. And actually the JVM does here the right thing (as the VM is the only mostly sane part of Java…). Just doing nothing and than exploding with completely random behavior at the other end of the world, like JS does, is imho much worse. That’s exactly one of the main reasons why JS is dreaded by so many people: If you don’t work with the highest caution things in JS just blow up at “random” places.

@alvae

Swift will fail loudly for any conversion that cannot be guaranteed correct at runtime. That’s because as! (and as?, which is the feature returning the optional that I wish Scala had) are safe operations.

The issue here is that asInstanceOf is the “unsafe” “hey compiler, look the other way” operator. It’s not as. It’s the “bitcast”.

And such an operation is needed.

Only that it shouldn’t be completely stupid imho. It should bail out on obviously bad cases. (See C#). But it still needs to work for the cases where the compiler can’t determine whether something is “right or wrong” (that’s where the C# thingy gets annoying because it doesn’t do that, it’s too strict, so you need to do nonsense like “double casting” going through object, if you really know better but the compiler can’t understand that). Otherwise we wouldn’t have an “escape hatch” for the cases where our type system would reject valid and useful programs. (Every static type system that guaranties that it never accepts invalid, unsound programs will experience cases where it rejects valid programs. That’s a hard rule, afaik there is a formal prove for it, so “escape hatches” are strictly needed. But they shouldn’t be maximally “blind”, and there should be some “safe variants” also, imo).

[Sporarum]

@MateuszKowalewski

@Sporarum

I would go even further, I think 1.asInstanceOf[String] should not throw an error at runtime !

So it shouldn’t compile? (Like in C#?)

Because not throwing an error here at runtime is not an option. This needs to fail.

The only alternative were nasal daemons… Nobody wants that.

I’m not sure you understood me:
(The code = 1.asInstanceOf[String])

1. The code currently compiles
2. I believe it should keep compiling (i.e. not emit an error)
3. I believe it might be dangerous for it to emit a warning
   1. As this might make it seem safer than it is
   2. “The compiler warned me last time, surely it will warn me again if I mess up”
   3. This might be the case even if the warning is explicit about not being reliable
4. I believe it might be better if it actually did not return a class cast exception, and instead randomly failed somewhere random
   1. I know this is probably impossible
   2. Users might get burned harder, but this would give the operator a very real sense of danger, and that there is no safeguard

And overall, I feel the most constructive given the current situation is to make the documentation around asInstanceOf better
Only then, if it is still a problem, should we think of renaming it

[MateuszKowalewski]

@Sporarum

I believe it might be dangerous for it to emit a warning

1. As this might make it seem safer than it is
2. “The compiler warned me last time, surely it will warn me again if I mess up”
3. This might be the case even if the warning is explicit about not being reliable

But that would be the same kind of warning you get with pattern matching (or ==).

It does not tell you that something is safe, but it tells you in case it would fail for sure.

Or would you also remove the other warnings and errors? Should "23" == 23 compile?

@Sporarum

I believe it might be better if it actually did not return a class cast exception, and instead randomly failed somewhere random

1. I know this is probably impossible
2. Users might get burned harder, but this would give the operator a very real sense of danger, and that there is no safeguard

Burn down your house to teach the children not to play with matches?

I’m not sure this is a good idea…

Thanks God at least on the JVM you will get a helpful, fast failing exception when it’s messed up obviously. That’s build in into the VM so Scala can’t change that.

@Sporarum

And overall, I feel the most constructive given the current situation is to make the documentation around asInstanceOf better
Only then, if it is still a problem, should we think of renaming it

I fully agree.

It’s not like asInstanceOf would be a big problem in Scala. You usually don’t use it. (So you usually also don’t see it much in other people’s code). So it’s not the biggest issue. There are more important things. Better docu is always good, but else? I would like to have warnings for obviously bad cases, if it’s not to difficult. But I don’t think this has high prio on the wish list.