Is it normal that lambda not having any class symbol?

In the latest reflection API in scala-2.12, I realise that the standard code to detect a (probably erased) type works strangely in some cases.

Namely, detection works for anonymous class:

{
    val ll = new AnyRef with Function1[String, Int] with Serializable {

      override def apply(v: String) = v.toInt
    }

    val clazz = ll.getClass
    val mirror = runtimeMirror(clazz.getClassLoader)

    val sym = mirror.classSymbol(clazz)

    print(sym.selfType)
}

String => Object with Serializable{def apply$mcZD$sp(v1: Double): Boolean; def apply$mcDD$sp(v1: Double): Double; def apply$mcFD$sp(v1: Double): Float; def apply$mcID$sp(v1: Double): Int; def apply$mcJD$sp(v1: Double): Long; def apply$mcVD$sp(v1: Double): Unit; def apply$mcZF$sp(v1: Float): Boolean; def apply$mcDF$sp(v1: Float): Double; def apply$mcFF$sp(v1: Float): Float; def apply$mcIF$sp(v1: Float): Int; def apply$mcJF$sp(v1: Float): Long; def apply$mcVF$sp(v1: Float): Unit; def apply$mcZI$sp(v1: Int): Boolean; def apply$mcDI$sp(v1: Int): Double; def apply$mcFI$sp(v1: Int): Float; def apply$mcII$sp(v1: Int): Int; def apply$mcJI$sp(v1: Int): Long; def apply$mcVI$sp(v1: Int): Unit; def apply$mcZJ$sp(v1: Long): Boolean; def apply$mcDJ$sp(v1: Long): Double; def apply$mcFJ$sp(v1: Long): Float; def apply$mcIJ$sp(v1: Long): Int; def apply$mcJJ$sp(v1: Long): Long; def apply$mcVJ$sp(v1: Long): Unit; def apply(v1: Any): Object}

(I know this output looks super weird but at least it is an erased type)

… but doesn’t work for anonymous function / lambda:

    val ll = { v: String =>
      v.toInt
    }

    val clazz = ll.getClass
    val mirror = runtimeMirror(clazz.getClassLoader)

    val sym = mirror.classSymbol(clazz)

    print(sym.selfType)

assertion failed: no symbol could be loaded from class org.apache.spark.ml.dsl.utils.refl.ScalaTypeSpike$$Lambda$356/2114537280 in package refl with name ScalaTypeSpike$$Lambda$356/2114537280 and classloader sun.misc.Launcher$AppClassLoader@18b4aac2
java.lang.AssertionError: assertion failed: no symbol could be loaded from class .refl.ScalaTypeSpike$$Lambda$356/2114537280 in package refl with name ScalaTypeSpike$$Lambda$356/2114537280 and classloader sun.misc.Launcher$AppClassLoader@18b4aac2
at scala.reflect.internal.SymbolTable.throwAssertionError(SymbolTable.scala:184)
at scala.reflect.runtime.JavaMirrors$JavaMirror.classToScala1(JavaMirrors.scala:1061)
at scala.reflect.runtime.JavaMirrors$JavaMirror.$anonfun$classToScala$1(JavaMirrors.scala:1019)
at scala.reflect.runtime.JavaMirrors$JavaMirror.$anonfun$toScala$1(JavaMirrors.scala:130)
at scala.reflect.runtime.TwoWayCaches$TwoWayCache.$anonfun$toScala$1(TwoWayCaches.scala:50)
at scala.reflect.runtime.TwoWayCaches$TwoWayCache.toScala(TwoWayCaches.scala:46)
at scala.reflect.runtime.JavaMirrors$JavaMirror.toScala(JavaMirrors.scala:128)
at scala.reflect.runtime.JavaMirrors$JavaMirror.classToScala(JavaMirrors.scala:1019)
at scala.reflect.runtime.JavaMirrors$JavaMirror.classSymbol(JavaMirrors.scala:231)
at scala.reflect.runtime.JavaMirrors$JavaMirror.classSymbol(JavaMirrors.scala:68)

Is it going to be a deliberate design for Scala 2.12+ and Dotty?

1 Like

Note that there is some discussion already at https://github.com/scala/bug/issues/11828

1 Like

Because of the issues with getClass.getSimpleName, this is my preferred workaround for getting a reasonable name for something’s class or type, so I’d really like this to return something reasonable.

Note: yes the issue for this is closed, but it’s closed as “not our problem, upgrade to JDK9”, which isn’t really feasible (at least based on the posted guidelines, the most recent fully-supported JDK is 8)

It is NOT relevant to #2034, the error being thrown here is AssertionError, not ClassNotFoundException.

For #2034 though: why abandoning a JDK that is in active support by oracle, and is the latest version that is being actively supported by the scala team?

To clarify: it’s related because it demonstrates that none of the available methods are universal - there isn’t a single way to get a printable representation of the type that will work in all circumstances.

getSimpleName dies if it’s doubly nested, and selfType fails for lambdas.

1 Like

Good to see that it is not normal, is there a roadmap published to address problems like this?

Isn’t Java 8 no longer supported by Oracle?

Moreover, #2034 is really a Java bug, as linked on the ticket: https://bugs.openjdk.java.net/browse/JDK-8057919. We don’t follow the Java language spec, because we’re not Java; we do follow the JVM spec here, but the implementation of getSimpleName doesn’t. Note that the original JDK bug report mentions groovy, not scala: this isn’t because Scala itself is doing anything wrong.

Paul fixed it a while ago, but his patch is probably lost to time, and now that it is fixed in the JDK I think it’s perfectly reasonable to say “use a non-buggy version of the software with the bug”.

1 Like

This would be true, if any of the non-buggy versions of the JDK were fully supported by Scala. As it is, the point isn’t that Scala is doing anything wrong per se, it’s that the alternative to selfType is also currently broken in some cases.

As they didn’t backport the fix to JDK 8, fixing selfType is pretty much the only way forward we have any control over.

Well, it’s not even selfType here but classSymbol, and the problem is simply that the class org.apache.spark.ml.dsl.utils.refl.ScalaTypeSpike$$Lambda$356/2114537280 doesn’t exist until the JVM makes it at runtime; its name isn’t even stable! So from Scala’s perspective it’s not easy to talk about that class as a “class”. Seth did a better job of explaining this on the ticket.

Also, you’re mixing up the two problems here: @tribbloid’s complaint isn’t fixed in Java 9.

We could also get to a point when we can say Java 9 is fully supported. I believe the only missing bit is compatibility with JPMS, which has a prototype implementation already.

1 Like

@hrhino is right: we were mostly talking about an error case where a different exception (MatchError which generally indicates an incomplete pattern matching warning at compile time) is thrown at a different time.

I’m wondering why the runtime class needs to be stable? Both java & scala supports dynamic classloading, metaprogramming capability is not supported directly but is mentioned in several libraries.

  • So from Scala’s perspective it’s not easy to talk about that class as a “class”

Yes but functions should have classes, otherwise how would you define ‘high-level function’?

Eh? Re: 11828, I don’t think there is any agreement here that there is even any actual problem, let alone a plan or roadmap for a “solution”. (The getSimpleName stuff, 2034, is something else.)

1 Like

I’ve had a good poke at the OpenJDK internals, and I’m pretty sure there’s no way to go from a lambda class to its implementing method. So, even if we wanted to allow classSymbol(((x: Int) => x).getClass), I don’t think there’s a way we can do that without moving away from Java 8’s LambdaMetaFactory.

@tribbloid: can you describe your actual goal here? It’s possible that your use case can be supported somehow else (but I make no promises).

1 Like

In my previous discussion with Seth (https://github.com/scala/bug/issues/11828), I’m already quite clear about my goal:

The lambda type can be easily inferred in compile-time (using implicit), since it is not a generic type, it should not be affected by type erasure, and inferring it in runtime should be equally easy or only slight difficult using reflection.

“to go from a lambda class to its implementing method”: I haven’t tried it yet but I was under the impression that Java reflection can do that without much surprised, let me post the result later …

What you say is quite clear isn’t quite specific. Specifically, it uses terms in a context that aren’t defined cleanly.

Do you have a concrete proposal at this time, or are you still gathering input for shaping your proposal? If it’s the latter, some practical use cases would be good.

1 Like