Is it normal that lambda not having any class symbol?

Scala Standard Library
1

In the latest reflection API in scala-2.12, I realise that the standard code to detect a (probably erased) type works strangely in some cases.

Namely, detection works for anonymous class:

{
    val ll = new AnyRef with Function1[String, Int] with Serializable {

      override def apply(v: String) = v.toInt
    }

    val clazz = ll.getClass
    val mirror = runtimeMirror(clazz.getClassLoader)

    val sym = mirror.classSymbol(clazz)

    print(sym.selfType)
}

String => Object with Serializable{def apply$mcZD$sp(v1: Double): Boolean; def apply$mcDD$sp(v1: Double): Double; def apply$mcFD$sp(v1: Double): Float; def apply$mcID$sp(v1: Double): Int; def apply$mcJD$sp(v1: Double): Long; def apply$mcVD$sp(v1: Double): Unit; def apply$mcZF$sp(v1: Float): Boolean; def apply$mcDF$sp(v1: Float): Double; def apply$mcFF$sp(v1: Float): Float; def apply$mcIF$sp(v1: Float): Int; def apply$mcJF$sp(v1: Float): Long; def apply$mcVF$sp(v1: Float): Unit; def apply$mcZI$sp(v1: Int): Boolean; def apply$mcDI$sp(v1: Int): Double; def apply$mcFI$sp(v1: Int): Float; def apply$mcII$sp(v1: Int): Int; def apply$mcJI$sp(v1: Int): Long; def apply$mcVI$sp(v1: Int): Unit; def apply$mcZJ$sp(v1: Long): Boolean; def apply$mcDJ$sp(v1: Long): Double; def apply$mcFJ$sp(v1: Long): Float; def apply$mcIJ$sp(v1: Long): Int; def apply$mcJJ$sp(v1: Long): Long; def apply$mcVJ$sp(v1: Long): Unit; def apply(v1: Any): Object}

(I know this output looks super weird but at least it is an erased type)

ā€¦ but doesnā€™t work for anonymous function / lambda:

    val ll = { v: String =>
      v.toInt
    }

    val clazz = ll.getClass
    val mirror = runtimeMirror(clazz.getClassLoader)

    val sym = mirror.classSymbol(clazz)

    print(sym.selfType)

assertion failed: no symbol could be loaded from class org.apache.spark.ml.dsl.utils.refl.ScalaTypeSpike$$Lambda$356/2114537280 in package refl with name ScalaTypeSpike$$Lambda$356/2114537280 and classloader sun.misc.Launcher$AppClassLoader@18b4aac2
java.lang.AssertionError: assertion failed: no symbol could be loaded from class .refl.ScalaTypeSpike$$Lambda$356/2114537280 in package refl with name ScalaTypeSpike$$Lambda$356/2114537280 and classloader sun.misc.Launcher$AppClassLoader@18b4aac2
at scala.reflect.internal.SymbolTable.throwAssertionError(SymbolTable.scala:184)
at scala.reflect.runtime.JavaMirrors$JavaMirror.classToScala1(JavaMirrors.scala:1061)
at scala.reflect.runtime.JavaMirrors$JavaMirror.$anonfun$classToScala$1(JavaMirrors.scala:1019)
at scala.reflect.runtime.JavaMirrors$JavaMirror.$anonfun$toScala$1(JavaMirrors.scala:130)
at scala.reflect.runtime.TwoWayCaches$TwoWayCache.$anonfun$toScala$1(TwoWayCaches.scala:50)
at scala.reflect.runtime.TwoWayCaches$TwoWayCache.toScala(TwoWayCaches.scala:46)
at scala.reflect.runtime.JavaMirrors$JavaMirror.toScala(JavaMirrors.scala:128)
at scala.reflect.runtime.JavaMirrors$JavaMirror.classToScala(JavaMirrors.scala:1019)
at scala.reflect.runtime.JavaMirrors$JavaMirror.classSymbol(JavaMirrors.scala:231)
at scala.reflect.runtime.JavaMirrors$JavaMirror.classSymbol(JavaMirrors.scala:68)

Is it going to be a deliberate design for Scala 2.12+ and Dotty?

1 Like
2

Note that there is some discussion already at https://github.com/scala/bug/issues/11828

1 Like
3

Because of the issues with getClass.getSimpleName, this is my preferred workaround for getting a reasonable name for somethingā€™s class or type, so Iā€™d really like this to return something reasonable.

Note: yes the issue for this is closed, but itā€™s closed as ā€œnot our problem, upgrade to JDK9ā€, which isnā€™t really feasible (at least based on the posted guidelines, the most recent fully-supported JDK is 8)

4

It is NOT relevant to #2034, the error being thrown here is AssertionError, not ClassNotFoundException.

For #2034 though: why abandoning a JDK that is in active support by oracle, and is the latest version that is being actively supported by the scala team?

5

To clarify: itā€™s related because it demonstrates that none of the available methods are universal - there isnā€™t a single way to get a printable representation of the type that will work in all circumstances.

getSimpleName dies if itā€™s doubly nested, and selfType fails for lambdas.

1 Like
6

Good to see that it is not normal, is there a roadmap published to address problems like this?

7

Isnā€™t Java 8 no longer supported by Oracle?

Moreover, #2034 is really a Java bug, as linked on the ticket: Loading.... We donā€™t follow the Java language spec, because weā€™re not Java; we do follow the JVM spec here, but the implementation of getSimpleName doesnā€™t. Note that the original JDK bug report mentions groovy, not scala: this isnā€™t because Scala itself is doing anything wrong.

Paul fixed it a while ago, but his patch is probably lost to time, and now that it is fixed in the JDK I think itā€™s perfectly reasonable to say ā€œuse a non-buggy version of the software with the bugā€.

1 Like
8

This would be true, if any of the non-buggy versions of the JDK were fully supported by Scala. As it is, the point isnā€™t that Scala is doing anything wrong per se, itā€™s that the alternative to selfType is also currently broken in some cases.

As they didnā€™t backport the fix to JDK 8, fixing selfType is pretty much the only way forward we have any control over.

9

Well, itā€™s not even selfType here but classSymbol, and the problem is simply that the class org.apache.spark.ml.dsl.utils.refl.ScalaTypeSpike$$Lambda$356/2114537280 doesnā€™t exist until the JVM makes it at runtime; its name isnā€™t even stable! So from Scalaā€™s perspective itā€™s not easy to talk about that class as a ā€œclassā€. Seth did a better job of explaining this on the ticket.

Also, youā€™re mixing up the two problems here: @tribbloidā€™s complaint isnā€™t fixed in Java 9.

We could also get to a point when we can say Java 9 is fully supported. I believe the only missing bit is compatibility with JPMS, which has a prototype implementation already.

1 Like
10

@hrhino is right: we were mostly talking about an error case where a different exception (MatchError which generally indicates an incomplete pattern matching warning at compile time) is thrown at a different time.

Iā€™m wondering why the runtime class needs to be stable? Both java & scala supports dynamic classloading, metaprogramming capability is not supported directly but is mentioned in several libraries.

  • So from Scalaā€™s perspective itā€™s not easy to talk about that class as a ā€œclassā€

Yes but functions should have classes, otherwise how would you define ā€˜high-level functionā€™?

11

Eh? Re: 11828, I donā€™t think there is any agreement here that there is even any actual problem, let alone a plan or roadmap for a ā€œsolutionā€. (The getSimpleName stuff, 2034, is something else.)

1 Like
12

Iā€™ve had a good poke at the OpenJDK internals, and Iā€™m pretty sure thereā€™s no way to go from a lambda class to its implementing method. So, even if we wanted to allow classSymbol(((x: Int) => x).getClass), I donā€™t think thereā€™s a way we can do that without moving away from Java 8ā€™s LambdaMetaFactory.

@tribbloid: can you describe your actual goal here? Itā€™s possible that your use case can be supported somehow else (but I make no promises).

1 Like
13

In my previous discussion with Seth (https://github.com/scala/bug/issues/11828), Iā€™m already quite clear about my goal:

The lambda type can be easily inferred in compile-time (using implicit), since it is not a generic type, it should not be affected by type erasure, and inferring it in runtime should be equally easy or only slight difficult using reflection.

ā€œto go from a lambda class to its implementing methodā€: I havenā€™t tried it yet but I was under the impression that Java reflection can do that without much surprised, let me post the result later ā€¦

14

What you say is quite clear isnā€™t quite specific. Specifically, it uses terms in a context that arenā€™t defined cleanly.

Do you have a concrete proposal at this time, or are you still gathering input for shaping your proposal? If itā€™s the latter, some practical use cases would be good.

1 Like