[JVM] Use nest-based access control for accessing private members in companion types

Hi all, this is an issue that has been on my mind for a while. I was playing with javap for a bit and noticed that the Scala compiler outputs public bridging methods for private fields such that they can be accessed by the companion object of a class.
Ever since JEP 181 this should be possible without making a public bridge when both the class and the companion object are part of the same nest.
The benefits of this approach would be twofold:

  • Runtime visibility is the same as the compile time visibility of class members, meaning java programs cannot access the public bridge, preserving integrity.
  • Allows for more accurate escape analysis by the JIT.

I am currently not aware of any downsides of to approach, but I may be ignorant here.

To prove this approach is possible, I made a simple PoC:

Hello.scala:

class Hello {
    private val world: String = "world"
}

object Hello  {
    def main(args: Array[String]): Unit = {
        val h = new Hello();
        System.out.println(h.world)
    }
}

We then compile the classes using scala compile --scala-version 3.5.0 --compilation-output . Hello.scala and watch Hello.class and Hello$.class being created.

Next, I modified the bytecode of the generated classes such that Hello.class will be the host of a nest containing both classes. I also made the bridge method for world private.

  • For Hello.class: the NestMember attribute was added, containing only the Hello$ name. Nothing else is necessary here since according to the JVM specification a nest host is already implicitly a member of its own nest.
  • For Hello.class: ā€˜publicā€™ accessibility modifier of the Hello$$world method is removed, and ā€˜privateā€™ accessibility modifier is added.
  • For Hello$.class: the NestHost attribute was added containing the Hello name.

Here is some code which accomplishes this transformation using the ASM library:

Nestify.scala:

//> using dep org.ow2.asm:asm:9.7

import java.nio.file.Files
import java.nio.file.Paths

import org.objectweb.asm.ClassVisitor
import org.objectweb.asm.MethodVisitor
import org.objectweb.asm.ClassReader
import org.objectweb.asm.ClassWriter
import org.objectweb.asm.Opcodes

object Nestify {

    def main(args: Array[String]): Unit = {

        val helloBytes: Array[Byte] = Files.readAllBytes(Paths.get("Hello.class"))
        val hello$Bytes: Array[Byte] = Files.readAllBytes(Paths.get("Hello$.class"))

        val helloReader = new ClassReader(helloBytes)
        val hello$Reader = new ClassReader(hello$Bytes)

        val helloWriter = new ClassWriter(0)
        val hello$Writer = new ClassWriter(0)

        helloReader.accept(new Nestifier(helloWriter), ClassReader.EXPAND_FRAMES)
        hello$Reader.accept(new Nestifier(hello$Writer), ClassReader.EXPAND_FRAMES)

        val newHelloBytes = helloWriter.toByteArray()
        val newHello$Bytes = hello$Writer.toByteArray()

        Files.write(Paths.get("Hello.class"), newHelloBytes)
        Files.write(Paths.get("Hello$.class"), newHello$Bytes)
    }
}

class Nestifier(delegate: ClassVisitor) extends ClassVisitor(Opcodes.ASM9, delegate) {

    override def visit(version: Int, access: Int, name: String, signature: String, superName: String, interfaces: Array[String]): Unit = {
        name match {
            case "Hello" =>
                 visitNestMember("Hello$")
            case "Hello$" =>
                 visitNestHost("Hello")
        }

        super.visit(version, access, name, signature, superName, interfaces)
    }

    override def visitMethod(access: Int, name: String, descriptor: String, signature: String, exceptions: Array[String]): MethodVisitor = {
        var newAccess = access

        if ("Hello$$world".equals(name)) {
            newAccess = newAccess & (~Opcodes.ACC_PUBLIC)
            newAccess = newAccess | Opcodes.ACC_PRIVATE
        }

        super.visitMethod(newAccess, name, descriptor, signature, exceptions)
    }

}

We execute this program and end up with the following class files:
javap -p -c Hello.class:

Compiled from "Hello.scala"
public class Hello {
  private final java.lang.String world;

  public static void main(java.lang.String[]);
    Code:
       0: getstatic     #15                 // Field Hello$.MODULE$:LHello$;
       3: aload_0
       4: invokevirtual #17                 // Method Hello$.main:([Ljava/lang/String;)V
       7: return

  public Hello();
    Code:
       0: aload_0
       1: invokespecial #21                 // Method java/lang/Object."<init>":()V
       4: aload_0
       5: ldc           #22                 // String world
       7: putfield      #24                 // Field world:Ljava/lang/String;
      10: return

  private java.lang.String Hello$$world();
    Code:
       0: aload_0
       1: getfield      #24                 // Field world:Ljava/lang/String;
       4: areturn
}

javap -p -c Hello$.class:

Compiled from "Hello.scala"
public final class Hello$ implements java.io.Serializable {
  public static final Hello$ MODULE$;

  private Hello$();
    Code:
       0: aload_0
       1: invokespecial #15                 // Method java/lang/Object."<init>":()V
       4: return

  public static {};
    Code:
       0: new           #4                  // class Hello$
       3: dup
       4: invokespecial #18                 // Method "<init>":()V
       7: putstatic     #20                 // Field MODULE$:LHello$;
      10: return

  private java.lang.Object writeReplace();
    Code:
       0: new           #24                 // class scala/runtime/ModuleSerializationProxy
       3: dup
       4: ldc           #4                  // class Hello$
       6: invokespecial #27                 // Method scala/runtime/ModuleSerializationProxy."<init>":(Ljava/lang/Class;)V
       9: areturn

  public void main(java.lang.String[]);
    Code:
       0: new           #2                  // class Hello
       3: dup
       4: invokespecial #31                 // Method Hello."<init>":()V
       7: astore_2
       8: getstatic     #37                 // Field java/lang/System.out:Ljava/io/PrintStream;
      11: aload_2
      12: invokevirtual #41                 // Method Hello.Hello$$world:()Ljava/lang/String;
      15: invokevirtual #47                 // Method java/io/PrintStream.println:(Ljava/lang/String;)V
      18: return
}

When we execute the Hello program using java Hello the program still runs and prints world as expected.

One possible improvement would be not even generate the bridge method, and just access the ā€˜worldā€™ field directly using a getField bytecode instruction.

6 Likes

I believe it might be considered when we finally.drop the JDK 8 support in the compiler. I think it should be possible when considering the second LTS series, so after at least 1 year from now. However, PoC could be implemented in the compiler right now and used only when targeting at least JDK 11.

4 Likes

I made a start but didnā€™t get back to it at https://github.com/scala/scala/pull/10765.

2 Likes

The Scala 2 PR is ready for review and the Scala 3 port has a draft PR.

4 Likes

Wow awesome! You accomplished that way quicker than I could have done it myself.

One question about the Scala 3 PR: should the ExpandPrivate phase also be updated?
From reading the code it looks like this phase is responsible for allowing access to private members from enclosing classes and nested classes, just like the old Java 8 compilation strategy. We should be able to update this strategy too, but maybe this fits better in a separate PR. Iā€™d be willing to give this a shot myself.

If you click on the ā€œfilesā€ tab of the PR, github shows a nice view of changed files, where you can see ExpandPrivate.scala and click on it. The interesting consequence is that the backend must choose how to invoke the private member.

The test shows the sorts of use cases I was aware of.

My TIL was that nested classes in a companion module have private access to the companion class. So there are plenty of opportunities to learn in a contribution of limited scope, and as usual I apologize for taking that from anyone. Itā€™s a case of the early bird, or finders keepers, or last one in is a rotten egg. But there are other avenues, such as the periodic bug spree, or GSOC just concluded. I havenā€™t tried the spree because I assume my computer isnā€™t set up for it.

Possibly, no one will have time to review this low-priority PR, so that in a year you will have a chance to re-open it.

Also, we forgot to welcome you to the community! Thanks for posting.

2 Likes

Thanks for the welcome. Regarding the current PRs, I do agree itā€™s better if you finish them indeed. And you were the first finder, so no need to apologise.