Scala Compiler Plugin naming issues after typer


#1

Hello,

I’ve been working on a compile plugin whose components run after typer and I’ve run into a handful of different issues that seem to relate to naming. I have a couple examples here which produce different but seemingly related errors. Can someone help with looking into these errors? Thanks.

My plugin component

to which I’ve added to Transformer.transform in each of the cases

private class Component extends PluginComponent with TypingTransformers with Transform {
    val global: TestPlugin.this.global.type = TestPlugin.this.global
    val runsAfter = List[String]("typer")
    val phaseName = "myPlugin"

    protected def newTransformer(unit: CompilationUnit): Transformer = new Transformer(unit)

    class Transformer(unit: global.CompilationUnit) extends TypingTransformer(unit) {
      override def transform(t: global.Tree): global.Tree = {
        t match {
          // Add more cases here...
          case _ => super.transform(t)
        }
      }
    }
  }

Case 1: Renaming/creating new function and calling it

Expectation: I expect my program to print out hello after this transformation.
Reality: Compilation error
Note: If I print out the tree after the transformation the tree looks as I would expect, with hello replacing goodbye properly.

Source File:

object Test extends App {
    def goodbye(): Unit = { //Transform to def hello()
        println("goodbye")
    }
    goodbye() // Transform to hello()
}

Added match-case in transform: (this fails even if the funcCall case is in a different plugin component that runs after)

case def @ DefDef(_, TermName("hello"), _, _, _, _) => 
    val tree =
      q"""
      def hello(): Unit= {
        println("hello")
      }
    """.asInstanceOf[DefDef]
    val DefDef(modifiers, name, tparams, vparams, tpt, rhs) = tree
    val source = treeCopy.DefDef(tree, modifiers, name, tparams, vparams, tpt, rhs)
    localTyper.namer.enterDefDef(source)
    localTyper typed source
...
// transform goodbye() to hello()
case funcCall @ Apply(Select(This(TypeName("Test")), TermName("goodbye")), List()) =>
    val source = Apply(Select(This(TypeName("Test")), TermName("hello")), List())
    localTyper typed source

Error when compiling:

[error] scala.reflect.internal.Types$TypeError: value goodbye is not a member of object test.Test

Case 2: Overwriting an object /copying an object (post-typer)

Expectation: Should compile fine with nothing effectively changed since I’m replacing object Test with a copy of itself.
Reality: Compilation Error
Note: If I print out the tree after the transformation, the tree looks as I would expect, exactly like the source file should look at this point in the compiler.

Source File:

object Test extends App {
    val x = 0 // works without this line.
}

Added match-case in transform: (this fails even if the funcCall case is in a different plugin component that runs after)

case obj@ModuleDef(_, termName("Test"), _) => 
    val q"$mods object $tname extends { ..$earlydefns } with ..$parents { $self => ..$body }" = obj
    //Effectively just copy the tree
    val source = q"$mods object $tname extends { ..$earlydefns } with ..$parents { $self => ..$body }".asInstanceOf[ModuleDef]
    localTyper.namer.enterModuleDef(source)
    localTyper typed source

Error when compiling:

scala.reflect.internal.Types$TypeError: value x in object Test cannot be accessed in object Test
exception when typing Test.this.x/class scala.reflect.internal.Trees$Select

#2

Have you considered implementing a macro annotation instead of compiler plugin? Synthesizing members like def hello() is quite tricky, see this comment for a more detailed explanation.

I’m not sure if it’s helpful but you might get inspiration from the sources of scalamacros/paradise if macro annotations don’t support your use-case https://github.com/scalamacros/paradise/tree/2.12.8/plugin/src/main/scala/org/scalamacros/paradise/typechecker


#3

Found a solution:

case obj@ModuleDef(_, termName("Test"), _) => 
    val q"$mods object $tname extends { ..$earlydefns } with ..$parents { $self => ..$body }" = obj
    //Effectively just copy the tree
    val source = q"$mods object $tname extends { ..$earlydefns } with ..$parents { $self => ..$body }".asInstanceOf[ModuleDef]
    obj.symbol.owner.info.decls.unlink(obj.symbol)
    localTyper.namer.enterModuleDef(source)
    source.symbol.owner.info.decls.enter(source.symbol)
    localTyper typed source

You must unlink the old symbol and enter the new symbol from the owner (in this case it was the package that the module was in).


#4

Hi Irene,

The answer is a bit more complicated than that, and you might still run into errors even if you unlink and enter a new symbol. The problem is that the old symbol was already used by other parts of the code, where the object (method, etc) was referenced. You can’t go back and replace those, and at some point there will be errors like value x in object Test cannot be accessed in object Test. The two Test are different symbols with the same name.

Pro tip: compile with -uniqid. This will add a number at the end of each identifier, so you will easily see if two different symbols with the same name are used.

Generally, it’s really hard to rename existing symbols from the source code, especially if they are public and may have been referenced by other compilation units. Always try first to achieve what you need by transforming only the implementation, or adding new members (so you know they haven’t been referenced yet).

You could also just rename a symbol by assigning a different name (all existing symbol references would still be valid).

The “right way” is to catch the symbols you want to modify during namer, when they are entered in scope. Not sure what are the exact hooks, but I remember something like pluginEnterSymbol in the API. Warning: At that point trees are not yet typed, but if all you need is available in syntactic trees, you’re good.


#5

Hi @dragos, thanks very much for your helpful response. You are definitely right; I have also been noticing that my solution is too fragile and only works for small toy cases where I am transforming every single line in my source file. Through some debugging, I found that symbol.info.decls (the scope within the copy of my object Test) was not properly populated with its members (or rather, the members from the object body); it was actually empty. This would require me to re-add all the members and, as you said above, still would not solve the issue of being referenced in other compilation units.

I was originally separating out my plugin components such that it partially ran after the parser phase, but I was hoping the entire plugin could run after the typer phase. It does, however, seem like it might be best to keep that separation moving forward.

Btw, the -uniqid tip is fantastic.

cc: @samdow


#6

@olafurpg thanks for your helpful response. Unfortunately, macros won’t work for our project; we want most, if not all, of the plugin to run after the typer phase. Btw, the comment is a good, interesting find, thanks!