Scala 3, macro annotations and code generation
Hi all,
Back in 2018, Macros: the Plan for Scala 3 | The Scala Programming Language described how we expected macros, and in particular macro annotations, to eventually look like in Scala 3:
[Macros] will run after the typechecking phase is finished because that is when Tasty trees are generated and consumed. Running macro-expansion after typechecking has many advantages
- it is safer and more robust, since everything is fully typed,
- it does not affect IDEs, which only run the compiler until typechecking is done,
- it offers more potential for incremental compilation and parallelization.
Since we recently merged a first draft for macro annotations support, I wanted to revisit the issue of porting existing macro annotations that expect to run during typechecking because they add new members to classes. For example, take @alexarchambault’s, data-class:
Use a @data annotation instead of a case modifier, like
import dataclass.data @data class Foo(n: Int, s: String)This annotation adds a number of features, that can also be found in case classes:
- sensible equals / hashCode / toString implementations,
- apply methods in the companion object for easier creation,
[…] It also adds things that differ from case classes:
- add final modifier to the class,
- for each field, add a corresponding with method (field count: Int generates
a method withCount(count: Int) returning a new instance of the class with
count updated).
For many years, our answer to “How do I do this in Scala 3?” has been “Use code generation”. But it seems that no popular code generation framework for Scala has emerged during this time (scalagen seemed interesting but was archived).
More recently, we’ve been considering having something like @data built into the language but there’s some concerns that this would bloat the language, and it wouldn’t help with other macro annotations.
Meanwhile, the Scala 3 compiler grew a -rewrite flag which can be used to automatically fix errors. For example,
def f(): Unit = ()
f
does not compile in Scala 3, but if I pass -source 3.0-migration -rewrite to the compiler, the source file will be patched to obtain:
def f(): Unit = ()
f()
Currently, the rewrite mechanism is only used to ease migrations, and it is difficult to trigger since it requires fiddling with compiler flags. But in the future we should be able to expose this better to the outside world so that you can apply rewrites from the comfort of your IDE by clicking a button.
This brings me back to macro annotations: even if we cannot add new definitions visible during typechecking, we could have the macro just check if appropriate definitions exist, and emit an error with an appropriate automatic rewrite if they don’t. For example given,
@data class Foo(n: Int, s: String)
Running this code in my IDE should give me a red underline, clicking on the “fix it” button, could then rewrite the code as follow:
@data final class Foo(n: Int, s: String) {
def withN(n: Int) = data.generated()
def withS(s: String) = data.generated()
override def equals(x: Any): Boolean = data.generated()
override def hashCode: Int = data.generated()
// ...
}
object Foo {
def apply(n: Int, s: String): Foo = data.generated()
// ...
}
where data.generated is an inline def which generates the correct method body depending on the context (we could also generate the actual method body inline, but relying on an intermediate method keeps the amount of generated code to the minimum needed).
If at a latter point I decided to add an extra field x to Foo, I would then get a new error and clicking on the “fix it” button for that error would add the necessary withX method to the class while leaving everything else as-is. While this is more laborious than what was possible with Scala 2 macros, it means that the generated APIs are now easily readable for both humans and computers without having to understand macro code.
In other words, I’m suggesting we use existing facilities in the compiler to turn it into a code generation tool. This mean we wouldn’t have to worry about having to setup a separate tool and integrate it in our build pipelines. To ease cross-compilation, existing Scala 2 macro annotations could also be adapted to allow this style (by just not doing anything when detecting methods with the correct signature and body).
The main thing that will be needed to make this practical is some convenience methods in the reflection API for doing code generation (this won’t be completely trivial since we’ll have to handle transforming classes with existing definitions, and ideally avoid using fully-qualified names where possible for readability).
Before exploring this further, I’d be interested in hearing from implementers of macro annotations: would you be interested in using this pattern? For example, scio defines some powerful macro annotations for generating full case classes from a schema which seems like they could fit into this pattern, but I’m not familiar with how they’re used in practice.
Let me know what you think!
) but it won’t find much adoption unless it can be a transparent part of the compilation pipeline (i.e. it does not modify source files, only modifies the AST that goes to the next compilation phase – basically what annotation macros do in Scala 2)