Thanks a lot to Ólafur, Eugene and the Scala Center in general for setting up such a thorough and transparent process.
Here are my personal thoughts, as an extensive user of macros:
- Quasiquotes
The first use-case for whitebox macros that comes to mind is of course quasiquotes, because we often want what is quoted to influence the typing of the resulting expression. This is invaluable when one wants to design type-safe quasiquote-based interfaces. For example, see the Contextual library. Haskell has similar capabilities thanks to Template Haskell.
- Type-Safe Metaprogramming
This extends the point above, but it goes much further.
We have been working on Squid, an experimental type-safe metaprogramming framework that makes use of quasiquotes as its primary code manipulation tool. Squid quasiquotes are statically-typed and hygienic. For example { import Math.pow; code"pow(0.5,3)" }
has type Code[Double]
and is equivalent to code"_root_.Math.pow(0.5,3)"
.
(You can read more about Squid Code
quasiquotes in our upcoming Scala Symposium paper: Type-Safe, Hygienic, and Reusable Quasiquotes.)
The main reasons for using whitebox quasiquote macros here are:
-
to enable pattern matching: we have an alternative
code{pow(0.5,3)}
syntax that could be a blackbox, but it doesn’t work in patterns (while the quasiquoted form works); making patterns more flexible might be a way to solve this particular point; -
to enable type-parametric matching: one can write things like
pgrm.rewrite{ case code"Some[$t]($x).get" => x }
. This works thanks to some type trickery, namely it generates a local modulet
that has a type membert.Typ
, and types the pattern code using that type, extracting anx
variable of typeCode[t.Typ]
. This is somewhat similar to the type providers pattern. Therewrite
call itself is also a macro that, among other things, makes sure that rewritings are type-preserving. -
to enable extending Scala’s type system: we have alternative
ir
quotation mechanism that is contextual in the sense that quoted term types have an additional context parameter. This (contravariant) type parameter expresses the term’s context dependencies/requirements. Termval q = ir"(?x:Int).toDouble"
introduces a free variablex
and thus has typeIR[Double,{val x:Int}]
where the second type argument expresses the context requirement. (IR stands for Intermediate Representation.) Expressioncode"(x:Int) => $q + 1"
had typeIR[Int => Double,{}]
because the free variablex
inq
was captured (this is determined statically). That term can then be safely be ran (using its.run
method, which requires an implicit proving that the context is emptyC =:= {}
). Thus we “piggyback” on Scala’s type checker in a modular way to provide our own user-friendly safety checking that would be very hard to express using vanilla Scala.
As you have guessed, this relies on invoking the compiler from within the quasiquote macro. I understand that this is technically tricky and makes type-checking “inseparably intertwined” with macro expansion, but on the other hand that’s also an enormous advantage. If it’s possible to sanitize the interface between macros and type-checkers, that would give Scala a very unique capability that puts it in a league of its own in terms of expressivity –– basically, the capability to have an extensible type system.
Could Squid’s quasiquotes be made a compiler plugin? Probably, though I’m not knowledgeable enough to answer with certainty, and I suspect it would be very hard to integrate these changes right into the different versions of Scala’s type checker.
As an aside, in Squid we also came up with the “object algebra interface” way to make language constructs expressed in the quasiquotes independent from the actual intermediate representation of code used. This seems similar to the way the new macros are intended to work –– the main difference being that we support only expressions (not class/method definitions).
- The
Dynamic
trait
I think the usage of the Dynamic
trait becomes extremely limited (from a type-safe programming point of view) if we don’t have a way to refine the types of the generated code based on the strings that are passed to its methods selectDynamic
& co. (doing so is apparently even known as the “poor man’s type system”).
If that is possible to do in a sane way, I could not recommend going with that possibility enough!