Motivation
After much effort, as a user of the scala language, it seems apparent that the manner in which the quotes API is defined offers an unfortunately low level of usability. This is caused by several factors.
-
The manner in which the std lib makes all the types within
quotes.reflectabstract makes the API extremely difficult to use.-
Intellij can not figure out the extension methods whatsoever. So, for the many developers who use Intellij and would like to program against this API, they are severely limited in the code completion capabilities. This is debilitating when trying to use an API with compiler-level complexity. Maybe metals is better, not sure.
-
One might say āIntellij just needs to fix its completionā, but this does not fix the fact that this manner of definition is
A:strange, andB:completely loses exhaustive pattern matching capabilities. Now, imagine trying to writesymbol.tree mat...looking to match on the tree of a symbol, and not only can the IDE not find.tree, but even if you told it with(symbol.tree: Tree) mat.., you canāt exhaustively match on what the options are. Many people would probably quit here. Speaking from personal experience, I gave up on this API twice before I finally wrote an entire wrapper around it, and was then able to make actual progress.type DefDef <: ValOrDefDef given DefDefTypeTest: TypeTest[Tree, DefDef] val DefDef: DefDefModule trait DefDefModule { this: DefDef.type => def apply(symbol: Symbol, rhsFn: List[List[Tree]] => Option[Term]): DefDef def copy(original: Tree)(name: String, paramss: List[ParamClause], tpt: TypeTree, rhs: Option[Term]): DefDef def unapply(ddef: DefDef): (String, List[ParamClause], TypeTree, Option[Term]) } given DefDefMethods: DefDefMethods trait DefDefMethods: extension (self: DefDef) def paramss: List[ParamClause] def leadingTypeParams: List[TypeDef] def trailingParamss: List[ParamClause] def termParamss: List[TermParamClause] def returnTpt: TypeTree def rhs: Option[Term] end extension end DefDefMethods
-
-
The scoped nature where types like
TermandTypeReprare defined make it very restrictive to work with at scale. When writing a large or complicated program, its necessary to be able to split code up into multiple files, and define types and abstractions for your domain logic.-
Imagine trying to define the following type:
final case class Function( rootTree: Tree, params: List[Function.Param], body: Term, ) object Function { final case class Param( name: String, tpe: TypeRepr, tree: Tree, fromInput: Option[Expr[Any] => Expr[Any]], ) } -
In order to do this, you have to either define it within your function body, like:
def myCode(using quotes: Quotes): Any = { import quotes.reflect.* final case class Function // ... // do stuff with Function } -
Or within a class, like:
final class MyCode(using val quotes: Quotes): Any = { import quotes.reflect.* final case class Function // ... // do stuff with Function } -
It would be insane to try and define all this within a single function, so many open-source libs go with an approach like
#3. But, even then, its very easy to end up with multi-thousand line files inside something likeMyCode, because the type system is making it very difficult to split things out. It is sometimes possible, if you try really hard, to split things out into separate files, but even then there are limitations, and it makes things very messy:final class Types1(using val quotes: Quotes) { import quotes.reflect.* final case class Function( rootTree: Tree, params: List[Function.Param], body: Term, ) object Function { final case class Param( name: String, tpe: TypeRepr, tree: Tree, fromInput: Option[Expr[Any] => Expr[Any]], ) def parse(term: Term): Function = ??? // TODO (KR) : } } final class Types2[Q <: Quotes](using val quotes: Q) { import quotes.reflect.* final case class Function( rootTree: Tree, params: List[Function.Param], body: Term, ) object Function { final case class Param( name: String, tpe: TypeRepr, tree: Tree, fromInput: Option[Expr[Any] => Expr[Any]], ) def parse(term: Term): Function = ??? // TODO (KR) : } }final class Logic1(using val quotes: Quotes) { val types: Types1 = Types1(using quotes) import quotes.reflect.* import types.* def getFunction(term: Term): Function = Function.parse(term) // error, wrong `Quotes` type } final class Logic2(using val quotes: Quotes) { val types: Types1 = Types1(using quotes) import types.* import types.quotes.reflect.* // this matters def getFunction(term: Term): Function = Function.parse(term) } final class Logic3(using val quotes: Quotes) { val types: Types2[quotes.type] = Types2(using quotes) import quotes.reflect.* import types.* def getFunction(term: Term): Function = Function.parse(term) } -
It seems like it should be very intuitive to be able to do something along the lines of:
import scala.quoted.ast.* final case class Function( rootTree: Tree, params: List[Function.Param], body: Term, ) object Function { final case class Param( name: String, tpe: TypeRepr, tree: Tree, fromInput: Option[Expr[Any] => Expr[Any]], ) def parse(term: Term): Function = ??? // TODO (KR) : }def myCode(expr: Expr[Any])(using quotes: Quotes): Function = Function.parse(expr.asTerm) -
As a general principle, it seems that if using an API borderline forces you to define any related logic in a single file, it is not designed properly.
-
Potential Downfalls
It is possible that there is something inherent to the Quotes API that forces all instances to be scoped to the same Quotes instance, but this seems unlikely, for a few reasons:
- The API enforces that the only way you are getting an instance of
Quotesis via aninline def+interpolate impl, so its not like there are many instances of Quotes coming from different roots. You are only ever getting an initial instance from 1 place, and any other nested instances are derived from that one. - If you really care about the exact instance of Quotes which a
SymbolorTreebelongs to, it seems far more detrimental to have an API that encourages files thousands of lines long, with dependent types everywhere, and the only thing making it usable is a globalimport quotes.reflect.*at the top. Therefore, if you are quoting and splicingExprs, and have helper types with something likefinal case class MyType(repr: TypeRepr), then anyMyTypecreated in some nesting technically has the wrongQuotesinstance.
Suggested Design
package scala.quoted.ast
trait Quoted private[ast] {
def quotes: Quotes
}
trait Symbol private[ast] extends Quoted
sealed trait Tree extends Quoted {
def symbol: Symbol
}
sealed trait Statement extends Tree
sealed trait Term extends Statement
sealed trait Definition extends Statement
sealed trait ValOrDefDef extends Definition
trait ValDef private[ast] extends ValOrDefDef
object ValDef {
def apply(symbol: Symbol, rhs: Option[Term])(using quotes: Quotes): ValDef =
quotes.reflect.ValDef.apply(symbol, rhs)
}
This way, you still need an instance of Quotes to create instances of things, but you are not burdened with AST nodes being scoped as an inner class. And then the implementations can happen elsewhere, like:
package scala.quoted.ast.impl
private[quoted] trait Tree { self: ast.Tree =>
def symbol: ast.Symbol = implemented
}
private[quoted] final case class ValDef(quotes: Quotes, /* ... */) extends ast.ValDef