No, arguments will have been typed against Any (as expected type) before macro expansion, but they will still retain their precise types, which will be used to refine the type of the expanded code.
Unless your var-args have a uniform type and you make the macro’s signature accept a T* (which would work for List.apply), you won’t be able to leverage expected types to guide type inference of the arguments.
So it’s not that they’d be erased to Any, just that the macro would need to accept Any*, effectively skipping type checks, which would then need to be implemented by the macro?
Yes, precisely. There is probably a way to check for subtyping in the macro API, but I’m not entirely sure.
There are 3 issues in the macro implementation of curried varargs:
Considering issue 3, this proposal has the potential to even reduce the complexity of the compiler by helping implementing string interpolation and HList as ordinary libraries instead of instrument functions.
Scala 3 does have some fashion of whitebox macros. Is it not enough?
https://dotty.epfl.ch/docs/reference/metaprogramming/inline.html#specializing-inline-whitebox
I don’t think macros actually convert ASTs to Tasty. They just use the Tasty-reflect interface into the compiler’s AST, as far as I know (needs confirmation).
They now are (see @soronpo’s link).
True, but if the macro ends up working well, it could be integrated into the standard library.
That said, I also think the overall experience (and type-based documentation) would be better with a first-class mechanism. The tradeoff with respect to making the language more complex is not that obvious, though. Indeed, allowing it as a plugin could change the parameters of the tradeoff, but then again giving this kind of power to people is a sensitive matter.
It is slower than analytic plugin or compiler built-in implementation because it type-checks twice and needs additional conversion between AST and Tasty in Scala 3.
This remains to be verified. I am not sure it will be much slower, nor that it would matter in the big picture.
Whitebox macros are supported in Scala 3 by now!
Sure. But, if this version is successful, then it might well migrate into the core, and then core libraries could depend on it.
Unfortunately currently Dotty does not support macro with varargs. The current type of 'vararg is Expr[Seq[_]], while we need Seq[Expr[_]]. Also there is no AST of each parameter when unseal the expr, instead it returns a Inlined(Nil, _, Indent("vararg"))
Check out how it was done here: https://infoscience.epfl.ch/record/267528
Not sure whether the code still works or not.
You probably need to call underlyingArgument, for up-to-date macro examples see https://github.com/lampepfl/dotty/blob/master/library/src-bootstrapped/dotty/internal/StringContextMacro.scala, GitHub - lampepfl/xml-interpolator: XML String Interpolator for Dotty, https://github.com/dotty-staging/scalatest/tree/dotty-community-build/scalactic.dotty/src/main/scala/org/scalactic
It’s concerning that we’re being told both: “you should do this via metaprogramming” and “you’re going to need this method, which is not mentioned in the docs”
This is particularly troublesome as the only supported IDE does not support jump to definition outside the current project, which really hurts discoverability.
Are there any up-to-date resources geared towards teaching this? Dotty is still experimental, so if I’m going to sacrifice my evenings and weekends to dig through this, I’d appreciate something more than, “here are some examples, subject to change without notice”.
For the general case, this would would require specifying a return type of <: Any, which seems to work against the point of having the limitation of only specializing a return type. I don’t have a particular problem with that, other than it feels like the sort of thing that the compiler is going to complain about.
Current Dotty macros are still not enough for implementing this proposal because it lacks of the ability to produce untyped AST. For example, in Scala 2, a macro author can create the untyped select Select(callee, TermName("applyBegin")) in spite of the definition of applyBegin. However it will crash in Dotty if applyBegin requires an additional type parameter.
Well, that was throughly demoralizing.
I’ve gone through all of the source files in those examples, re-read the docs multiple times, and I’m no closer to implementing a “show” interpolator which could serve as a POC for Curried Varargs than I was when I started.
I can go from Expr[Seq[Any]] to the individual Terms, but there doesn’t seem to be a path from Term to Type, which I’d need to summon the needed implicits. Found a whole bunch of examples of widening the type to Any, and a whole bunch of examples doing type tests against a hierarchy of known types, but nothing even hinting that going the other way is possible.
I even tried grepping my way around the Dotty source code, but 42 pages of results for “QuoteContext” is beyond my fortitude at the moment. As “jump to definition” isn’t currently supported, I’m pretty much stuck.
Finding the docs for underlyingArgument were two sections removed from “Macros” (both of those two sections dealing with compilation details), after a disclaimer that basically says, “Don’t use this unless you know what you’re doing”, was particularly disheartening.
Between this, and the results of my experiment with significant whitespace (I built a tool I’ve been wanting for a while, and it was every bit as painful as I remember from Python), I’m really losing enthusiasm for the switch over.
Maybe my suggestion is silly, but what if you developed the macro within the dotty project, so you can jump to definitions?
Here’s a small interpolator I threw together around a week ago that does plenty of type inspection.
The ExprSeq extractor is there for that. I added some documentation and an extra underlyingArgument directly on expressions to make it simpler to use in https://github.com/lampepfl/dotty/pull/7697
Thanks for the example, but this has the same problem as the other examples WRT being relevant to what I’m trying to do: it’s checking against a known list of types.
I’ll try and explain a little more clearly I’m looking to do, as the provided examples seem to indicate I have not explained it well.
show"Tried to foo the $bar, and got a $baz instead"
new StringContext("Tried to foo the ", ", and got a ", " instead")
.show(
Shown.mat(bar)(implicitly[Show[Bar]]),
Shown.mat(baz)(implicitly[Show[Baz]])
)
This works because it’s driven by implicit conversions, which for Scala 2 only need to be enabled at the definition site. This approach would be extremely problematic because, at least according to my experiments, in Dotty (and thus presumably Scala 3) you need to import the feature flag at both the definition site and the use site. It makes zero sense to require enabling a feature known to make code less comprehensible on a near global level to opt-in to a feature intended to make code less error prone.
new StringContext("Tried to foo the ", ", and got a ", " instead")
.s(
bar.show(given summon[Show[Bar]]),
baz.show(given summon[Show[Baz]])
)
Unfortunately, this means that I need to be able to summon a known typeclass for each passed argument. While the compiler apparently knows these types, I can’t seem to find any documentation or example hinting at how to access so I can splice together the calls. I’ve found plenty of examples of how to check if it’s one of a handful of types, so I accept that the compiler has this information, but as far as I can tell, it’s completely inaccessible if you don’t know it ahead of time.
While the above is interesting because I’d like Curried Varargs so I can do this reasonably, it’s also a much simpler problem and if we can’t do this it casts signifiant doubt on the plausibility of implementing Curried Varargs via a macro (as was suggested).
It is possible to find implicits within a macro using searchImplicitExpr http://dotty.epfl.ch/docs/reference/metaprogramming/macros.html#find-implicits-within-a-macro.
I will extend the documentation to add an example where it is used to find implicits based on types of some arguments. Note that I am updating the API to use summonExpr instead in https://github.com/lampepfl/dotty/pull/7687/files, I will probably add the documentation in that PR.
It might also be possible to find implicits using only inline and summonFrom http://dotty.epfl.ch/docs/reference/metaprogramming/inline.html#summoning-implicits-selectively. Though I’m not sure if it would work in this particular case.
I’m aware of searchImplicitExpr and summonFrom, I’ve gone over what’s in the docs multiple times. Once I get the type, summoning the instance isn’t a problem.
Getting the type is the problem, so if your example could include a case where the type isn’t known ahead of time (specifically, going from Expr[Any] or Term to Type) it would be extremely helpful.