Please, please keep the Tasty Reflect APIs simple enough for unfamiliar developers

@nicolasstucki and @liufengyun. First of all, I would like to thank you, very, very, very much for your amazing work with Tasty-Reflect! What you are doing will enable the future Scala Macro Writers to build amazing things that we never dreamed possible before! Dotty macros and ‘inline’ are phenomenal and using them has been an absolute pleasure for me. Also @nicolasstucki, I owe you a personal, and heartfelt thanks for working directly with me on migrating Dotty-Quill from 0.27 to 3.0.0-M2 which has been wonderful. I think the entire Scala community owes you a debt of gratitude!

Secondly, I would like to beg you to please, please keep the Tasty-Reflect APIs friendly to people who do not know the internals of Tasty-Reflect. The concept of Sealing and Unsealing trees is a really, really beautiful one and the .seal / .unseal API was very intuitive. In the past few months I have been teaching Tasty-Reflect to a group of brave Drexel students and the following diagram has been enough to demonstrate the duality of Expr/Tree in Tasty-Reflect.

Unfortunately, the change to Term.of(expr) and term.asExprOf makes this much more confusing from the perspective of a newcomer.

I realize why these changes have been made and that the reason for these operations is a good one. I understand that going down to the Term layer is for power-users while most usage should be staying up in the Expr layer. As I have tried to demonstrate in 10685 however, because Quoted-Matching is much more restrictive then quasi-quoting, working with Quoted-Matching will require going down to the Term layer much, much more frequently then was the case for quasi-quoting. Once the terms are matched, they will all need to be sealed back up before any further action is possible therefore sealing is going to be a very frequent operation.

This is not to say that there is anything wrong with Quoted-Matching. I think Quoted-Matching is a wonderful construct that is much, much more powerful and useful then quasi-quoting ever was and I absolutely love Quoted-Matching. However, it is important to recognize that because of it’s restrictiveness, going down to the Term layer will be necessary for many, many cases (as I have tried to demonstrate in 10685.

Finally, I would like to ask you to please, please not rename the Expr constructor. I understand that the motivation in 10673 is to line up the API with the Value matcher with the rest of the Expr API, which makes complete sense from an internal perspective. For a external user however, this is completely unintuitive. In most of the Scala ecosystem we are used to regular type-constructors like List(x), Set(x), Option(x), Future(x) etc… becoming List[X], Set[X], Option[X], Future[X] etc… so having the standard way to construct Expr[X] by doing Value(x) is very, very un-intuitive.

@nicolasstucki, @liufengyun please consider these thoughts from a ignorant user like myself who does not know Tasty-Reflect internals. I believe my thoughts will be reflected by the feelings of many macro authors will start working with Tasty-Reflect to port their frameworks in the coming year. In the field of use-ability, there is an idea called “The Curse of Knowledge” which means that someone who knows a system from the inside can see certain things as completely intuitive, which a person not knowing the system will be completely baffled by. I think a little bit of that is happening now with Tasty-Reflect. There are not a whole lot of ignorant users like myself using Tasty-Reflect who can provide you this feedback but I would like to open up this question to the scala community.

To anyone working with Dotty macros (in the present or future), what do you guys find more intuitive:

To Seal/Unseal Tasty Trees:

val expr: Expr[T] = ???
// This:
val t: Term = Term.of(expr)
// Or this?
val t: Term = expr.unseal

val term: Term = ???
// This:
val e: Expr[T] = term.seal // or term.sealAs[T]
// Or this:
val e: Expr[T] = term.asExprOf[T]

To Construct an Expr

// This:
val e: Expr[T] = Value(v)
// Or this:
val e: Expr[T] = Expr(v)
3 Likes

I haven’t been following very closely, but from your post I understand that Value(v) is now used to construct constant literal expressions, correct?

If that’s the case, the name seems very much inadequate – but so does Expr(v)! I think an intuitive name would be Const(v) or Literal(v).

As for seal/unseal or Term.of/asEprOf, I don’t really have an opinion, though the symmetry in the former seems nicer and more logical, whereas the latter seem a bit strange and idiosyncratic.

4 Likes

The Value(x) constructor is being replaced with the Expr(x) constructor in the general case. This includes Values(list) instead of Exprs(list) as well as for the unapplys. See 10673.

1 Like

There have been some confusion on how this all started. The Value as an extractor was propose by some a a way to make the intent of the Expr extractor clearer. I argued that that change would cause some complications. Rather than keep arguing in the comments I just said I would create a PR for it and see what happens. Now that we have seen what this addition in code and see a larger list of drawbacks I do not believe this is something we would ever merge. That PR spawn the discussion that I was expecting but it got a bit out of hand as I could not state my intention for this to be rejected and just followed the proposal.

Another noteworthy point to take into account for the changes is that we have removed some duplications and unnecessary indirections in the API to help with binary compatibility. May may of the helper extractors or extension method will be added back in a library providing scala.quoted.util where we aren’t as constraint with versioning. I imagine other such packages should exist to provide common utilities an convinience methods for each specific domain.

That works for me! :+1:

I created a PR proposing a new version of the logic of Term.of with the simpler syntax that unseal used to provide. This version has both the advantage of being a method and having a name that describes the intention. See https://github.com/lampepfl/dotty/pull/10694

2 Likes

@nicolasstucki Here is another case-in-point with 10661.

Showing extractors on an Term used to be as simple as term.showExtractors.
Now I have to remember how the entire tree-printing subsystem is organized in order to just print out the Term extractors I need to use.

Printer.TreeStructure.show(term)

Can’t we just have a simple one-word function like showExtractors to quickly get a matching tree construct?

(Btw, I’m changing the title since I think this topic is more broad.)

The idea is to use it as term.show(using Printer.TreeStructure) or setup the PrinterStructure as the current given for Printer[Tree] and just use term.show.

I guess that makes sense from a design perspective but it’s yet-one-more-thing a user has to remember to do… and there’s getting to be more, and more, and more of these yet-one-more-things happening. The other thing is, most of my usages of showExtractors are just a quick debug for one-off purposes i.e. I quickly want to get a tree from a particular spot and then don’t need it ever again. That way, just putting in showExtractors is super-easy; all the alternatives involve multiple steps.

I’m sorry if I’m annoying but is there any chance would also have showExtractors? I know it’s another alias you’d have to maintain but maybe it’s worth it?