Since this proposal is implemented before type checking, any IDE based on presentation compiler should support XML literals automatically.
Metals is supporting the current macro based implementation at the moment.
1 Like
IntelliJ Scala plugin has its own Scala language parser. JetBrains plan to make some hybrid approach, i.e. use some of LSP functionalities to enrich assistance, but they don’t plan to give up on their parser. That custom parser will be the foundation of IntelliJ Scala plugin for the foreseeable future.
The last SIP meeting ended up being private-only, but we did discuss this SIP. We will discuss again in public on November 27th, but here is nevertheless the gist of the feedback from the SIP committee.
Overall, there seem to be agreement that the name-based approach can be good. However, there is still a lot of reluctance in supporting first-class XML literals in the language. Instead, we would like to see this implemented as a library using a string interpolator macro. Since pattern matching does not need to be supported, it should be possible to do this both in Scala 2 with scala-reflect macros and in Scala 3 with inline, quotes and splices.
Therefore, the feedback of the committee is that the author (or anyone else) is encouraged to try and implement the proposal as a macro library. @yangbo Given your track record of macro implementer, it also seems to us that you should be more than qualified to do so.
I feel sad for the SIP meeting becoming private.
At the next meeting, you’ll have the opportunity to propose changing the committee name from “Secrets In Private” to “Sad In Public.” Of course, this is a far cry from its original charter, “Seemingly Impossible Proposals.”
1 Like
Let me give some additional context: I think this is very nice work. However, I would like to be able to replace XML literals in Scala 3 with a xml"..." string interpolator. So far, that proposal is on hold since we want to have a good official interpolation library in place first. We would hope to be able to ship such a library with the standard Scala 3 distribution (this would be an exception to the comment in SIP: Curried varargs - #38 by sjrd). I see a lot of potential for a variant of name-based XML literals to become that library. So, I would be happy to see a worked out proposal along these lines.
One other remark: Scala-3 macros are typed so we cannot simply issue an identifier and hope that it can resolve to something. However, it seems the desired effect could be achieved in a safer way with implicit function types. I.e. the interpolator would generate instead of an XML tree an implicit function type that takes implementations of the generators and yields an XML tree.
I believe the alternative approach requires dependent-type type classes, but I wonder if the optimizer can inline functions in type classes.
@yangbo Thank you for writing this SIP draft! https://docs.scala-lang.org/sips/name-based-xml.html
Scala.js is near to the first 1.0 release, and having this simplified XML literal will help users a lot to develop type-safe web applications at ease with Scala.js.
I’m not sure using Scala interpolator + macros is the right approach to implement such an XML literal support. What’s the downside of this? (e.g., compilation speed, etc.) Parsing XML literal inside Scala macros was not so fun experience for me because of its slow performance. If we have no support from Scala compiler, we need to write an XML parser, which runs inside Scala macros, by ourselves, but it should not be a big issue.
1 Like
In theory an XML parser that would run within a macro will be faster than the parser from scalac as it parses a more concrete syntax. The overhead of calling a macro might have some performance overhead.
This proposal, or similar mechanism is not able to be implemented in macros because macros do not support untyped AST generation, which is required by any named based semantic sugars. All the existing semantic sugars, including scala.Dynamic, name-based pattern matching, for comprehension are implemented in the built-in scalac/dotc parser or typer.
1 Like
It is not clear to me why you would need untyped ASTs for these cases. Could you provide a concrete example?
If it is only about the sugar un the parser, then it is always possible (and recommended) to call directly the desugared vesions of those calls when generating the code.
I think the right way to get around that is to replace name based by implicits. I.e. where previously you demanded a name X to be bound, you now demand an implicit for type X. This is awkward in Scala 2 because it would mean that all names/implicits that could be demanded must show up in a function’s signature. But in Scala 3 we have scala.compiletime.summonFrom which allows to request implicits that are not part of the interface and to handle the problem gracefully if they are absent. summonFrom is very powerful in what it can do. It deserves to be better known.
Possibly, but I don’t see that happening without considerably expanded documentation. The state of macros is currently just this side of black magic, as the documentation is either non-existent or quite sparse.
This is particularly unfortunate as, from what little documentation does exist, and the various references to “you should just use a macro for that” from folks closer to the implementation, hint there’s quite a bit of functionality there. Tupled functions are in a similar state.
It’s true that we need better docs for macros. But the people who work on them can only do so much. We are already severely overstretched. Also, the people who work on the new meta programming facilities generally understand compilers much better than applications which means it’s harder for them to come up with realistic examples.
So a plea to the community: Please contribute tutorials! Various people do get interesting stuff done with the new macros. It would be great if you write up what you have learned in an easily digestible way.
4 Likes
I understand that, and empathize with how much you’ve got on your plate. It’s just a little disappointing to be told, “macros can do this”, but have no way to proceed other than forking the compiler and implementing the macro there first - and yes, that was a real suggestion.
An example of this was the back-and-forth in the thread on Curried Varargs, which started at the very end of November and involved over a week of burning my evenings failing to figure out something that @nicolasstucki was able to produce in a couple of hours (and I cannot stress enough how much I appreciate him doing that).
I’d hazard a guess that, if each time someone close to the implementation was tempted to comment something along the lines of, “have you tried a macro for this?”, that if they provided a concrete example of how one could go about doing this using a macro or other metaprograming techniques, then it would come up less often as the body of examples grows.
If they don’t have time to provide some actionable hint, and I totally understand that may be the case as everyone’s time is both limited and generally overbooked, maybe “have you tried a macro for this?” isn’t actually helpful, and it might be worth considering not commenting it.
It may not be immediately useful, but the point of such a comment is to indicate that this is the sort of problem which is expected to be solvable using macros, which means that it doesn’t need to be implemented as a language feature and that the SIP committee doesn’t need to be involved. This is a good thing! It means that the problem is solvable without waiting for it to be officially included in the language. It does not necessarily mean that implementing it will be easy, but it’ll certainly require less man-hours than debating, adding and vetting a new language feature. And once it’s implemented as a macro, it can quickly evolve based on usage instead of being frozen because of compatibility requirements.
1 Like
If the intent of such a statement is to end further discussion, I suggest actually saying that. “This is something we expect macros to eventually support and we do not intend to entertain further discussion on this point. You’ll need to do without in the mean time” would be less problematic, or at least far less ambiguous.
As it is, “have you tried a macro for this?” implies a lack of due diligence by the person asking for the feature, which isn’t really fair given the lack of a realistic path to acquiring the necessary knowledge to use them. Not everyone can sink the required days or weeks into getting familiar enough with the compiler internals to get to the point where they can forgo the need for public documentation.
No, it’s really not. It means that someone with intimate knowledge of the macro implementation thinks the problem might be solvable via macros, not that it’s actually solvable by someone without that knowledge.
Even seeing if a particular problem is actually solvable using macros requires waiting for the documentation to be fleshed out enough for someone outside the compiler group to have a go at it. In the cases where it’s not at all clear macros are a viable solution (at least with the understanding publicly available), as is the case with Curried Varargs, it becomes a situation where there’s a decent chance that we’ll need to wait until macros are documented enough to have a go at implementing it, then wait again for it to potentially be officially included in the language.
This is why I’d like to suggest that, should someone be interested in using macros to give the brush-off for a feature, they put their money where their mouth is and at least try to implement a POC and provide it as an example. If nothing else, dogfooding the macros this way will result in a better implementation.
Edit: fixed a wrong “there”
1 Like
FWIW, I agree that the “have you tried a macro?” language is unhelpful if macros are not sufficiently well-documented for it to be reasonable to expect the reader to actually be able to write the macro.
Instead, something like, “In principle, this functionality could be provided by a macro,” would signify that the poster believes that (1) the feature could, in fact, be provided by a macro, but (2) it may not be true that anyone who is asking for it could create it with macros.
2 Likes
What is the benefit of summonFrom in comparison to searchImplicits in a macro?
summonFrom is a language feature. searchImplicits is some lower-level functionality in the compiler that may or may not be exposed to macros. It probably won’t be exposed, since summonFrom can do the job already.