I don’t think I can define further more better than authors of scala-cli.
But I think such specification should be. And it should be on base of well known language.
I can suggest ideas of course:
I personally like the original idea of the scala-cli directives being very (stupid) simple and easy to use when writing scripts, sharing snippets, or just playing with some scala code or libraries.
My proposal is to keep only simple //> key value format and delegate all complex definitions to the respective config files with @file(...) or @url(...) syntax, possibly supporting as many data formats as we like. Example:
I agree, but I would also keep the using keyword. Scala is a keyword first language, it does not generally start something important following a symbol. Also, using can be syntax highlighted. Verbosity does not matter here; these are highly specific commands, it’s important to have some textual clue what they are. And, yes, I know the argument against the double use of using, but I don’t think it will be an issue.
That’s quite true, but this isn’t Scala, at this point. With the revised conceptualization, it’s just a build tool directive, with ScalaCLI being a very low-ceremony build tool.
In this way it’s more like Scaladoc comments (which IDEs have no trouble syntax highlighting).
/** Hello, scaladoc!
* @param bar I am an example parameter!
*/
def foo(bar: Bar): Foo = ???
//> main hi.i.am.a.build.directive.MainClass
Aside from the @, which is necessary because scaladoc by default takes raw text, and the multi-line vs. single-line comment, it’s a close parallel.
So I think we have precedent for doing things this way, too.
If we use doc comments, then we can arrive full circle at Activator for Scala 3.
I don’t get this fetish for the command line. Let me open a browser, review the doc for whatever I’m about to run before I run it, and instead of running sample code, I click to build and run the app.
I think there was already a thread about “scaladoc should have a built-in web server”.
The SIP should read: scaladoc as the the new scala command.
If it were closer to April 1, I would expand on this idea at greater length.
I would like to add another argument against the idea of having both use (in directives) and using (in Scala code) that I did not see so far. Since the words use and using are similar, I believe developers may confuse them and may write using in place of use and vice versa.
If we want to keep having a keyword after the //>, I think it should be either using or a completely different word.
The using directives syntax isn’t proposed as a standard for build configuration for Scala. The confusion stems from the Pre-SIP proposing standard build definition as an element of Scala 3 syntax. It was dropped, and later Scala CLI adapted //> comments as means of configuration. Even though it has sparked discussions for such a standard being established, it is perhaps material for a separate SIP.
Using directives as defined in SIP-46 are to be treated strictly as configuration syntax for Scala CLI.
There is no updated SIP document, it exists in the form it has been initially approved for the implementation phase.
However, the RFC specification of Scala CLI included in the SIP is based on the RFC Runner Specification page from the Scala CLI docs, link below:
That page is generated automatically after each Scala CLI release and should be the most up-to-date.
We have decided to drop the multiline syntax, yes.
The change is already on a pull request by @tgodzik, here:
Further changes will be made to remove them from any examples and relevant documentation.
It has been decided that using directives will indeed use a flat key-value structure.
This in turn means that for each singular key the user will be able to pass a single value, no value (just declaring the directive with the key) or multiple values.
The keys are flat, there is no enforced structure on this level.
However, a key can include a . character in its name, which doesn’t make it any less flat.
Kebab-case named keys for using directives aren’t supported, this should already be tidied up in the next Scala CLI release.
It is not the goal of SIP-46 and should be addressed in a separate discussion, if a desire for it is present in the community (and given the number of reactions on this topic, it probably is a worthy, but separate, goal).
As mentioned earlier, using directives will implement a flat key-value structure.
Hey, so the directive you mention (//> using publish.developer "(...)") is actually an experimental feature, so technically it is not even a part of the SIP.
Now, we currently think there should be no DSL or custom syntax to parse complex values in directives.
Using directives should effectively implement the simple key-value structure.
The 3 segments of the value should effectively indeed be under 3 independent keys, as was suggested.
Before (and if) the publish sub-command and tied directives would be raised as part of scala in another future (separate) SIP, this syntax would have to be changed. And even that is strictly hypothetical at this stage.
As it is not part of SIP-46 and is experimental, the quoted syntax should also be treated as just that - experimental. I encourage any feedback about experimental features to be raised as discussions or tickets in the Scala CLI GitHub repo, btw.
The . character can, by all means, exist in a directive key, but as mentioned in an earlier post, it is still a flat namespace.
The clarifications seme to address my concerns, if I understand correctly how they apply to the SIP.
It is still unclear to me how “not standardising” directives is going to look like for the final proposal.
Does this mean that the SIP scala-cli variant will not support directives?
If the SIP (or even just the final tool) does include any form of directives, I think there is a high-risk that they become a defacto standard for build tool definitions – in which case I think there would still be all the issues people have been raising.
Note that given the simplified proposed grammar, it may also not be too problematic to standardize on this (for now). Maybe consider requiring [ and ] around the Values definition, which would make the proposed value format effectively a valid subset of JSON (thus YAML), and TOML.
The SIP scala-cli variant (which since v0.2.0 is indistinguishable from its experimental SIP scala distribution, btw) will support directives, but at this point the directives are to be treated strictly as configuration for Scala CLI and not part of the Scala language as a whole.
The door is not closed for standardisation in future SIPs.
It is not impossible for using directives to be deprecated at some point if a standard for the Scala language as a whole is established in the future, although we’d still need to keep them for a while for backwards compatibility’s sake.
This is the part that I think won’t be true. It is true today: Scala CLI is not Scala. But it won’t be true once we literally replace the scala command with Scala CLI. At that point, it will be the official syntax, since it is the official scala command. Perhaps it will be seen as even more official than whatever “scala” dialect that SBT or Ammonite or Mill support!
Once ScalaCLI has replaced the official scala runner, trying to argue to newbies
“hey this stuff you are running with the official scala command you installed following the official getting started instructions you read on the official scala-lang website isn’t official Scala syntax”
seems like something liable to cause endless confusion
I agree that the syntax is not set in stone, and can change later. But the cost of changing goes up as adoption grows. We already have people saying that changing the syntax today would be terribly expensive. But I imagine after ScalaCLI replaces the official scala command for a year or two, the cost of changing would be an order of magnitude more than it is today
Don’t get me wrong, I do agree those fears aren’t ungrounded.
However, at the same time, we have to draw the line somewhere to get anything done.
SIP-46 has been up for a good while. This thread itself is over a year old, by the way.
I don’t think it reasonable to expect a single SIP to address such a wide scope.
Setting such high expectations is bound to cause decision paralysis, while the ecosystem needs to grow at a steady pace.
I do absolutely agree that having a configuration syntax standardised in the ecosystem is a worthy idea that should be raised in a future SIP, the sooner the better.
However, it is not and should not be a part of SIP-46, which addresses a completely different scope: introducing a new runner to the ecosystem.
That’s the extremely simple grammar given that you have a Scala parser lying around.
Again, that makes sense if it’s part of the Scala language. It does not make sense any longer if the directives are metadata for an external tool. I think it bears re-examination. Even if ScalaCLI uses the Scala parser to parse the directives, unless it’s actually part of the Scala language, it’s better if it’s more accessible to other tools.
If you actually unpack stringLiteral and numericLiteral it’s not exactly simple any longer, especially if it’s Scala string literals which include unicode escapes and different quote types (including multi-line strings), and Scala numeric literals which admit _ characters.
This is not trivial to parse unless you have the Scala compiler lying around. And, for reference, the VirtusLab implementation does not use the Scala compiler, and has about 1400 lines (of Java) devoted to parsing (not all of which is needed if there’s no multi-line syntax, but still). It has about 150 lines just devoted to parsing strings.
In contrast, here’s the actual full grammar for a bare-bones key-value pair (using regex notation for character strings):
The spec you gave potentially–depending on what we think a string literal is–admits stuff like
//> using key """Hi, I am
//> using a "multi-line quote"
//> which is a bit odd"""
if the comment is stripped and passed in a block. And if not, um, well, why not? (Note that I have constructed the example to be maximally confusing to line-by-line parsers, also, illustrating that you can’t just ignore stuff you don’t understand and keep what you do, if this is a possibility.)
So, again, I think when this was going to be part of Scala language syntax this made sense.
Now that it’s not, I think it very much needs to be revisited. Assumptions about what is simple and what is hard are different in each case.
And the reason that this isn’t equivalent to some other part of Scala syntax is that despite you calling it a “runner”, its functionality overlaps with current build tools. So one of the very first things I’d want to be able to do is automatically extract ScalaCLI directives to a build tool of my choice. If I can’t parse the metadata, it’s really a pain in the neck to do that–you have to count on ScalaCLI itself to do a sane export.
And that means there will be even more pressure on ScalaCLI to take on more and more build tool functionality, because people won’t want to switch because automated tools are hard to find. People already don’t want to switch between their other build tools for exactly this reason.
Starting down this path because “we have to draw the line somewhere to get anything done” is not advisable. Some things are worth getting right at the outset because when done not-right-enough, they cause a whole lot of avoidable discomfort.
Now that I think about it, though, I think the multi-line comment version should be right out. The reason is that whatever your configuration language, you have problems like
/***
[scalacli]
problem = "My favorite symbol is ***/"
***/
In contrast, with single-line comments, nothing you put there can confuse the compiler. So I think regardless of what the configuration syntax is going to be, it should be single-line comments.
If you want strings, and you want numbers, and you want true and false, and you want multiple items, then I suggest that one-line JSON should be the value. You already get all that for free, and the only thing you have to put up with is nested arrays and objects and null, which isn’t such a bad thing to have to put up with. It can even be helpful! The only thing missing is _ separators in numbers. (And possibly infinity/NaN, but the VirtusLab parser doesn’t seem to get those anyway. Maybe I overlooked it.)
Reading a metadata block is then a process of converting a bunch of //> using key singleLineJsonValue statements into a Map[String, List[Option[Json]]].
My current thinking, informed by this engaging debate (thanks all!!), including arguments and counter-arguments for using using as keyword or not, is to give my (revised) recommendation as a SIP-46 reviewer as follows:
keep the one-line using-syntax as is, release scala-cli as scala, after multi-line directives syntax is removed + tidy up docs + update SIP-46 accordingly while promoting it from experimental to stable as soon as possible,
and then encourage a new SIP with a proposal based on more analysis of different use cases and requirements on a possible configuration syntax including the potential of alignment across runner and other tools, in dialog with maintainers of such tools. If it should be based on some subset of JSON, TOML, YAML, …, is part of that investigation, in conjunction with a new SIP.
If such a future SIP proposal is successful, then the current //> using- syntax can work alongside a potential new config syntax, perhaps for as long as until a couple of LTS releases or similar.
I realized a while ago that I revived this old thread, although there is a newer follow-up thread created in relation to that the SIP was accepted as experimental. So further discussions should preferably head over to this thread: SIP-46 - Scala CLI as default Scala command - #16 by tgodzik
@moderators Perhaps close this in favor of the newer thread?