SIP-46 - Scala CLI as default Scala command

Tooling
21

Even though it works with any compiler version, an updated scala-cli might be necessary if my script relies on a using directive introduced in a new scala-cli version. If the scala version using directive ensured that the scala and scala-cli versions were guaranteed to be in sync, this wouldn’t be an issue.

1 Like
22

Even though it works with any compiler version, an updated scala-cli might be necessary if my script relies on a using directive introduced in a new scala-cli version. If the scala version using directive ensured that the scala and scala-cli versions were guaranteed to be in sync, this wouldn’t be an issue.

It would probably be better in some cases, but that clashes with the whole idea of package managers if we did a behind the scenes update.

We can do that though I think via parameter --scala-cli-version so that would theoretically be possible.

23

This can work currently via for example

scala --cli-version 0.2.0 version

1 Like
24

This proposal was discussed during the SIP meeting today. There was a long discussion (see below), but ultimately the Committee voted to accept moving the proposal from the experimental stage to the completed stage. Here are the points that were raised by the Committee:

  • The content of the proposal should be updated to reflect the changes that happened during the experimental phase (e.g. drop of multi-line using directives in the syntax specification, possibly express their syntax without relying on the grammar of Scala).
  • The versioning adopted by Scala CLI has complex implications (in addition to the points raised by @smarter above):
    • following the same versioning as Scala 3 may foster some confusion because the users will believe that they need to update Scala CLI to use a newer version of Scala although what they need to do really is to bump the version of Scala in their using directive (ie, they may think that Scala and Scala CLI are the same things, although they are not exactly the same).
    • on the other hand, with separate versioning, beginners may perceive some unnecessary complexity (“why do I need a tool besides the compiler itself?”). Also, if Scala CLI gets installed via a package named scala (as in cs install scala or brew install scala), it has to use a version that is higher than the current version of the existing scala packages (someone suggested using a version prefixed by the year, like 2023.1.0.0).
3 Likes
25

This proposal was discussed during the SIP meeting today. There was a long discussion (see below), but ultimately the Committee voted to accept moving the proposal from the experimental stage to the completed stage. Here are the points that were raised by the Committee:

This is great to hear! I finally have some time to put things in motion.

  • The content of the proposal should be updated to reflect the changes that happened during the experimental phase (e.g. drop of multi-line using directives in the syntax specification, possibly express their syntax without relying on the grammar of Scala).

Next ScalaCLI version will only support one line using directives, which has already been merged. I am also working on removing any leftovers of the previous approaches.

As for versioning, I don’t think we have a better choice than using the current Scala version and adjusting release cycle to resemble it. This is because the scala command will be expected to be the same version and anything else can cause confusion. This might not always work ideal, but will probably cause least amount of confusion. The default used Scala version in each Scala CLI will match that same version.

The next release of Scala CLI will be 3.3.0-RC4 to match the next RC of Scala 3 which will be most likely released. We will be able to do some last checks at that point, prepare PRs and once 3.3.0 is out we can release that and officially become stable.

Let us know if you have any doubts or questions!

26

Maybe I’m misunderstanding, but if the release versions will be tied to one another, what happens if scala-cli needs to make a patch release in-between the Scala releases?

4 Likes
27

This hopefully shouldn’t happen, but we could do 3.3.0-P1 or something similar as a workaround. Or just wait until the next release of Scala. We could make RC versions follow Scala RCs and this way we would have a ton of testing time for the final version.

28

Another option we could consider is having date based versioning such as 2023.04.01, however this has some drawbacks:

  • decoupled versioning scheme, user installs newest version and the package manager says the version is not what they could expect. This also breaks the current contract for coursier and brew.
  • it’s not semantic versioning, which might not be necessary for a CLI tool, but one could imagine users of certain jar components of the runner.

While I think the drawbacks of the suggested solutions are mostly to do with possible breakages:

  • we might have a bug inside ScalaCLI that needs fix right away (should be solved by releasing RC versions)
  • if a bug doesn’t need a quick fix, we can wait 6/8/10 weeks until the next Scala version is released

I think the new workflow with the aligned versioning would be:

  • Scala x.y.z-RC1 is released
  • Scala CLI branches x.y.z
  • Scala CLI releases RC1
  • if Scala and Scala CLI both work correctly next versions will be also released together
  • in case there are breakages in Scala CLI we could do more RCs (would be very rare)
  • if no breakages happen we just release RCs along with the compiler until the full version is out.
29

Thank you Tomasz for elaborating on that.

I don’t understand what you mean by “user installs newest version and the package manager says the version is not what they could expect”.

What is the issue with Coursier or Brew?

I think this is a good point. A solution could be to follow a scheme like epoch.major.minor.patch, where epoch would be 2023, for instance, followed by the usual major.minor.patch numbers of semantic versioning. But I agree that it’s a bit convoluted…
Regarding the possible issue you mentioned about the jars, I would advise using different versioning for the jars and the CLI since the interpretation of the semantic versioning is completely different for them (ie a breaking change at the bytecode level can be completely unrelated to a breaking change at the CLI level).

30

How about the Scala 2 side of the story? You write:

The default used Scala version in each Scala CLI will match that same version.

But when asked for “Scala 2” (or “Scala 2.12” or “Scala 2.13”), will it also default to whatever the latest such release was at the time Scala 3/Scala-CLI was released? I think that’s what I suggest, but perhaps it needs discussion?

One subtlety here is that Scala 2 versions lag Scala 3 versions. Scala 3.3.0 will use the Scala 2.13.10 standard library, but then within a week or so after the release of 3.3.0, Scala 2.13.11 will come out. Eventually Scala 3.3.1 will come out, and then the versions will be in sync again. Maybe that’s the best we can hope to accomplish?

31

The default version of the runner is a Scala 3 version. If Scala 2 is specified it will use that specific version or the newest possible if for example “2” or “2.13” string is used. I think trying to match both Scala 3 and 2 release cycles might become a bit problematic. Also, SiP is only for Scala 3 as far as I understand and we don’t plan to make the changes for Scala 2? Or would we want also 2.12.17 runner to default to Scala CLI?

In that case we would most likely need to have a separate release cycle altogether :thinking:

32

Let’s say a user hears about this new amazing thing “Scala 3”. They do brew install scala and see a different version installed. They start googling and investigating what is that about and most likely finds a most upvoted comment on stack overflow that explains the differences and what Scala CLI is. This is way too much information compared to what they needed.

33

I think a scala-cli bug should be treated like any other scala bug. If it’s critical enough, we could make a new patch release of scala, otherwise the fix will come along following the regular release schedule.

4 Likes
34

I am not sure about this argument. This is what I currently see:

$ scala version
Scala code runner version: 0.2.1
Scala version (default): 3.2.2

For me, it is clear that the runner will use Scala 3.2.2 by default. I think we could also inform the users better in the documentation about the difference between the CLI and the Scala compilers.

To be honest, I am more worried that your proposal may cause more confusion if the users try to update the version of Scala of their project by updating their scala package without success instead of bumping the Scala version in the //> using scala "3.2.2" directives.

2 Likes
35

I think the intention of package managers is to use them to update the version. We provide alternatives, but in essence we want package managers to deal with it.

Maybe it’s not that big of an issue I agree, but still I think with using an aligned version we can at some point even integrate the runner back into the compiler while with the other approach we commit to that specific way of versioning forever.

Do you have specific points against using aligned versioning? I think I am missing it or maybe if you put that into a short list it would be easier to weigh all the pross and conds.

36

I am surprised to read that, I would expect the recommendation would be to set the Scala version of Scala programs via a directive //> using scala …, not via the version of the scala package that is globally installed. This goes against the goal of reproducibility and portability.

37

I think the only drawback I see is the one I already mentioned: users may get confused that their program does not use the same Scala version as the runner (because it uses the Scala version defined in a //> using scala … directive). (But maybe that can be addressed by informing well the users.)

38

We should not force users to set Scala version especially if the program is trivial. We can recommend that for industry or CI scripts etc. But it’s not really necessary for beginners or learning. We also don’t have a way of informing of that recommendation unless we show a warning about it, which we shouldn’t as it will not be useful for a lot of people. So in an absence of a clear way of enforcing this policy we should at least adhere to have package manager does things, since this is something that the user will most likely understand.

If you want something to be reproducible/portable that’s when you add the version directive.

In this case they have the version shown in the script, this is less confusing than having a random version in case of absence of the directive.

39

I think that it should be fine to continue the versioning scheme as is, or at least bump to 1.0.0 - e.g. homebrew formulas can install binaries with different versions to the “main” package, people should be familiar with this (e.g. rust formula install cargo with a different version). If we adapt the current home-brew formula, we can instead only install the scala-cli tool (as scala). When Scala 3.3.0 (from lampepfl/dotty) releases, the homebrew formula just needs to wait until the scala-cli version that defaults to 3.3.0 to release to then update the installed binary.

Edit: the new homebrew formula would also not need to wait to include some patches while dotty is between releases.

3 Likes
40

Would that work in coursier and other package managers? Brew is one thing, but we want Scala runner to be much more available than it is today. And we would get some older releases in some of them, which if not forced to ScalaCLI wouldn’t even install for users. Or could we force it everywhere?