SIP-46 has been accepted and now is in the implementation phase. Its intention is to enhance the default scala runner with a larger number of features using scala-cli. This doesn’t mean that all Scala CLI features will be available in the default Scala runner, but the main ones and the overall UX will be included. Some features might be introduced later on within additional SIP.
Additionally, in order to replace it we need to make it as compatible with the older Scala runner as possible. The main compiler options should be available to use (with some required to be passed with -O). More details here.
There are currently two ways of installing the experimental version of scala runner:
with brew:
brew install virtuslab/scala-experimental/scala
with coursier:
cs setup
cs install scala-experimental ← this command will replace the default scala runner
Quick overview of current features can be found at scala-cli website
Please let us know if you have any questions or issues. We would be happy to try and improve anything related to setup or features of ScalaCLI as Scala.
This doesn’t mean that all Scala CLI features will be available in the default Scala runner
Can you say a little bit more about this? I thought the idea was that scala-cli would be scala and we wouldn’t have to keep telling people to install something else.
The general idea is that the new scala command will initially support a subset of features from scala-cli that more or less map to the current scala command. Those are the MUST features in the proposal. scala-cli (which will get installed together with scala) will support all of the features.
As those features stabilize, they will gradually move into the main scala binary via additional SIPs.
I think it’s a bit different: The SIP specced features that a scala runner must have, and it will stop there. So it specced a minimum functionality of a runner, but not the precise functionality. The scala command is free to support other features as well. No SIP will be needed for these, unless we want to postulate that they will also have to be in any future runner we might think of.
So from that perspective I see no impediment to simply rename scala-cli to scala. There might be another impediment though that I have overlooked here.
I was referring to this part of the SIP that got approved:
In order to be able to expand the functionality of Scala CLI and yet use the same core to power the scala command, we propose to include both scala and scala-cli commands in the installation package. Scala CLI already has a feature to limit accessible sub-commands based the binary name (all sub-commands in scala-cli , and a curated list in scala ). On later date, more features from scala-cli could be included into scala command by additional SIPs or similar processes.
Whether we need a SIP or not to “enable” more commands in the main scala runner is probably not as clear cut as I implied. It could probably be a less formal process.
I am a bit worried that introducing a new script named scala while still recommending / distributing other scala scripts will cause confusion. I think we should have a plan / discussion about what happens to the download / installation page and how the introduction of scala-cli as scala affects building releases / distributions / other packages.
It’s fine to share it with contributors early, though IMO it wouldn’t be too hard for contributors to create an alias or symlink for the moment.
This is a temporary solution only to allow the community to test it and give us feedback. After the SIP Committee decides to make it “stable”, we will update the official Scala distribution to use that scala command.
I have an issue with install-completions. It works with scala-cli install-completions but not with the new scala after scala install-completions after installing the new scala with cs install scala-experimental. The scala help text says it should work and the text about source bashrc comes up so it seems as if scala thinks it has installed completions but actually no completions work…
$ scala uninstall-completions
Problem occurred while uninstalling scala-cli completions
Updated /home/bjornr/.bashrc
scala-cli completions uninstalled successfully
$ scala install-completions
Updated /home/bjornr/.bashrc
It is recommended to reload your shell, or source /home/bjornr/.bashrc in the current session, for its changes to be taken into account.
$ scala set<TAB> #nothing happens
Btw. currently the best reference of what scala-experimental can do might be found at Commands | Scala CLI since it list everything in a much more compact matter. I haven’t previously linked to that directly.
Edit: I posted a more elaborated comment below that overlaps with this one.
I’ve tried to use the new scala command to run an old script I have that used to work with Scala 2.13.
I noticed the following compatibility issues:
the shebang header is different. It now has to be the following:
#!/usr/bin/env scala -S shebang
I wonder if it would be possible to remove the shebang command at all (like in the Scala 2 runner)? Otherwise, there should be a good documentation explaining how to migrate from the Scala 2 runner. I wonder if this issue is related to that as well.
self-executable scripts without extension are not supported. You have to add either the .sc or .scala extension. When I looked at that, I noticed that the file extensions have different meanings for Scala CLI (ie, .sc files are interpreted as a “worksheet” whereas .scala files are interpreted as regular Scala program— ie, with a “main” method). It seems that the Scala 2 runner supports both use cases regardless of the file extension: if a standard main method is detected, that method is called, otherwise we fallback to the “worksheet” mode (which is similar to the .sc mode of Scala CLI). There is an open issue related to that (but there is no discussion about the differences between .sc and .scala scripts).
the -save option is not supported anymore. My understanding is that Scala CLI now always behaves like the Scala 2 runner with the -save option. I would suggest recognizing it but showing a warning like “Unnecessary option: -save. This option is always enabled.”
We just had a SIP meeting where we discussed the action plan for Scala CLI to become the default scala command. Here is the outcome of the discussion, with links to the corresponding issues or discussions in the scala-cli repository.
Users should ultimately have only one tool on their machine. They will still be able to use the “restricted” features via the --power runner argument, so there is no need to have both scala and scala-cli. Related discussion: #351.
The documentation should be updated to reflect that change of perspective: instead of presenting scala as a subset of scala-cli, it would present scala --power as a way to get extra features (with no stability guarantees) over scala. This means that the website should primarily document the stable features, and then have extra sections to cover the --power features. Related issue: #1700.
The fact that the new runner requires a special shebang command to be used in scripts is an annoying incompatibility with the current runner. Would it be possible to fix it (ie, to remove the need to use shebang)? Related issue: #603.
The fact that the new runner differentiates “scripts” from “programs” based on their file extension is an incompatibility with the current runner, and it is also added complexity for the users (who have to reason about .sc vs .scala files). Would it be possible to follow the same approach as with the current Scala 2 runner, which is to fall back to the script mode if there is no “main method” defined in the program? Related discussion #1668.
As a by-product, solving the above issue would also solve the current impossibility of defining a script with no file extension. Related issue: #466.
There should be a mechanism to ensure that any “option” that can be supplied via a command-line argument can also be consistently supplied via a “using directive”. There are several possible strategies here, which are discussed in #1524.
Coming back to the conversation after some work on the ScalaCLI side I am happy to report that most of the issues were resolved.
Users should ultimately have only one tool on their machine. They will still be able to use the “restricted” features via the --power runner argument, so there is no need to have both scala and scala-cli. Related discussion: #351 .
We now have the same behaviour regardless of the name of the binary. Advanced features are only available after --power or after specifying scala config power true
The documentation should be updated to reflect that change of perspective: instead of presenting scala as a subset of scala-cli, it would present scala --power as a way to get extra features (with no stability guarantees) over scala. This means that the website should primarily document the stable features, and then have extra sections to cover the --power features. Related issue: #1700 .
We now have a single documentation website with power commands being clearly marked and accompanied by warning sections.
The fact that the new runner requires a special shebang command to be used in scripts is an annoying incompatibility with the current runner. Would it be possible to fix it (ie, to remove the need to use shebang)? Related issue: #603 .
The discussion concluded with us still needing the shebang command for scripts. We would otherwise need to change the default command to shebang instead of run, which differs in its syntax from all the other commands and would thus be confusing to users. Shebang will only accept one file/directory as input and the rest will be arguments for the script. For run you can add multiple files and -- separates the program arguments, which allows us to be more flexible with sources. -- is a known pattern used for example in rust.
The fact that the new runner differentiates “scripts” from “programs” based on their file extension is an incompatibility with the current runner, and it is also added complexity for the users (who have to reason about .sc vs .scala files). Would it be possible to follow the same approach as with the current Scala 2 runner, which is to fall back to the script mode if there is no “main method” defined in the program? Related discussion #1668 .
As a by-product, solving the above issue would also solve the current impossibility of defining a script with no file extension. Related issue: #466
This would require some additional steps (probably double compilation) to determine if something can be treated as a script. We decided against it and instead only when using shebang command we will default to script, so it’s now possible to define a no extension or a custom extension script. That behaviour is also not supported by the old Scala 3 runner.
There should be a mechanism to ensure that any “option” that can be supplied via a command-line argument can also be consistently supplied via a “using directive”. There are several possible strategies here, which are discussed in #1524 .
Almost all the options can be both supplied by a using directive as well as in the command line. If there is any that is not available and would make sense to be included in both, we can fix it promptly. I am against having a using directive to use any cmd option since that would make it less obvious for users what is going on. If there is an option from command line that is needed we have to make sure it’s available in using directives.
Let me know if that answered your questions and worries before the next SiP.
As for the current plan for switching to Scala CLI backed scala launcher is as follows:
These might require changing ScalaCLI version to be compatible with Scala releases (starting with 3.3.0) as the package managers might not update it properly otherwise.
Create or update other possible installation methods which are not currently well supported by Scala:
These might require changing ScalaCLI version to be compatible with Scala releases (starting with 3.3.0) as the package managers might not update it properly otherwise.
What does compatible mean? In general, I’m concerned by the fact that both the scala compiler and scala-cli have their own release schedule. How do we decide which scala version ships with which scala-cli version? Is there a way with using directives to specify a minimum scala-cli version supported by the script?
I wonder if it wouldn’t make more sense to have scala-cli use the same version number and release schedule as scala itself. This would match how cargo works in Rust and should be less confusing.
What does compatible mean? In general, I’m concerned by the fact that both the scala compiler and scala-cli have their own release schedule. How do we decide which scala version ships with which scala-cli version? Is there a way with using directives to specify a minimum scala-cli version supported by the script?
I wonder if it wouldn’t make more sense to have scala-cli use the same version number and release schedule as scala itself. This would match how cargo works in Rust and should be less confusing.
Sorry, I wasn’t very clear. By compatible I meant that we would most likely need to release together with the Scala release. So after 3.3.0 version is released we release scala-cli 3.3.0 etc. The default version would then match the Scala version.
I anyone wants to use a specific Scala version they can always set that in the script, otherwise we would just have the same version.
So if I have scala-cli from Scala 3.3.0 installed, and my script requires Scala 3.3.1 via a using directive, will scala-cli itself take care of downloading the new scala-cli from Scala 3.3.1 and use that to run my script?
So if I have scala-cli from Scala 3.3.0 installed, and my script requires Scala 3.3.1 via a using directive, will scala-cli itself take care of downloading the new scala-cli from Scala 3.3.1 and use that to run my script?
Since ScalaCLI works with any compiler version, it will only download the compiler jars etc and compile. New ScalaClI version will need to be updated via your package manager. Otherwise, it would be a nightmare to figure out.