Scala 3 has very good backward compatibility guarantees between the minor versions. On the other hand, after the recent release of Scala 3.1, we can see that the libraries should be really cautious with updating the compiler version, as it is forcing the bump on every user of that library. We do not want library authors to be stuck on old versions of the compiler as that would mean that they are locked out of many bugfixes, or we would need to spend enormous effort on backporting every bugfix to all past versioning lines.
To mitigate that, we propose the following plan.
- Newer version of the compiler should be able to generate output that can be consumed by older ones
- Library authors should be able to declare parts of API that require a newer version of the language than the rest of the library
- Users should be able to safely use symbols added to the language API in
3.x, when they are using
3.yas their output version, as long as those symbols do not require language features added after
3.y, even if
x > y.
We propose that the compiler should accept the flag
--scala-target that accepts the Scala version as an argument. The specified output version needs to be lower or equal to the used version of the compiler. The default value for this flag is its maximal value - the version of the compiler itself.
Specifying output version makes sure that tasty files produced during compilation would have a version matching output version. It also makes sure that no symbol is used that was added to the language api in any version newer than the specified output version. If the code is using any language feature that was added in a version newer than the output version, the compilation error should be raised. All new symbols added to stdlib in future versions need to be marked with some annotation specifying the version, e.g.
Local output version
There should be a possibility to mark any file in the project to be using a higher output version than what is specified in the project configuration. It can be implemented by top-level import, such as:
This should override other means of specifying output versions.
A symbol can reference a symbol from other files if and only if the file containing referencing symbols has a higher or equal output version than the file containing referenced symbols. This allows us to sort compiled sources at the start of the compilation. This means that if multiple files would be compiled together, the compiler first processes files with output version 3.0, then those with 3.1 (remembering symbols from previous files), then those with 3.2, and so on.
This feature gives library maintainers the possibility to gradually extend APIs with features requiring a newer compiler, without modifying the core of the library that can be used with older compilers. This has some limitations, e.g. it is not allowing for adding a new supertype for the existing type. On the other hand adding new types, new given instances and extension methods to existing types is possible and should be sufficient for relatively fast and stable improvement of library API.
Adding local output versions means that now it is a normal situation for the compiler to encounter tasty files with versions newer than the specified output version for the current compilation run. The compiler shouldn’t raise an error but instead should just ignore those tasty files together with corresponding classfiles. In the future, we may want to read classfiles to know what symbols can potentially reside there and use that information for better error messages for not resolved references that can potentially be resolved with a higher output version.
Language API compatibility
Multiple symbols that are added to the standard libraries in new releases do not require any new features of the language. Even though they won’t be available in projects with a lower output version. To mitigate that with every new release of the compiler we can also release the
compat artifact, which would contain all symbols added between 3.0 and the current version of the compiler. Every one of them would be defined in the file with output version matching the earliest version of the language that given symbol can be defined in. They would have the same qualified name as the “true” symbols.
compat artifact can be then added as an ordinary dependency to any project. This assures that the symbols are available both in compiletime and in the runtime. Those
compat symbols need to be marked in some way so the compiler recognizes them and ignores them if “true” symbols are also on the classpath. This case can occur when library
B is depending on
B has a higher output version than
A is using
compat as a dependency. We can implement mentioned marking either by new annotation or by some information in the metadata of the
The action plan
@sinceannotation and releasing them in Scala 3.1.1
- Gathering community feedback and implementing local output version. This will be released no earlier than with Scala 3.2.
- Creation of
compatartifacts. Those can be published independently from Scala’s releases schedule. Moreover, we do not need to work on them right now as only a minimal number of symbols was added to the stdlib in 3.1, and right now, the benefit from the
compatartifact would be negligible.