The introduction of @experimental annotation seems to have some unforeseen consequences which might force us to rethink how it is supposed to be used, what restrictions it should imply and in what circumstances.
Currently both defining and using experimental features requires that you use a snapshot or nightly version of the compiler. Introducing new experimental definitions requires annotating them with @experimental and you also need to annotate all subclasses of classes or traits defined as experimental. Usage of experimental features, without exposing them further in one’s API in general doesn’t require annotating but there are some corner cases, e.g. instantiating an experimental class works perfectly fine without any boilerplate but you cannot analogously instantiate an experimental trait (see Experimental trait cannot be instantiated anonymously · Issue #13091 · lampepfl/dotty · GitHub).
@experimental also causes problems while bootstrapping the compiler. As scala 3 standard library codebase contains features marked as experimental (including the @experimental annotation itself) it cannot be built with a previous stable release or a published RC (unless it’s a nightly), which forces us to still use 3.0.0 as the reference compiler (which didn’t yet have all the restrictions related to @experimental).
Also authors of libraries might want to add experimental features to their APIs without using experimental features defined in their dependencies and yet still be able to compile with a stable version of the compiler and even publish binaries containing experimental definitions but having a guarantee that nobody else will be able to depend on this part of the API unless just for experimenting.
It’s also not clear whether the sole fact of using a snapshot version of the compiler should be enough to enable usage of experimental features or this should be enabled explicitly e.g. with a compiler option or a language import.
Any suggestions about how to solve the problems described above would be welcome.
1 Like
IIUC, the idea to limit experimental features to snapshot/nightly came from Rust, so what happens there?
I propose a different scheme that may be too radical, but maybe it’s something that can be done more easily than I think. This will create two versions for each binary- an experimental one, and none-experimental one.
So if test libraries in SBT can be defined like so
libraryDependencies += "org.scalameta" %% "munit" % "0.7.27" % Test
Then libraries with experimental features can be used like:
libraryDependencies += "org.myorg" %% "mylib" % "0.1.0" % Experimental
Or without the experimental features:
libraryDependencies += "org.myorg" %% "mylib" % "0.1.0" //No experimental features
Also a brief meta-comment on the process here. IMO, it would have been better to ask the community for feedback before adding the experimental scheme in 3.x, just like any change to the compiler. I think we all come out wiser during these discussions, even if they sometimes explode into hundreds of bikeshedding replies like in the optional braces thread.
4 Likes
Personally I’m not very familiar with Rust but it seems that in general to enable experimental features one has not only to use an unstable (snapshot) version of the compiler but also each feature needs to be explicitly enabled with a compiler flag or an appropriate setting in the build definition: Unstable Features - The Cargo Book.
Regarding the bootstrapping problem, assuming nothing has considerably changed since mk: Bootstrap from stable instead of snapshots by alexcrichton · Pull Request #32942 · rust-lang/rust · GitHub, each stable release has a magic value which can be set as an environmental variable to allow compiling experimental features (not sure however if this only removes the requirement of using an unstable compiler version or it also makes the flags for experimental features redundant).
Another difference between Scala and Rust with regard to experimental features seems to be that in Rust only features of the compiler itself can be experimental. In scala @experimental annotation is public so it can be used by authors of libraries to mark parts of their APIs as experimental. It might make sense to assume that if your library depends on some experimental features of another library, it should be treated as experimental as well. But it doesn’t seem necessary to require that it’s built using a snapshot version of the compiler if it doesn’t at the same time use experimental compiler features. So it would probably make sense to divide the concept of experimental for the compiler and for libraries.
There is plenty of symbols in Rust API annotated as experimental (or #[unstable] as it is called in code). For example, some methods of Option are still considered experimental, even though they do not use any unstable compiler feature in their implementation. To use them you absolutely need a nightly build of the compiler.
Moreover, they are transitive, in such a way that if some library is using any unstable call, all downstream projects consuming this library, must be also compiled with a nightly compiler. Of course, most libraries are using #[cfg] annotations to by default only expose stable parts of their api. Unstable parts can be accessed by using cargo features (example).
4 Likes