We started an initiative focusing on creation of the new API of the Presentation Compiler to be used by Scala tooling. The main idea behind that project is to create a public, stable API that would stay binary compatible.
Currently, Metals, scalafix and other tools, rely on the
interactive package implemented inside the compiler. It provides a base implementation for the various IDE capabilities such as code completions, hovers, signature help or even go to definition. Secondly, it provides an API to manually implement new logic on abstract syntax trees such as
Interactive.pathTo which finds the enclosing tree for the given position.
Then, when the new version of the Scala compiler is published, we need to make a new release in those tools that hopefully don’t require any changes or, on the other hand, need special handling to make them work after changes in the compiler. As compiler developers are working on the new version of the compiler on every release (even the release candidates), we must either republish tools with the newest version, or make a workaround such as fallback to the previous version (which could break, and not work at all due to binary incompatible changes). This leads to a situation, where tools such as Metals support only a subset of Scala Versions.
As you can notice, it also introduces the next problem: some bugs, won’t be detected in the dotty repository until they are tested in dependent repositories, as
interactive test coverage of IDE capabilities is not as vast as for e.g. Metals.
We intend to solve those issues by moving the Stable Presentation Compiler API to the
dotty repository. Its definition is available in metals repository: metals/mtags-interfaces/src/main/java/scala/meta/pc at main · scalameta/metals · GitHub
Moving the implementation of this interface will force every fix to happen in the compiler while making it easy to always support any Scala 3 version, by releasing it along the compiler. It will solve the issue with only a subset of supported versions. Its implementation will be based on
mtags implementation, as it already handles most of the use cases.
The stability will also be improved, as instead of relying on tests defined in
DottyLanguageServer we will have a new test suite ported from Metals to dotty. With this change, we will be able to detect errors and bugs before they are merged. Previously, tools such as Metals relied on nightly releases of the compiler, to run tests suite manually in their repository which sometimes led to failures, which couldn’t be resolved for some time and every CI action was failing because of that change.
- (done) Set up an initial meeting to clarify the goals of the projects and the role of each team member
- (done) Onboard the Metals team (they will maintain the implementation)
- (done) Agree on the proposed architecture and next steps.
- (done) Add MIMa for Presentation Compiler Interfaces
- (done) Separate shared Mtags into a new module
- (done) Add architecture to test Metals with a presentation compiler published from
- (WIP) Add
mtags-sharedas compile from sources dependency.
- (WIP) Move mtags implementation to the dotty repository as a new module
- (WIP) Port mtags tests to
- Add a nightly release pipeline for presentation compiler
Q: How is this going to work with Scala versions that existed before the stable interface?
A: Older versions of Scala will not provide a stable interface for the presentation compiler. The first version with official support will be the Scala 3.3.0. Tools which are using presentation compiler already, should remain using it the same for older versions.
Q: What will happen to Metals support for the older versions of the Scala 3 compiler?
A: All new features for Scala 3 IDE will be implemented firstly inside the new stable presentation compiler implementation, and then they can also be implemented in
mtags for versions before LTS. We plan on gradually dropping the support for those versions and gradually removing
mtags completely from Metals in favor of compiler provided API.