(The Scala Center team is dedicated to providing regular and transparent community updates about project plans & progress. In this forum we created a new topic category to allow quicker orientation going forward: “Scala Center Updates”. Even though all feedback is welcome, we keep the right to make executive decisions about projects we lead. Overview of all our activities can always be found at https://scala.epfl.ch/records.html)
Dear Scala Contributors,
In this article, we will provide some updates on progress on the TASTy Reader for Scala 2 project and outline some milestones for our future work.
How you can try it out
While we wait for the support to land in the official Scala release, you can still try it out by building for yourself the Scala 2.13.2 compiler with support for reading TASTy. Clone the repository at tasty-reader-compat-testkit and adapt it to try out with your own Scala 3 dependencies.
Initiated in September 2019, the primary goal of this project is to allow a gradual migration to Scala 3, which will arrive in late 2020, with many changes to source compatibility with Scala 2.13, and some new APIs added to its standard library. This project will enable users of Scala 2.13 to continue to use libraries that have migrated to Scala 3.0, giving a long window for projects to migrate one by one.
We achieve this forward compatibility by having bidirectional compatibility with two interfaces of the Scala language: the binary runtime interface, handled by the Dotty team at LAMP in addition to help by the Scala Center; and signatures of library dependencies, handled by this project. TASTy Reader enables Scala 2.13.x to read signatures of compiled code produced by Scala 3.0, so that 3.0 libraries may be freely consumed by Scala 2.13.x, provided they only use API’s that can be projected to the common language subset, explained further on. More information about how this works can be seen in my January 2020 talk Taste the Difference with Scala 3.
As of May 4, 2020, we have already opened a PR to scala/scala, where we support all of the common language between Scala 3.0 and Scala 2.13 in API signatures, excluding annotations, which may only have simple expression arguments. Libraries published for Scala 3.0 are able to take advantage of some new features and remain consumable from Scala 2.13, with some caveats:
- Enums (equivalent to case classes and Java enum values)
- Intersection types (the same as compound types)
- Higher Kinded type lambdas (as Scala 2 style type lambdas)
- Opaque type aliases (opacity is lost)
- Scala 3 extension methods (as ordinary methods)
- New syntax for implicits (as its desugared form)
- Inheritance of open classes (the modifier is ignored)
- Exported definitions (used as ordinary definitions)
Looking forward, there are still some core features we would like to achieve in the coming months:
Milestone 1: Use Scala 2 macro definitions from a library published for Scala 3.0
A Scala 2.13 library that provides macros needs to rewrite the implementations to be compatible with Scala 3.0. However, currently, there is no way to provide a macro definition that can link in 2.13 with a library built by Scala 3, so a project must cross build. We would like to enable Scala 3 to provide definitions of macros to Scala 2.13 and 3.0 using the same API and avoid the need to cross build in this scenario.
Enable Scala 3.0 to provide a definition of a scala 2 macro in TASTy, alongside a scala 3 macro definition with the same signature. Scala 2.13 will instead link with the Scala 2.13 version. Scala 2.13 macro implementations should then be isolated to a library with a namespace that does not clash with the Scala 3 implementations, and be provided on the same classpath. Or the tooling can be adapted to provide both implementations in a single jar with jar versioning and the compiler selects the correct partition to search for implementations (how does this work with plain directory dependencies)
Extra burden is placed onto library maintainers to duplicate definitions of macros in the Scala 3.0 version
Milestone 2: Support arbitrary definitions in annotations in the cross compatible subset of Scala 2.13/3.0
Annotations can embed arbitrary trees, which may be used for metaprogramming, and it is an arbitrary restriction to stop at simple expressions.
Enable unpickling definitions to provide trees, not just symbols.
It is not yet known how much would have to change to support this feature, e.g. Dotty compiler relies on lazy reading of class/method bodies, which is not possible in the Scala 2 compiler.
Milestone 3: Support Union types in Scala 2
Union Types allow more precision in an API than a simple least upper bound. In most cases it should be possible to approximate something compatible with Dotty
Edit: Repurpose a similar type to Scala.js Union in the frontend of Scala 2.13 and erase it to match the JVM signature produced by Dotty.
So far Scala.js Union can not derive all constraints necessary for full equivalence with Scala 3.0. Is it possible to adapt it? Are the restrictions good enough?
Maintenance Goal: Remain compatible with TASTy format changes
With each Dotty public release, there may be changes to the TASTy format, codified by an explicit version change. Changes to the format may be made for many reasons, including changing the representation of an existing feature, with no effect on semantics. We need to maintain compatibility by adopting changes to the TASTy format.