The Scala 2 to Scala 3 typer and implicit resolver

(The Scala Org aims to release Scala 3 by the end of fall 2020. We are about 15 employees (some of whom work part-time), spread in 4 organisations (+ active community members), focusing on finalising 52 essential projects in 6 months. As of today, project leads will publish the road-maps under the category “Scala 3 release projects” to share with you what is to be expected and hopefully get your advice & contributions as well. All the projects’ road-maps come after an extensive feedback gathering, rounds of discussion, and involvement of major stakeholders, we now need the community to help push this effort over the line. Your collaboration is highly appreciated, thank you in advance!)

Main goal

The Dotty compiler will be the main tool for syntactic rewrite to scala 3. Where scalafix would make a huge difference is if it could make everything that’s inferred by scalac explicit:

  • Inferred types of all fields and inferred result types of all methods (provided mostly in an existing rule: Explicit Result Types)
  • Inferred type arguments
  • Inferred implicit parameters
  • Inserted implicit conversions

The goal is to provide a tool that will find the minimum set of types required explicitly to make dotty compiling a project without changing its meaning.

We believe that Scalafix provides the necessary infrastructure to enable this project. It’s widely used and tested among the community projects. This tool targets projects that aim to cross-compile in scala 2.13 and 3.0. Other steps may be necessary to fully migrate to scala 3 (taking advantage of features not available in Scala 2), and those will be covered by the migration guide.

Technical solution

The solution consists of 3 independent steps that will be packaged in a long-running script The compile inputs that are needed to compile both with scala 2 and dotty, will be provided by a sbt-plugin that we will need to develop.

First step: Make sure that all methods/functions of the public API have an explicit result type. If not, we require to run Explicit ResultType rule, which will type explicitly public and protected methods. Those types will be kept even after scala 3 migration. This is a good practice anyway, even in the context of a single Scala version, so that the APIs do not change based on the whims of the compiler’s type inference. It will ensure that the following steps do not alter the public API.

Second step: Find the minimum set of types required explicitly to make dotty compiling a project. The solution will be responsible for:

  1. Adding type annotations showing the types inferred by Scala 2 to the rest of the codebase. This rule should return a list of patches including the changes. This will be implemented as a Scalafix rule, technically similar to ExplicitResultTypes but covering many more subexpressions and type arguments.
  2. If adding only inferred types does not allow compilation to succeed, we will add resolved implicit params and conversions using another scalafix rule.
  3. Compiling with dotty (with -source:3.0-migration) after applying the patches on memory.
  4. Following a dichotomic algorithm to remove half of the type annotations (ie patches) and testing if dotty can still compile the project or not.
  5. Repeating steps 2 and 3, until we find the minimum set of types required for compilation.

At this point, we have a codebase that a) compiles with Dotty and b) preserves the public APIs (because of step 1). However, we have no guarantee that it preserves the semantics of the bodies of the methods, as different terms can be inferred due to different inferred types and/or implicit resolution rules. This is addressed by the third step.

Third step: Compare synthetics (i.e., compiler-generated code) and more precisely implicit parameters and implicit conversions inferred in scala 2 and dotty, and produce a report. This step will be more detailed as we discover it, but will mainly rely on SemantiDB.

Possible Optimization

  • After the first step, we could also infer types for private packages and vals. This way we only need to add types inside functions/methods and therefore we can use incremental compilation instead of the full compilation
  • Adding explicit types directly on the tree level to avoid parsing each time source files

Milestones

Milestone 1: Validate the technical solution [Done]
Milestone 2: Develop a scalafix rule to add type annotations for inferred types [Done]
Milestone 3: Develop the dichotomic approach to remove added type annotations [Done]
Milestone 4: Develop a scalafix rule to add resolved implicit params and conversions [In progress]
Milestone 5: Develop the sbt-plugin by the end of November
Milestone 6: Develop the production of a report based on the comparison of synthetics between dotty and scala 2
Milestone 7: Testing on real projects and optimizations

13 Likes