Scala3-migrate: a tool making it easier to migrate to Scala 3

Scala 3 Release Projects
1

(The Scala Org aims to release Scala 3 by the end of fall 2020. We are about 15 employees (some of whom work part-time), spread in 4 organisations (+ active community members), focusing on finalising 52 essential projects in 6 months. As of today, project leads will publish the road-maps under the category “Scala 3 release projects” to share with you what is to be expected and hopefully get your advice & contributions as well. All the projects’ road-maps come after an extensive feedback gathering, rounds of discussion, and involvement of major stakeholders, we now need the community to help push this effort over the line. Your collaboration is highly appreciated, thank you in advance!)

Main goal

The Dotty compiler will be the main tool for syntactic rewrite to scala 3. Where scalafix would make a huge difference is if it could make everything that’s inferred by scalac explicit:

  • Inferred types of all fields and inferred result types of all methods (provided mostly in an existing rule: Explicit Result Types)
  • Inferred type arguments
  • Inferred implicit parameters
  • Inserted implicit conversions

The goal is to provide a tool that will find the minimum set of types required explicitly to make dotty compiling a project without changing its meaning.

We believe that Scalafix provides the necessary infrastructure to enable this project. It’s widely used and tested among the community projects. This tool targets projects that aim to cross-compile in scala 2.13 and 3.0. Other steps may be necessary to fully migrate to scala 3 (taking advantage of features not available in Scala 2), and those will be covered by the migration guide.

Technical solution

The solution consists of 3 independent steps that will be packaged in a long-running script The compile inputs that are needed to compile both with scala 2 and dotty, will be provided by a sbt-plugin that we will need to develop.

First step: Make sure that all methods/functions of the public API have an explicit result type. If not, we require to run Explicit ResultType rule, which will type explicitly public and protected methods. Those types will be kept even after scala 3 migration. This is a good practice anyway, even in the context of a single Scala version, so that the APIs do not change based on the whims of the compiler’s type inference. It will ensure that the following steps do not alter the public API.

Second step: Find the minimum set of types required explicitly to make dotty compiling a project. The solution will be responsible for:

  1. Adding type annotations showing the types inferred by Scala 2 to the rest of the codebase. This rule should return a list of patches including the changes. This will be implemented as a Scalafix rule, technically similar to ExplicitResultTypes but covering many more subexpressions and type arguments.
  2. If adding only inferred types does not allow compilation to succeed, we will add resolved implicit params and conversions using another scalafix rule.
  3. Compiling with dotty (with -source:3.0-migration) after applying the patches on memory.
  4. Following a dichotomic algorithm to remove half of the type annotations (ie patches) and testing if dotty can still compile the project or not.
  5. Repeating steps 2 and 3, until we find the minimum set of types required for compilation.

At this point, we have a codebase that a) compiles with Dotty and b) preserves the public APIs (because of step 1). However, we have no guarantee that it preserves the semantics of the bodies of the methods, as different terms can be inferred due to different inferred types and/or implicit resolution rules. This is addressed by the third step.

Third step: Compare synthetics (i.e., compiler-generated code) and more precisely implicit parameters and implicit conversions inferred in scala 2 and dotty, and produce a report. This step will be more detailed as we discover it, but will mainly rely on SemantiDB.

Possible Optimization

  • After the first step, we could also infer types for private packages and vals. This way we only need to add types inside functions/methods and therefore we can use incremental compilation instead of the full compilation
  • Adding explicit types directly on the tree level to avoid parsing each time source files

Milestones

Milestone 1: Validate the technical solution [Done]
Milestone 2: Develop a scalafix rule to add type annotations for inferred types [Done]
Milestone 3: Develop the dichotomic approach to remove added type annotations [Done]
Milestone 4: Develop a scalafix rule to add resolved implicit params and conversions [Done]
Milestone 5: Develop the sbt-plugin by the end of November [Done]
Milestone 6: Develop the production of a report based on the comparison of synthetics between dotty and scala 2 [Won’t do]
Milestone 7: Testing on real projects and optimizations [In progress]

16 Likes
2

TL DR; Scala3-migrate, a tool to make easier the migration to Scala 3, releases its first version 0.3.1.
We would love to hear your opinion, feedback is welcome!

Scala3-migrate: First release

Scala3-migrate tool is part of a series of initiatives to make the migration to Scala 3 as easy as possible.
The project formerly known as The Scala 2 to Scala 3 typer and implicit resolver has adopted its new name scala3-migrate in early September 2020. We have been working on it actively for the last months, and we are happy to share with you its first release.

The tool is still under development, and we would love to hear from you. Every feedback will help us to build a better tool: typos, clearer log messages, better documentation, bug reports, ideas of features, so please open GitHub issues or comment this contributor post.

1917×880

Updates

The scope of this tool has been extended to cover more aspects of the migration: migrating dependencies and scalacOptions.

It proposes an incremental approach that can be described as follows:

  • Migrating the library dependencies: using Coursier, it checks, for every library dependency, if there are versions available for Scala 3.
  • Migrating the Scala compiler options (scalacOptions): some compiler options of Scala 2 have been removed in Scala 3, others have been renamed, and some remain the same. This step helps you find how to evolve the compiler options of your project.
  • Migrating the syntax: this step relies on Scalafix and on existing rules to fix deprecated
    syntax that no longer compiles in Scala 3
  • Migrating the code: Scala 3 has a new type inference algorithm that may infer a different type than the one inferred by the Scala 2 compiler. This last step tries to find the minimum set of types to explicitly annotate in order to make the Scala 3 compiler work on a project without changing its meaning.

How to use it

Requirements

  • scala 2.13, preferred 2.13.5
  • sbt 1.4 or higher
  • Disclaimer: This tool cannot migrate libraries containing macros.

Installation

Currently, you can use scala3-migrate via an sbt plugin. You can add it as follows to your build.

// project/plugins.sbt
addSbtPlugin("ch.epfl.scala" % "sbt-scala3-migrate" % "0.4.0")
// sbt-dotty is not required since sbt 1.5.0-M1
addSbtPlugin("ch.epfl.lamp" % "sbt-dotty" % "0.5.3")

4 commands are provided that operate on one module at a time:

  • migrate-libs projectID: helps you update the list of libraryDependencies
  • migrate-scalacOptions projectID: helps you update the list scalacOptions
  • migrate-syntax projectID: fixes a number of syntax incompatibilities in scala 2.13 code
  • migrate projectID: tries compiling your code in scala 3 by adding the minimum required inferred types and implicits.

For detailed documentation regarding scala3-migrate usage, refer to the migration guide in Scala3-migrate section.

20 Likes
3

great initiative! - thanks for that! If I have a simple scala file (no complex project,…) written in Scala 2 - is there any possibility to rewrite (syntax) to Scala 3? (e.g. I have hello world scala 2 file - how can I rewrite to Scala 3)

thank you

4

Thanks!
scala3-migrate has two commands to rewrite syntax, and to migrate to scala 3: migrate-syntax root and migrate root.
Once done successfully, you will be able to update the scalaVersion of your build:

ThisBuild  /  scalaVersion := "3.0.0-RC1"

If you have any issue, please report it here Issues · scalacenter/scala3-migrate · GitHub.
Thanks

1 Like
5

Scala3-migrate Updates

Hello,

We are happy to announce the release of 0.4.5 of scala3-migrate.
This release brings a number of improvements:

  • Better support for scala.js: now we are able to check for Scala.js dependencies if there are available versions for Scala 3 and then display them with the right the syntax to add them to the build.
  • Use of the new syntax introduced with sbt 1.5.x: migrate-libs command uses now the new syntax introduced in 1.5.x to display how to update dependencies that requires CrossVersion.for3use2_13 or CrossVersion.for2_13use3
  • Ability to migrate other configurations including custom configurations: automatically, when migrating a module/project, Compile and Test configuration will be migrated along with any other custom configurations you would have defined.

To update or use the plugin, you will need sbt 1.5.x, and then add or update your plugins list.

// project/plugins.sbt
addSbtPlugin("ch.epfl.scala" % "sbt-scala3-migrate" % "0.4.5")

Thanks for all reported issues, and don’t hesitate to reach us on github for any question or remark.

5 Likes