Scala 3 Migration Guide (continuation)

Scala 3 Release Projects
1

(The Scala Org aims to release Scala 3 by the end of fall 2020. We are about 15 employees (some of whom work part-time), spread in 4 organisations (+ active community members), focusing on finalising 52 essential projects in 6 months. As of today, project leads will publish the road-maps under the category “Scala 3 release projects” to share with you what is to be expected and hopefully get your advice & contributions as well. All the projects’ road-maps come after an extensive feedback gathering, rounds of discussion, and involvement of major stakeholders, we now need the community to help push this effort over the line. Your collaboration is highly appreciated, thank you in advance!)

The Scala Center team is coordinating the efforts with many stakeholders of the community in creating a well-rounded Scala 3 migration guide, mostly targeted at library authors, application maintainers and developers that have interest in one of these:

  • cross-compiling and cross-publishing a library to Scala 2.13 and 3.0, and possibly older versions as well
  • upgrading a module or an entire application to benefit from the new features of Scala 3
  • acquiring knowledge about the evolution and compatibility between Scala 2 and Scala 3

This work is a continuation of this Scala Center project that has been leveled up to the Scala 3 release context. It aims to provide the motivation to undertake the migration, along with the theoretical knowledge and the practical information that will make it a success.

Contributions are very welcome! At any moment you can get in touch with me to take part in the effort.

The migration guide is scheduled to be released by fall 2020 and it will keep evolving beyond that. The following milestones describe the work.

M1 - Dotty rewrite rules documented :white_check_mark:

The Dotty compiler provides a rewrite capacity that covers a fair part of the language changes and most occurrences of incompatibilities. It will efficiently support the migration of many codebases, those which make moderate use of advanced type features, implicits and metaprogramming.

The usage of this compiler tool and most of its rules are already documented here. We will complement further the documentation of the rewrite rules.

M2 - Community knowledge transferred :white_check_mark:

The Dotty community build contains a number of well-known Scala 2 libraries that have been ported to Scala 3. From this centralized source of knowledge we will extract a lot of valuable information:

  • A list of incompatibilities and possible solutions
  • A first draft of the migration procedure on sbt and mill
  • A draft strategy for dealing with metaprogramming code

Many of you have already started porting some code to Scala 3, or you will soon do. We need your help to complete the list of incompatibilities we are building up in the migration guide repository. You are very much encouraged to report them here.

M3 - Compatibility reference published :white_check_mark:

Scala 3 is a game changer in terms of compatibility in the Scala ecosystem. It will improve the day to day experience of the Scala programmers, starting from the migration. We will write a reference page that details the level of compatibility at the language level, at compile-time and at runtime and explain how it relates to the migration. It will cover the Tasty reader, the Scala 2 unpickler, the shared ABI, the shared standard library and it will be illustrated by diagrams of dependencies between modules or libraries.

M4 - Early adopter company partnered with Scala Center :white_check_mark:

The Scala Center will partner with companies that are willing to migrate to Scala 3. They will beta-test the migration guide and provide feedback that will help us improve it. If you are interested, please let us know.

M5 - Metaprogramming migration guide initiated

The metaprogramming redesign in Scala 3 is a breaking change that will prevent all macro defs and macro annotations to compile. The reason is mostly that the Scala 2 metaprogramming framework is tied to the compiler. Hopefully Scala 3 is bringing stability and compatibility to metaprogramming in Scala by leveraging on the Tasty format.

Scala 3 metaprogramming is a whole new topic. This guide will focus on a few guidelines and techniques to help you port your library while maintaining its API. The reimplementation in itself will require reading the Dotty documentation and understanding the new capabilities of the Dotty macros and other reflection based features.

M6 - Library tutorial published and tested

The library migration tutorial will guide you, step-by-step, towards the publishing of the Scala 3 compiled artifacts of your library. It will be based on sbt. A decision tree will help you choose the more suited migration strategy. Then it will cover sbt configuration, dependency management and rewrite tooling that will make you fix the incompatibilities, compile the code, audit the changes and eventually publish the artifacts.

Some hints to translate this tutorial to Mill and other build tools will be included where possible.

M7 - Application tutorial published and tested

The task of migrating an application is different from migrating a library. It is less concerned with cross-compilation and API compatibility, while focusing on maintaining the runtime behavior. Therefore it will have its dedicated tutorial. The first draft will be published at about the same time but it will be improved continuously after the release of Scala 3.

M8 - Motivation page published

The motivation page will constitute the introduction of the migration guide. It will enumerate the motivations for moving to Scala 3.

M9 - Migration guide integrated to website

The final milestone of this roadmap consists of the integration of the guide into the Scala website. The final delivery will consist of:

  • A motivating introduction
  • A Scala 2 to Scala 3 compatibility reference
  • A step by step tutorial for library authors
  • A step by step tutorial for application maintainers
  • A metaprogramming migration guide
21 Likes
2

My presentation on the community build is available here: Community Build - Google Präsentationen

2 Likes
3

I am happy to announce that the migration guide has its own temporary website, powered by mdoc and Docusaurus:

It features:

The Compatibility Reference

It describes the compatibility between Scala 2 and Scala 3 at different level of the program:

  • language level
  • compile time
  • runtime
  • metaprogramming

It is illustrated by real dependency patterns (in sbt).

The Dotty Migration Mode

A quick tour of dotc capabilities regarding the migration

The Incompatibility Table

A classification and description of the found incompatibilities between Scala 2.13 and Scala 3.0. The status of each incompatibility is comprised of:

  • does Scala 2.13 emit a feature or deprecation warning?
  • does Dotty provide a migration rewrite?
  • do we have a Scalafix rule?

Special thanks to @giabao for writing some of the Scalafix rules.

13 Likes
4

Call for contributions

Many of you are undertaking the migration from Scala 2 to Scala 3!
From 48 published libraries in Dotty in early July, we are now counting 87 of them!! :tada: :tada:

You can easily help us make a better migration guide!

Here is how:

1- You stumble upon an incompatibility:

If the answer is no, then please open an issue with a short description of the incompatibility and an example or link toward your code.

2- You are migrating the Scala 2 macros of your metaprogramming library

Open an issue containing the name of the library and the status of the migration (on-going, cross-compiled, cross-published, on hold…) to update the list of macro libraries.

3- You have succeeded in cross-building your library

Give us (in a reply) a link towards your migration commits/PRs so that we can learn from your work.

4- Anything you wish to know before starting the migration

Comment in this thread or open an issue.

Thank you all for your help :pray:

7 Likes
5

Here’s the scala-parallel-collections changes:

The only real brain-twister was “replace existentials with explicit type parameters”. The rest was pretty straightforward. Hopefully not too many codebases are affected as strongly by this as this codebase was.

As for “add @uncheckedVariance as needed to satisfy compiler”, it’s normal in the collections library to need @uncheckedVariance in order to abstract over mutable and immutable collections. Informally, it appeared to me that Dotty was probably right to require the annotation in more places than Scala 2 did, but I didn’t really dig into it.

The migration guide was helpful.

1 Like
6

“add return types” was due to “infer type from overridden member instead of RHS.”

Starting scala3 REPL...
scala> trait T { def x: Any }
// defined trait T

scala> class C extends T { def x = "" ; def y = x.length }  // ok in Scala 2
1 |class C extends T { def x = "" ; def y = x.length }
  |                                         ^^^^^^^^
  |                                       value length is not a member of Any

scala> class C extends T { def x: String = "" ; def y = x.length }
// defined class C
3 Likes