Status of Rewrite Tool

Language Design
1

Dotty has a number of breaking changes, as well as deprecating many patterns in favor of new language features. This creates a huge amount of technical debt for users, which we’ve been told is solved by a rewriting tool.

What is the status of this tool? How many rewrites does it already do and how many does it not do yet, and how much real-world code has it been tested on?

Also, is there a way to test that even if the rewritten code compiles, that it still means the same thing?

Is the UX polished or is there a ways to go from being a tool that works to a tool that works seamlessly enough that people won’t see it as a burden?

What is the roadmap and timeline for it to be fully production-ready?

8 Likes
2

I will talk about the importance of this tool, the importance of integrating it into sbt, how it should be viewed as a critical path item, and the lack of a detailed road map for Scala 3, on Nov 27 at the Montreal Lambda Meetup. All relevant information that I learn before the presentation will be incorporated.

1 Like
3

I think the anxiety caused by such a large number of changes would be considerably reduced (nearly eliminated) if they had been done with a CI system that automatically applies a rewrite to the scala community build and then runs the latest dotty compiler on the code.

I know dotty might not have been able to (perhaps is still not able to) compile all that code, but I think demonstrating the tool rather than a “fix it in post” approach would be a relief.

The two large scala code bases I am most familiar with (Twitter’s and Stripe’s) are difficult to upgrade even in scala minor versions. e.g. Stripe is still on 2.11, and Twitter only recently adopted 2.12. If 2.11 => 2.12 is that hard for large codebases, I don’t have a lot of confidence 2.x => 3.0 will be as easy.

Indeed, I fear there is still a risk of a python 3 situation since even if tooling is created, how will it integrate with large codebases and for instance at Databricks who make arguably the most important scala product, spark. Most of these larger code bases use custom build tools, so “here’s an sbt plugin” isn’t going to cut it.

4 Likes
4

Nice idea, but the core of the build is too dependent on Scala 2 macros. A rewrite is impossible.

2 Likes
5

I think @oscar makes a good point that having a nice CI system implemented would have been helpful. I know CI systems like Jenkins and SonarQube have plugins that integrate nicely with building Scala projects. :slight_smile:

6

Hello,

Thanks for raising the issue of the rewrite tool.

The tool itself exists, it’s scalafix, which is a stable product. However, so far very few rules have been written for the Scala 3 migration.

scalafix has already been demonstrated to be successful in the real-world. Two prominent examples are:

I think those two examples validate that the tool works seamlessly enough.

Regarding the specific rules for the Scala 3 migration, this is now a priority at the Scala Center for the upcoming months.

7 Likes
7

I think when people inquire about the state of the Scala 2 to 3 rewrite tool, they mean the state of the rule set, not the state of scalafix.

4 Likes
8

Implicitly you can infer they don’t exist.

1 Like
9

Right, I got a few ScalaTest 3.1.0 scalafix migrations from Scala Steward. For some reason at the time I opened this thread it may have slipped my mind (that was before I got the scalatest pull requests).

However as @curoli said, my question was mostly about the ability use it to migrate from Scala 2 to Dotty, because that is the huge elephant in the room. Until those rules are demonstrated to be definable and to work on massive codebases, the premise that Dotty hinges on has not been proven. We can be optimistic, but it is a necessary premise that so far has not been actually proven. And until it’s proven, we can bet but we can’t be totally certain that such big language changes are viable.

Regarding UX, they should publish coursier-bootstrapped jars, I think that would make for a reasonable experience (right now https://scalacenter.github.io/scalafix/docs/users/installation.html looks fine for anyone investing in using scalafix, not for someone who needs convincing to even try Dotty in the first place).

With that out of the way let me restate the remaining questions:

  1. How many rewrite rules are implemented already?
  2. How many need to be implemented (total, or total minus #1)
  3. How much real-world code will it be tested on?
  4. Is there a way to test that even if the rewritten code compiles, that it still means the same thing?
  5. What is the roadmap and timeline for it to be fully production-ready?
6 Likes
10

I’ve pinged @fthomas on this privately and I know it’s not possible because sbt is simply too open and there are probably too many project variations out there but I think one step which might start take people forward is by adding a scalafix rule to add Dotty as a cross compile target.
I know it won’t work for 100% of the projects but it might at least give people the incentive to take a look as to why it’s not (the “we should have engaged early” lesson of every retrospective).
Anything we can do to accelerate the ecosystem today will help avoid the long tail of upgrades down the line. Imagine we get stuck in a Java 8 state of affairs and people target only 2.13 years after Scala 3 has been released. That would be a bummer (though I doubt it, my subjective feeling is that Scala libs are mostly still actively maintained and pruned probably because we’ve gone through this with 2.12 and 2.13 upgrade cycles).

1 Like