Roadmap for SCP-021: Zinc Improvements

(The Scala Center team is dedicated to providing regular and transparent community updates about project plans & progress. In this forum we created a new topic category to allow quicker orientation going forward: “Scala Center Updates”. Even though all feedback is welcome, we keep the right to make executive decisions about projects we lead. Overview of all our activities can always be found at https://scala.epfl.ch/records.html)

Dear Scala Contributors,

In December 2019, the Scala Center accepted the proposal SCP-021: Zinc Improvements. The primary goal of this project is to improve the incremental compilation experience of Scala by having a closer integration between Zinc, the incremental compiler service for Scala build tools, and the core Scala Compiler. We want to ensure that Zinc can continue to evolve to support features of Scala 3.

In collaboration with Lightbend, we propose that after this project, Zinc will still control the implementation of recompilation of sources, but allow the compiler more direct control over how Zinc receives the input it requires to make decisions. This could enable efficiency improvements as well as more accurate recompilation.

So far since December, we have made a prototype of an event based framework for exporting the API of code to Zinc, and until September we will be working towards the following goals:

Short Term:

Milestone 1: Dependencies phase redesign

What

Design an architecture for reimplementing the dependencies phase in terms of events, that can then be called within a single traversal.

Why

The dependencies phase allows the usage of an API to be traced so that a change to an API triggers recompilation of its usages. The issue is that it currently happens in a single traversal, which may hide temporal changes that are not affected by signature changes within a public API

How

Redesign dependencies phase in terms of calls to an event-based API that could be invoked anywhere, rather than only within a single traversal.

Impact

Zinc does not always get recompilation right, sometimes it does not recompile necessary sources (under-compilation), leading to errors at runtime, and sometimes it compiles more than it needs to (over-compilation), leading to longer build times. By allowing finer control, we hope to reduce both of these phenomena.

Milestone 2: Implement new phases private to the compilers

What

Implement in Dotty and Scala 2 the redesigned dependency and api analysis phases

How

Modules within Dotty or Scala 2 will contain the implementation of new API’s that then delegate to the current zinc callbacks. Ideally no observable changes occur from Zinc’s side. Tests should evaluate that current state is maintained.

Impact

We can see how the new phases compare to the status quo. The individual compilers will also be free to either leave the phases as-is or use the new dependency API’s elsewhere to prevent under compilation.

Long Term:

Milestone 3: Integrate implementation of new APIs with Zinc

What

Zinc adopts the API changes as its new public API, and provides an implementation.

How

Remove the private API implementation modules from both Dotty and scala 2 versions and reimplement them as a single API in Zinc itself, Zinc now exposes the new API and deprecates the old callbacks.

Impact

Reduction of maintenance burden

7 Likes