The Scala Center's roadmap for tasty-query

Scala Center Updates
1

The Scala Center team is dedicated to providing regular and transparent community updates about project plans & progress. In this forum we created a new topic category to allow quicker orientation going forward: “Scala Center Updates”. Even though all feedback is welcome, we keep the right to make executive decisions about projects we lead. Overview of all our activities can always be found at https://scala.epfl.ch/records.html .

tasty-query is a work-in-progress library by the Scala Center to read TASTy files, explore them, and ask semantic questions about them. It is an independent implementation that does not depend on the compiler. It is not usable yet. We are working on making it support the essential use cases of TASTy-MiMa, an upcoming tool “like MiMa, but for TASTy”.

Currently implemented

Thanks to the work of our former intern Katja Goltsova, the codebase can already read TASTy trees, and create symbols from them. It can read types in a “syntactic” form, but not yet with semantic information (like subtyping or type equivalence). It also has a good testing infrastructure.

In addition, we already have an open PR to create symbol information from Java .class files and Scala 2.13 so-called pickles.

Roadmap

Our roadmap for the next step is as follows.

Populate semantic type information

Now that we have syntactic information as well as symbols, the immediate next step is to populate symbols and trees with semantic type information. For Java and Scala 2, this is limited to public and protected APIs. For Scala 3 TASTy files, this also includes filling in type information for every single tree node, including in the bodies of methods. Few nodes directly store their type information inside the TASTy files, so it is up to tasty-query to fill them in, according to the Scala 3 type system.

We expect this phase to take about 1.5 months.

Semantic queries

To support the use cases of TASTy-MiMa, tasty-query will provide some critical semantic queries:

  • Are two types equivalent?
  • Is a type a subtype of another one?
  • Is a method overriding/implementing a method from a superclass/trait?

For a first implementation, we expect to leave out difficult corner cases from the subtyping test. Excluding those, we expect this phase to take about 1 month.

Prototype of TASTy-MiMa

Based on the features of tasty-query, we will implement a first prototype of TASTy-MiMa. TASTy-MiMa will be like MiMa, but for TASTy APIs. It will be able to check that a new version of a library is TASTy-compatible (as opposed to binary-compatible) with an older version.

Based on the experience gained with this prototype, we will further plan the future of tasty-query and TASTy-MiMa.

We expect to build this prototype in about a month.

Documentation

This “milestone” will happen in parallel with the technical developments. There are several areas to document:

  • The significance of TASTy as a compatibility medium, and why TASTy-MiMa will eventually become critical for the Scala ecosystem (like MiMa is today).
  • The API and use cases for tasty-query. We expect tasty-query to be used in other tools than TASTy-MiMa. For example, it could be used by static analysis tools who require accurate, semantic information. It could also be used by IDEs for advanced features.
  • The usage of TASTy-MiMa as a practical tool for library maintainers.
13 Likes
2

I have good news to share about tasty-query and TASTy-MiMa. We completed all the milestones mentioned above, and released a prototype of TASTy-MiMa.

Developing all the required features in tasty-query took more time than expected, but in the past few months, we made a lot of progress. We achieved enough coverage of the Scala type system to implement a meaningful prototype of TASTy-MiMa.

Relevant links

In particular, follow the link to tasty-mima for usage instructions in an sbt build.

Status and request for testing

We published a first prototype of TASTy-MiMa, along with an sbt plugin sbt-tasty-mima v0.1.2 v0.1.3. It is quite young, but you can already help by trying it out on your project. The readme of TASTy-MiMa contains essential usage instructions for sbt-tasty-mima.

If your public API exposes type refinements and/or match types, expect severe limitations. Otherwise, please file any issues you may encounter.

tasty-query is also used by other projects, including the debugger for Scala 3 in Metals, and a work-in-progress research project for a TASTy Interpreter.

Why not make this part of MiMa?

TASTy-MiMa and MiMa achieve different although related goals.
One can wonder why not merge them as a single tool?
There are several reasons for that, at least at this point:

  • Huge maturity gap: MiMa is very mature and should be used by every library that is serious about compatibility. TASTy-MiMa is still in its infancy and likely to break, so we cannot recommend using it yet.
  • Different underlying codebase: MiMa and TASTy-MiMa contain no code in common; their underlying model is vastly different (JVM type system versus Scala type system).
  • Different (although similar) user-level API: the set of problems reported by MiMa and TASTy-MiMa is not the same (because they don’t have the same underlying model at all), so even if they were exposed as a single sbt plugin, the configurations would have to be using separate keys. The only real key in common is the mimaPreviousArtifacts/tastyMiMaPreviousArtifacts.

Next steps

The next steps for this set of projects will be the following:

  • Enable TASTy-MiMa on real-world Scala 3 libraries
  • Identify and fix important bugs preventing the above
  • Expand the support of tasty-query for type refinements and match types
  • Provide a Mill plugin for TASTy-MiMa

How can you help?

  • Test sbt-tasty-mima on your library, if it does not expose type refinements nor match types in its public API. Report any issues you encounter.
  • If you are familiar with Mill plugins, contribute a Mill plugin for TASTy-MiMa. You will probably want to take inspiration from the sbt plugin of TASTy-MiMa as well the existing Mill plugin for MiMa.
16 Likes
3

Small update: we just published sbt-tasty-mima v0.2.0, which corresponds to tasty-mima v0.2.0.

Since the last update, we fixed a number of issues that we discovered while testing sbt-tasty-mima on some real-world libraries. In particular, thanks to @velvetbaldmime for trying it out on langoustine.

We now also support type refinements (of the form P { type T = ... } for example) as well as match types.

We are still looking for testers and early adopters, especially now that we have ironed out quite a few edges.

5 Likes