Preparing for Feature Freeze

Language Design
1

Winter is coming :snowman:

Next week we plan to release Dotty 0.20. The plan is that this will be the last release before we go into feature freeze. Assuming our normal release cadence, this means we have another 6 weeks in which features to add or to drop can be proposed.

What does feature freeze mean?

Feature freeze means that new proposed changes should be deferred to a version after 3.0. It does not mean that the current feature set is cast in stone. Many of the language changes incorporated in Dotty still have to go through the SIP process, which includes a community review period. Depending on the feedback received and the committee’s votes, a feature might be refined, deferred, or dropped altogether. So, feature freeze means that from now on the process concentrates on “refine and reduce” instead of “expand and explore”.

Which late proposals can be considered?

In the next six weeks we can still look at proposals. However, to have a chance of being accepted, a proposal needs to be either easy to implement or already be far along in its implementation. Furthermore, it should fall into one of two categories:

  • A restriction that makes Scala simpler or safer. Restrictions are much harder to do than additions. That’s why we should profit from the version jump from 2 to 3 to enable them. However, a restriction cannot force a rewrite of significant amounts of existing code, unless the rewrite can be done automatically or there is some other clear and feasible migration strategy.

  • An addition that makes Scala simpler to learn and to use for newcomers. Typically, a feature that would be in every intro tutorial for the language.

Other proposals are of course possible, but they will be considered for a version after Scala 3.0.

How to propose a late change?

You can post here on the thread with the rough idea. For submitting a worked out proposal, use Dotty’s issue tracker.

Anything still in the pipeline?

There are some in-flight Dotty PR’s that might still make it. In particular, we are taking a close look at null-safety.

6 Likes
2

What about the 14 currently open SIPs? Do they all get deferred to 3.1 at the earliest?

3

Strong :+1: here – IMO, this is one of the most broadly-applicable wins among the proposed features for 3.0, so I’d love to see it get in.

11 Likes
4

We need to take a pass at that list, which looks in part outdated. Several SIPs on the list do have implementations in Dotty, so I believe we can mark them completed. If others should still be considered, it would be good to bring them up now. E.g. Quote Escapes for Interpolations looks like a candidate fitting the criteria. We’d need a PR for this, though.

5

Thanks for the update! Really excited about getting closer to a shipped Scala 3 :-).

I am curious about where we can follow along with the results of the SIP process on a feature-by-feature basis. I found the SIP list but, as mentioned by Martin, it looks pretty outdated. Shouldn’t we be seeing an SIP in that list for each proposed Scala 3 feature?

I believe many Scala 3 features have already been discussed by the SIP Committee (for example, these meeting scheduling announcements), but without watching each meeting recording it’s hard to know what the end results were. There also doesn’t seem to be up-to-date minutes published here.

Thank you all for your hard work! I know it’s a lot to manage.

6

The SIP list is indeed lagging behind. I believe the standard SIP process, where features are defined, and implemented in isolation does not work well this time. All new features are already implemented and documented in the Dotty codebase. What we plan to do is that instead of force-fitting them into the standard SIP framework we will add status information to the Dotty reference pages.

7

Several SIPs on the list do have implementations in Dotty, so I believe we can mark them completed.

That effectively means that you can bypass the SIP process by implementing in Dotty. If that’s indeed the right way to go, then it’s questionable why other features that are not merged in dotty do need to go through the full SIP process.

If others should still be considered, it would be good to bring them up now.

Yes, I’m bringing them up now. I think the others should still be considered.

E.g. Quote Escapes for Interpolations looks like a candidate fitting the criteria. We’d need a PR for this, though.

That’s the most trivial SIP of the lot, and spending much time on it is IMO akin to spending a lot of time to trailing commas. The implementation is a small modification to a single line of code. A PR is linked in the SIP proposal.

I’m not sure where we are exactly in the process for that SIP. I think one step before formal presentation. The next steps would be that the process lead proposes to discuss the SIP at the next SIP meeting. That SIP meeting would then decide whether the SIP is numbered or not, and an issue to decide when to disucss the proposal itself is opened, and assign a reviewer.

Then, in the next SIP meeting, the reviewer will formally present the numbered SIP. The SIP committee can then decide what they want to do with the SIP.

That process strikes me as overly bureaucratic, but that’s the process the SIP committee has decided on. Changing that process may be a good idea, but basing it on exceptions probably isn’t.

Apart from the mentioned SIP, there are 5 pending SIPs from 2017, 2 from 2018, and 2 from this year. Does the SIP committee really want to say these proposals are too late now? If so, when should they have been proposed?

2 Likes
8

I find that regrettable, as it hurts the credibility of the SIP process.

3 Likes
9

Note that a lot of features currently in Dotty are extremely controversial, and Martin put them into the codebase under the promise that the features will go through a SIP before going into Scala 3. If they do not go through a proper SIP, for whatever reason, it really hurts the credibility of much more than just the SIP process.

9 Likes
10

Any chance to complete feature request #15 - Make the Scala runtime independent of the standard library? This would be a huge win for Scala.js, interop with the other JVM languages and maybe also performance.

I see that IArray has been added a while ago, which is a first step towards this feature.

1 Like
11

The feature freeze and the SIP process relates to language features, not the tooling. So, technically that change could be still done even after the feature freeze. That said, I am not sure who would have the time to put in the effort to make it for 3.0 final.

1 Like
12

And I have stated again in the beginning of this thread that they will go through the SIP process, which means that they go through public review and the SIP committee will vote on them. The SIP committee has also agreed that the mechanics how these proposal are treated will have to be streamlined in order to do this in reasonable time and effort.

The issue is that we started from a very unusual point. The standard SIP procedure (in very abbreviated form) is that someone makes a proposal for a single improvement, the proposal is documented and implemented in the Scala compiler, and then the SIP committee votes on it. But that’s not how Dotty happened. Dotty was its own language and code base with no relationship to the SIP process. About 18 months ago there was a collective decision of the relevant contributors of Scala 2 and Dotty that this codebase and feature set would become the basis for Scala 3. Another question was whether the SIP committee, which was up to that time only concerned with the evolution of Scala 2.x, should get involved with Scala 3 also. After some deliberation we decided that the answer should be yes, but that the process would have to be adapted to account for where this is coming from. Unlike with normal SIP features, everything is already implemented in one codebase and documented in one repository. Since that repository already exists, we will take it as the basis for tracking the state of the approval processes.

Concretely, each topic in the Dotty reference is treated like a separate SIP proposal. It is discussed by the SIP committee and if it passes a first round of review it will be submitted for public review. Sometimes major refinements will be needed (as in the implicits proposals) in which case the process can repeat. At the end, the SIP committee will vote on the feature. For better transparency, we are about to add status information about the current state of the approval process to each of these pages.

We will have to evaluate whether we also want to use a different scheme for proposals by others. It seems the scheme we currently have could be improved. I take note that the SIP committee should make a better effort to keep the set of pending SIPs up-to-date and moving along.

If one of the currently open SIPs should still be considered for 3.0, we would need to see a pull request against the Dotty codebase. The PR should contain a doc page, in the format of the Dotty language reference. We will make sure that the SIP committee considers any proposals that are updated in this way.

One final note of clarification: The SIP process decides what goes into the Scala language in general. It is not directly concerned with what improvement goes into which release. That’s for the maintainers of the released compiler version to decide.

4 Likes
13

I’m a bit confused. If a feature in Dotty will not be approved by the SIP committee before the feature freeze, will it be removed?

14

I think that’s clarified by my initial post on this thread:

Feature freeze means that new proposed changes should be deferred to a version after 3.0. It does not mean that the current feature set is cast in stone. Many of the language changes incorporated in Dotty still have to go through the SIP process, which includes a community review period. Depending on the feedback received and the committee’s votes, a feature might be refined, deferred, or dropped altogether. So, feature freeze means that from now on the process concentrates on “refine and reduce” instead of “expand and explore”.

1 Like
15

I hope that I’m misrepresenting you when I summarize this as “Thanks for writing a proposal back in 2017 but if you don’t resubmit it on a new template, and also implement your proposal in dotty then we aren’t going to even consider it, and you need to hurry up, because we’re not going to wait on you”.

Please say that I am and that I got it all wrong.

1 Like
16

I think the problem is that this was not really communicated, and as a result a lot of people are confused (probably including me). “The process is changing” is an important thing, at least as much as we hope that people care about the process. “We need a pull request against the Dotty codebase for any SIP to be considered” is a change that’s worth bringing up explicitly, separate from “here are all the changes we have in Dotty we would like to consider”.

I know times change and circumstances change, but the changing rules of the game is always a bit of delicate matter. Maybe for pragmatic reasons we have to limit Scala Language Improvements to the O(dozen) people who are able to contribute to one specific experimental codebase, but that hasn’t been what we’ve been told for the past few years. I was under the assumption that anyone could write a detailed language proposal and have a fair consideration.

Even if the change has no immediate impact, the fact that the process changed is a Big Deal to anyone who cares about the process. And I hope we all agree that people caring about the process of improving the Scala language is a good thing!

2 Likes
17

I think the problem is that this was not really communicated, and as a result a lot of people are confused (probably including me). “The process is changing” is an important thing, at least as much as we hope that people care about the process. “We need a pull request against the Dotty codebase for any SIP to be considered” is a change that’s worth bringing up explicitly, separate from “here are all the changes we have in Dotty we would like to consider”.

Yes, this should have been communicated better. But the fact is, we always required a full implementation of a feature in the current codebase. It could be done by the proposer or taken up by somebody else who supports the proposal and is willing to put in the effort. So, the requirement has not changed, but the relevant code base has changed. We should have communicated that earlier.

Since we are late with this, I suggest the following: if there is a proposal in the current SIP queue where the proposers (or somebody else) are willing to reimplement it in the Dotty codebase but need a bit more time than the remaining six weeks to do so, they should talk to us (before the freeze deadline). We might want to still consider the feature for integration afterwards.

2 Likes
18

Thanks for the clarification. There is some ambiguity in the phrasing “might be […] dropped”, but I take this to mean, yes, if it is not approved, it will be dropped for sure.

19

Which proposal are we talking about? Let’s not generalize prematurely, but talk about concrete cases. I am sure we can find a reasonable solution for each concrete case.

20

Hi Everyone,

As the SIP process lead in the last year, I feel it is my duty to clear our a couple of things.

  1. I take full responsibility for not updating the list(s). I need help, whoever has some time to invest to work with me to put up the missing minutes and make sure all updates are in order, it would mean a lot to me. @dwijnand joined SIP to help with upcoming meetings (big thanks!).

Here are the reasons I haven’t been able to keep up: at first I was only in charge of scheduling and minutes, then I picked up the lead role in SIP and organized Scala Days, and many other responsibilities came up. There are too many moving parts with all new features and events that happen, that it got increasingly harder as one person to keep up with the updates.

  1. If you are interested in the SIP committee role in Scala 3, please check the minutes from May 2019 and November 2018.

In short, the SIP committee is working closely on steering the Scala 3 release - but we do need to create the path as we walk on it.

  1. We should go over the SIPs that are pending.

  2. We agreed that some instances in the process itself, regarding proposing and stages, need to be revised. But we didn’t make final changes and notified the public. This happened 2 days ago, when we had a SIP meeting to discuss many organizational points (how we organize ourselves, who can invest how much time, what makes sense and so forth).

In conclusion, the Dotty team decided to go in the feature freeze soon. This will allow the SIP Committee to know what we are dealing with, no more chasing features. The next public meeting will be the end of November (still waiting on the exact date), another one before Christmas, retreat in February or March.

I hope this helps better understanding of where we stand with SIP and Dotty relations.

5 Likes