Preparing for Feature Freeze

Language Design
7

Several SIPs on the list do have implementations in Dotty, so I believe we can mark them completed.

That effectively means that you can bypass the SIP process by implementing in Dotty. If that’s indeed the right way to go, then it’s questionable why other features that are not merged in dotty do need to go through the full SIP process.

If others should still be considered, it would be good to bring them up now.

Yes, I’m bringing them up now. I think the others should still be considered.

E.g. Quote Escapes for Interpolations looks like a candidate fitting the criteria. We’d need a PR for this, though.

That’s the most trivial SIP of the lot, and spending much time on it is IMO akin to spending a lot of time to trailing commas. The implementation is a small modification to a single line of code. A PR is linked in the SIP proposal.

I’m not sure where we are exactly in the process for that SIP. I think one step before formal presentation. The next steps would be that the process lead proposes to discuss the SIP at the next SIP meeting. That SIP meeting would then decide whether the SIP is numbered or not, and an issue to decide when to disucss the proposal itself is opened, and assign a reviewer.

Then, in the next SIP meeting, the reviewer will formally present the numbered SIP. The SIP committee can then decide what they want to do with the SIP.

That process strikes me as overly bureaucratic, but that’s the process the SIP committee has decided on. Changing that process may be a good idea, but basing it on exceptions probably isn’t.

Apart from the mentioned SIP, there are 5 pending SIPs from 2017, 2 from 2018, and 2 from this year. Does the SIP committee really want to say these proposals are too late now? If so, when should they have been proposed?

2 Likes
8

I find that regrettable, as it hurts the credibility of the SIP process.

3 Likes
9

Note that a lot of features currently in Dotty are extremely controversial, and Martin put them into the codebase under the promise that the features will go through a SIP before going into Scala 3. If they do not go through a proper SIP, for whatever reason, it really hurts the credibility of much more than just the SIP process.

9 Likes
10

Any chance to complete feature request #15 - Make the Scala runtime independent of the standard library? This would be a huge win for Scala.js, interop with the other JVM languages and maybe also performance.

I see that IArray has been added a while ago, which is a first step towards this feature.

1 Like
11

The feature freeze and the SIP process relates to language features, not the tooling. So, technically that change could be still done even after the feature freeze. That said, I am not sure who would have the time to put in the effort to make it for 3.0 final.

1 Like
12

And I have stated again in the beginning of this thread that they will go through the SIP process, which means that they go through public review and the SIP committee will vote on them. The SIP committee has also agreed that the mechanics how these proposal are treated will have to be streamlined in order to do this in reasonable time and effort.

The issue is that we started from a very unusual point. The standard SIP procedure (in very abbreviated form) is that someone makes a proposal for a single improvement, the proposal is documented and implemented in the Scala compiler, and then the SIP committee votes on it. But that’s not how Dotty happened. Dotty was its own language and code base with no relationship to the SIP process. About 18 months ago there was a collective decision of the relevant contributors of Scala 2 and Dotty that this codebase and feature set would become the basis for Scala 3. Another question was whether the SIP committee, which was up to that time only concerned with the evolution of Scala 2.x, should get involved with Scala 3 also. After some deliberation we decided that the answer should be yes, but that the process would have to be adapted to account for where this is coming from. Unlike with normal SIP features, everything is already implemented in one codebase and documented in one repository. Since that repository already exists, we will take it as the basis for tracking the state of the approval processes.

Concretely, each topic in the Dotty reference is treated like a separate SIP proposal. It is discussed by the SIP committee and if it passes a first round of review it will be submitted for public review. Sometimes major refinements will be needed (as in the implicits proposals) in which case the process can repeat. At the end, the SIP committee will vote on the feature. For better transparency, we are about to add status information about the current state of the approval process to each of these pages.

We will have to evaluate whether we also want to use a different scheme for proposals by others. It seems the scheme we currently have could be improved. I take note that the SIP committee should make a better effort to keep the set of pending SIPs up-to-date and moving along.

If one of the currently open SIPs should still be considered for 3.0, we would need to see a pull request against the Dotty codebase. The PR should contain a doc page, in the format of the Dotty language reference. We will make sure that the SIP committee considers any proposals that are updated in this way.

One final note of clarification: The SIP process decides what goes into the Scala language in general. It is not directly concerned with what improvement goes into which release. That’s for the maintainers of the released compiler version to decide.

4 Likes
13

I’m a bit confused. If a feature in Dotty will not be approved by the SIP committee before the feature freeze, will it be removed?

14

I think that’s clarified by my initial post on this thread:

Feature freeze means that new proposed changes should be deferred to a version after 3.0. It does not mean that the current feature set is cast in stone. Many of the language changes incorporated in Dotty still have to go through the SIP process, which includes a community review period. Depending on the feedback received and the committee’s votes, a feature might be refined, deferred, or dropped altogether. So, feature freeze means that from now on the process concentrates on “refine and reduce” instead of “expand and explore”.

1 Like
15

I hope that I’m misrepresenting you when I summarize this as “Thanks for writing a proposal back in 2017 but if you don’t resubmit it on a new template, and also implement your proposal in dotty then we aren’t going to even consider it, and you need to hurry up, because we’re not going to wait on you”.

Please say that I am and that I got it all wrong.

1 Like
16

I think the problem is that this was not really communicated, and as a result a lot of people are confused (probably including me). “The process is changing” is an important thing, at least as much as we hope that people care about the process. “We need a pull request against the Dotty codebase for any SIP to be considered” is a change that’s worth bringing up explicitly, separate from “here are all the changes we have in Dotty we would like to consider”.

I know times change and circumstances change, but the changing rules of the game is always a bit of delicate matter. Maybe for pragmatic reasons we have to limit Scala Language Improvements to the O(dozen) people who are able to contribute to one specific experimental codebase, but that hasn’t been what we’ve been told for the past few years. I was under the assumption that anyone could write a detailed language proposal and have a fair consideration.

Even if the change has no immediate impact, the fact that the process changed is a Big Deal to anyone who cares about the process. And I hope we all agree that people caring about the process of improving the Scala language is a good thing!

2 Likes
17

I think the problem is that this was not really communicated, and as a result a lot of people are confused (probably including me). “The process is changing” is an important thing, at least as much as we hope that people care about the process. “We need a pull request against the Dotty codebase for any SIP to be considered” is a change that’s worth bringing up explicitly, separate from “here are all the changes we have in Dotty we would like to consider”.

Yes, this should have been communicated better. But the fact is, we always required a full implementation of a feature in the current codebase. It could be done by the proposer or taken up by somebody else who supports the proposal and is willing to put in the effort. So, the requirement has not changed, but the relevant code base has changed. We should have communicated that earlier.

Since we are late with this, I suggest the following: if there is a proposal in the current SIP queue where the proposers (or somebody else) are willing to reimplement it in the Dotty codebase but need a bit more time than the remaining six weeks to do so, they should talk to us (before the freeze deadline). We might want to still consider the feature for integration afterwards.

2 Likes
18

Thanks for the clarification. There is some ambiguity in the phrasing “might be […] dropped”, but I take this to mean, yes, if it is not approved, it will be dropped for sure.

19

Which proposal are we talking about? Let’s not generalize prematurely, but talk about concrete cases. I am sure we can find a reasonable solution for each concrete case.

20

Hi Everyone,

As the SIP process lead in the last year, I feel it is my duty to clear our a couple of things.

  1. I take full responsibility for not updating the list(s). I need help, whoever has some time to invest to work with me to put up the missing minutes and make sure all updates are in order, it would mean a lot to me. @dwijnand joined SIP to help with upcoming meetings (big thanks!).

Here are the reasons I haven’t been able to keep up: at first I was only in charge of scheduling and minutes, then I picked up the lead role in SIP and organized Scala Days, and many other responsibilities came up. There are too many moving parts with all new features and events that happen, that it got increasingly harder as one person to keep up with the updates.

  1. If you are interested in the SIP committee role in Scala 3, please check the minutes from May 2019 and November 2018.

In short, the SIP committee is working closely on steering the Scala 3 release - but we do need to create the path as we walk on it.

  1. We should go over the SIPs that are pending.

  2. We agreed that some instances in the process itself, regarding proposing and stages, need to be revised. But we didn’t make final changes and notified the public. This happened 2 days ago, when we had a SIP meeting to discuss many organizational points (how we organize ourselves, who can invest how much time, what makes sense and so forth).

In conclusion, the Dotty team decided to go in the feature freeze soon. This will allow the SIP Committee to know what we are dealing with, no more chasing features. The next public meeting will be the end of November (still waiting on the exact date), another one before Christmas, retreat in February or March.

I hope this helps better understanding of where we stand with SIP and Dotty relations.

5 Likes
21

Important:

  • if you want to know more about or help and support the SIP Committee work, please come chat with me at https://gitter.im/scala/center. I will be available as of Monday as of 9 AM CET to answer.

  • I would advise using this thread from here on to stay focused, feature freeze is coming, Martin nicely, timely, and precisely explained context and steps. This thread was proposed as drafting your ideas before the feature freeze.

  • I am personally happy feature freeze is around the corner, we’ve all been waiting for this glorious day (o:

3 Likes
22

As for the actual proposals, I really like and looked forward to https://docs.scala-lang.org/sips/refer-other-arguments-in-args.html (which is from 2017) and https://docs.scala-lang.org/sips/curried-varargs.html (which is more recent). Others may care about other ones more.

But the feeling that contributions that members of the community who jumped through all the hoops to make a SIP are then being treated as if they should be grateful if the time and effort they spend in making the language better for everyone is difficult to shake. When proposing a SIP makes you feel like a nuisance at best - and currently I feel like I nuisance for putting this on the agenda - then something is not right, and maybe you should just not allow for people who dont work for EPFL or Lightbend to make SIPs.

2 Likes
23

I totally understand the feeling and am sure not many people would disagree that that is how it comes across. There is a lot that goes in pre and post SIP meeting organization - which I could use help with, that would, in turn, unlock some of the bottle-necks and make this process fluid, thereafter more responsive, or at least stop appearing disrespectful to people’s time. We need to do better, and I can use all the help as I mentioned above.

But rest assured, there is no intention or consciences decision to “lead people on”, and any insinuation to that effect may actually directly hurt my feelings, being the one who is actively trying to navigate this very complex process, with very busy members, in a very particular period of the Scala language history, in the midst of very opinionated crowd. So again, I am finally asking for help, if you are interested in what it entails, ping me at https://gitter.im/scala/center.

1 Like
24

And - I personally want to thank all the members that are following, are interested in, are writing up SIPs, for all that share their ideas on how to improve the Improvement Process - you are already helping me, and I am looking forward to reaching the point of having a great SIP process that everyone is proud of.

2 Likes
25

This is already implemented in Dotty.

This is on the radar of the SIP committee (notice it was merged only a few days ago)

26

As some constructive feedback, here are some things that could have been done to make this look much better, even if none of the underlying facts are changed:

  1. Differentiate between Dotty and Scala: if we are freezing the SIP process for the next N weeks/months/years to try and land Scala 3, we should say so up front. This doesn’t just affect Dotty developers, so we should make that clear up front.

  2. Acknowledge that there’s a change in the SIP process, and what it is. Just saying “this is a thing we’re changing, for these reasons” is a big improvement over changing things and leaving it to others to notice that something changed, and trying to figure out which things they were told in the past are no longer true.

  3. Differentiate between the SIP process and Martin: it does not look good at all to have Martin come out and say “we’re changing the rules, and here are the new rules that just so happen to make it much easier for the proposals I’ve been working on to get in, and harder for others”. Even if it’s just someone else making the announcement that the SIP process changed, would look a lot better.

  4. Have a migration plan for people who are invested in the old process: there is a finite/small number of old SIPs, saying “these SIPs are migrated in this way, these SIPs we’ll reject, these SIPs seem inactive and we’ll close for now”. People put in a lot of effort and thought into those proposals, often at Martin’s request: “this will need a SIP” comes up quite often in discussions! These SIPs have been provided: now what?

  5. Anticipate and acknowledge those negatively affected by the change in process. There are O(1000s) who can contribute under the old process (experience writing compiler plugins or patching the Scala compiler), an O(10s) of people under the new process (experience working in lampepfl/dotty). Those 1000s of people are effectively locked out under the new process. I’m one of these people: what about us? Even just a simple note “sorry we’re de-facto locking you out of the SIP process for now, we need to do this in order to land Scala 3 on time” would make me feel a lot better than being casually excluded out of hand.

It’s inevitable that processes need to change and evolve, but we need to treat the process and its evolution with respect ourselves if we want other people to respect it. Having a proper process and having no process at all (i.e. closed development) are both reasonable places to be, but if we’re in an ambiguous state we should decide which direction we want to go so people in the community will know whether the SIP process is a good use of their time and energy.

9 Likes