A couple of years back the scala-library-next repository / project / initiative was started.
And, if my memory serves me correctly, during the first 6 months or so, it received a lot of traction and was shaping an interesting roadmap for the next release of the Scala standard library.
However, probably due to the imminent release of Scala 3, the pandemic, and other factors, the interest in the project slowly faded away. And, as of today, it is basically a ghost repository: that didn’t even manage to do a proper first release.
Nevertheless, questions about the next release of the standard library, proposals for additions, and the like are relatively common topics in the official Discord server (or at least, that is my humble impression).
You may see such interest reflected in the number of issues on Github and other posts here like Atomic (read-only) snapshot and clear for TrieMap? and Tailrec support for Option.fold
In conversation with @SethTisue on Discord the other day about this topic, we both saw that we have a chicken-and-egg problem with the repository.
On one hand, it seems folks directly related to the language development (like the center) are waiting for the community to try to contribute more to it.
On the other hand, it seems community folks (like me, so take this with a grain of salt) are waiting for an official action from the center or another official institution.
Thus, I am writing this with hopes of reaching folks who may be able to bring new life to the initiative again.
Of course, this is a complex topic, since IMHO, is not just about publicizing the repository; which is also an open topic: publicize the new repo · Issue #3 · scala/scala-library-next · GitHub . But rather, implies proper leadership of the initiative. A non-exhaustive list of things that I think should be done before properly publicizing the repository again are:
The new stdlib version will be part of 3.6.0 (LTS)(I am making this off just for the sake of example), as such, this library will be frozen with the release of 3.5.0 (experimental) which will be used as the last phase of testing and for caching bugs. We expect to release 3.5.0 in 2025 …
Define the signoff process(related to Criteria/signoffs for merging PRs · Issue #52 · scala/scala-library-next · GitHub): I am pretty sure anyone wanting to contribute something to the future stdlib knows the acceptance bar will be high. However, it would really help to make explicit what that means, how many approves are required, who will review, expectations around unit testing and documentation, etc. Even more general, what should be the process to propose changes to avoid “wasting time”, should folks always start with an issue rather than PR? Will there be some phases like, first most things will be accepted, and then in a second phase, the library will be trimmed down?
Scope of the library(related to What kinds of changes/additions are possible? · Issue #4 · scala/scala-library-next · GitHub): What kind of changes will be accepted, which ones won’t or can’t be accepted? Should this still be published as an external library that folks can depend on? Or rather should it be a fork of the stdlib so that changes can be made more organically?
This proposal is related to making the Scala 2 library accept updates and additions again. I have been lobbying for this for some time. If the community wants to put some votes behind it, this would be great.
The situation is that the Scala 2 releases need to change their update and compatibility policies to allow this. It would be good if someone from the Scala 2 team could explain further.
In Scala 3 alone we can add new classes to the standard library. But we cannot in general add new methods to existing classes; that requires a new Scala 2 library release.
We can also add extension methods for existing types, but these extension methods have to live in their own objects; we cannot add them easily to Predef or scala. So to be useful they’d have to be explicitly imported in client code.
Allowing the Scala 2 standard library to grow would change the equation fundamentally and would make such updates feasible.
Semi-annual notice that Valentine’s Day is just around the corner for a 2.14 release.
It could be the release where the library finally receives some love.
Probably, it was premature to cancel 2.14 with the expectation that no one would be “stuck” on Scala 2. The consequence was that the library has been locked up.
Edit: 2.13 was billed as a “library release”. That was what year is it now four years ago. It’s natural to expect a refresh.
IMO we definitely need to evolve the library. It’s been basically frozen for the last half decade.
A lot of migrations are frozen half done, e.g. deprecation of mapValues in preparation for making it non-lazy.
A lot of good stuff that could have gone in has not. Consider the case s"foo $bar" simple string pattern matcher. People like it. I’m lucky I got it in before the code freeze in 2018. A month later and it would still not have managed to get into the std lib even in 2023. There are probably countless other std lib improvements that weren’t so lucky
We can’t freeze the library forever. We probably don’t want to freeze the library for a decade at a time; even Java does not provide such guarantees for forwards compatibility, and adds new methods to existing classes yearly. Given 5 years have already passed since we froze the std lib, we need to make an explicit decision here. One of:
New library changes will go into Scala 3 only. Scala 2 is maintenance mode and is frozen inperpetuity, and Scala 3 will drop “std-lib-sharing” with Scala 2 and continue to evolve on its own
Loosen restrictions for forwards compatibility for Scala 2.13 to allow new library changes to land. Even if we can’t break backwards compat, just being able to add new stuff to the library opens up a world of possibilities. Though changes like the simple string interpolator above would still not have been able to go in under this. It would also mean breakage if e.g. libraries built against new versions of 2.13.x and run in applications on old versions, but this is a problem Java versions face as well, and the Java folks - with almost 3 decades of backwards compatibility - have decided that having such scenarios break is an acceptable compromise
Plan for Scala 2.14 for breaking library changes (w.r.t. both forwards and backwards compat) to land, so Scala 2 and Scala 3 can continue to share the std lib while preserving the Scala 2 compatibility policy
Any of these options are plausible, but the status quo of “nothing ever changes, forever” that we’ve had for the past half-decade isn’t something that’s viable to preserve for the decades to come
It’s explained here github.com/scala/improvement-proposals/pull/54/files. The necessary changes need to happen in the build tools, and some non-trivial adjustments for Scala.js (and possibly Scala Native) as discussed on the SIP PR.
As far as I can tell people seem supportive of that SIP. In the committee discussion it was requested to experiment with a modified version of sbt that has the changed behavior and then bring it back for discussion.
Dropping forwards binary compat would hopefully release enough pressure and allow maintaining the shared standard library between 2 and 3 for a couple more years until migration to 3 is mostly done. A 2.14 would be very expensive for the Scala 2 ecosystem, I think it would only make sense if it actually brings significant new value to Scala 2.
Could you elaborate why this would be beneficial to invest in when people are already cross publishing macros for 2.13 and 3? - you would still need to rewrite the macros,
Arguably, to help migration it would be most beneficial to the library authors to implement scala-reflect API with scala 3 but of course some APIs would be broken (we could still add an optional checker phase to lint broken APIs)
I mean maybe at this point the ship sailed. But things are way simpler and easier when you don’t need to maintain two separate versions of your source code.
I said this when Scala 3 was still in the design phase. People are much more okay with making a few changes at a time every so often than having this one “rewrite the world” hurdle. That is why every other software project (programming language and otherwise) evolves this way, rather than years ago when it was common to spend years building a completely different version of something.
Maybe it’s too late now. But I’m pretty sure a lot of the degree to which Scala has been losing mindshare is because of this. If a close approximation to the scala 3 metaprogramming system was released as part of otherwise-Scala 2, and all the other changes were deferred, there would have been a period of time when the community focused on porting macros, which would be a lot easier since they could be migrated one by one without worrying about other language changes. And Jetbrains Scala team would have been focused on implementing support for them. At that point the community could catch up to the Scala team and feel that the Scala team is in touch with the community and that they care about tooling. They would then have appetite for more improvements which would follow that cycle.
Instead, we’ve had a few years now where a lot of Scala developers feel stuck on Scala 2 and that the compiler team are out of touch and that the Scala 3 team doesn’t care about IntelliJ support and other tooling people rely on that needs a lot more TLC.
Would it honestly help at this point? Maybe not, I don’t know. My main vantage point is trying to get Slick released for Scala 3, which is now held back by some downstream libraries that need to be ported, as well as my own main codebase which is still on Scala 2 and I don’t know when I’ll have time to migrate it. I don’t have the broadest perspective. But if I could worry about just migrating macros, and worry about the other changes later (since vice versa is not an option), it would make it a lot more feasible.
