I have almost finished migration of my major sbt plugins (e.g., sbt-sonatype, sbt-pack, sbt-sql, etc.) so that we can drop Scala 2.10 support. However, migration to sbt-1.0 (Scala 2.12) was hard because I encountered various incompatibility issues (e.g., lack of Scala 2.12 supported sbt-plugins). I totally agree with the idea of dropping Scala 2.10, but dropping Scala 2.10 itself is already challenging because many libraries are already depending on Scala 2.10 or 2.11 code.
As another example of migration pains, Spark seems struggling to support Scala 2.12 because since Scala 2.12, Scala compiler started to use Java8 lambda (which sometimes generates non-serializable classes) https://issues.apache.org/jira/browse/SPARK-14540
Spark sends closures to remote machines to perform distributed processing, so if Scala compiler generates non-serializable lambda classes, it’s almost impossible to send and run a user code in remote like Spark.
We already have Scala 2.10.x, 2.11.x, 2.12.x, Scala.js, Scala.native, etc. I think I’m spending a lot of my effort only for configuring TravisCI, CircleCI, etc. to enable cross building for these versions and target platforms.
I think a current problem is sbt plugins are maintained by individual contributors, so the knowledge is quite distributed (mostly concentrated in sbt developers side) and making the migration to the latest one so difficult. Cross-building itself is not so much a pain if we have a working configuration for major CI tools. At least Scala 2.11 and 2.12 are almost compatible except a tricky case like Spark.
If we have a central repository for all major sbt-plugins and common Scala libraries, I think any incompatible changes in Scala language or sbt core side can be detected in continuous integration manner while developing core Scala tools. Then we can discuss how to address these incompatibilities in Scala language and sbt development side, before relying on individual contributors effort.
In a nutshell, I’d like to propose changing the current Scala development process from:
- (current) Ship it (a new Scala/sbt version) -> let individual contributors test and fix problems
- (future) Gather all core libraries and plugins to somewhere (e.g., Scala center?) -> Test incompatible changes for each Scala version-> Fix or provide workarounds -> Ship it