Nice!
In addition, I would say that Scala itself also follows this versioning scheme, because in “2.12.3”, 12 is really the major version. 2 is more like the “epoch”.
1 Like
The guide we’re discussing is aimed at library authors, not at users. Library authors should be taught that binary compat is a major breaking change. You want to prescribe what library authors should do, not describe to users what library versions mean. The latter should be explained by each library author for their own libraries (possibly referring to a common document shared by many libraries).
Just to clarify and avoid any misunderstanding – the above Akka versioning doc explains that we moved away from the [epoch.major.minor] and use [major.minor.patch] nowadays. This means we’re more strict than before with maintaining binary compatibility – 2.5.x is backwards compatible, and actually even 2.4.x is backwards compatible to 2.3.x. In the previous scheme, before 2.3 these would be binary breaking.
Currently we strive to never* (unless huge change, but that’s many years ahead if ever) have to break compatibility and aim to remain in the 2.x major timeline, never breaking anyone’s code (see the doc for exceptions from the rule (may-change and deprecated APIs).
I like this initiative, and wanted to write such a thing at some point as well as we do some crazy things to maintain compatibility sometimes… I’ll try to chip in a few examples, no hard promise though…
Good idea, thanks for picking up the topic!
– Konrad
I definitely do agree with what Akka and and Scala.js project is doing. Currently when a transitive library is evicted with a minor version bump, I have to dig deeper into these project’s release notes to find out whether I need to worry about binary incompatibilities. With guidance on how libraries are versioned, hopefully we’ll get to a point where a glance at the evicted versions will tell us whether we need to worry.
I would like to chime in to say that I believe Sébastien’s proposal is the best way to version Scala libraries and that we should make it an standard. I proposed it for typelevel/cats, but nobody seemed excited about it: https://github.com/typelevel/cats/issues/1233#issuecomment-320770859.
To me it seems that the decision was precisely to use this versioning scheme. It’s even slightly stricter, although only in a hand-wavy way, in that backward source breaking changes should be avoided as much as possible. So even better!
So @jatcwang that’s another big Scala library using this system: cats.
Also relevant is how sbt interpret version numbers, for Scala library the scheme used to check for BC is https://github.com/sbt/librarymanagement/blob/480a1a366ddd69e4179fb097eb84163a1ffbf42f/core/src/main/scala/sbt/librarymanagement/VersionNumber.scala#L135-L138, meaning that Scala.js 1.0.0 and 1.1.0 would not be considered binary compatible unfortunately.
Honestly, sbt’s auto detection of what it thinks Scala libraries consider as BC is just a nuisance. No offense, but sbt maintainers don’t see all the complaints that we get from people who are worried about sbt compatibility warnings about my libraries. I have to constantly tell people to ignore sbt eviction warnings. It’s really a pain.
I’ve mentioned that a couple of times already. I wish sbt just stopped with that, or at least took the conservative assumption (which is semver, i.e., the first number can break bin compat, nothing else).
Maybe what’s needed is some way for libraries to let sbt know what versioning scheme they’re using, e.g. with a field in .pom files? What do you think @eed3si9n ?
I would be happy if each project would just explain in some standard way what its approach to source and binary compatibility is. But it would certainly be nice to have a way to specify so sbt won’t complain.
Another thing I wanted to mention is that I’ve done something like this a few times:
// old method
private[mypackage] def foo(bar: Bar): Unit = newFoo(bar, defaultBaz)
// new method
def foo(bar: Bar, baz: Baz): Unit = ???
This is another example of a binary-compatible change that’s not source compatible. By using package private, anyone recompiling has to use the new method, but the old method is still available without polluting the public scala API. I think this is a nice strategy for evolving APIs in a binary compatible way.
3 Likes
Updated OP.
@gmethvin That’s super useful to know! I’m assuming that this works because package private is a scala-only concept so the running bytecode can still access the method.
Still looking for a few more examples of unexpected sources binary incompatibility & tips on library design so keep them coming 
Hum, could you point me to the specific comment (or PR) in that discussion that addresses this problem? It’s a pretty long thread, so I may have missed something, but all I see is improved presentation of the eviction warnings that were there before. They’re easier to understand, but they’re still annoying when they’re inaccurate (possibly even more so now, since you’re now drowned in a wall of warning text while previously you had some relatively discrete lines). I don’t see a way for libraries to tell sbt that they follow semver for binary compatibility, and that sbt should not report a warning at all when it evicts Akka 2.5.4 in favor of 2.6.8.
Yes, here’s an extract:
More complete solution might be: Allow modules to declare version semantics.
Later down the thread I say this is the right way to do it, which is exactly what Guillaume propoaed here.
Many thanks, @jatcwang. This sort of thing is immensely helpful to a lot of people. Thanks for undertaking this, really!
@sjrd, @ktoso, @jvican (and anyone else who has weighed in… maybe @smarter?) would you guys have a run through of @jatcwang’s proposed guide? You’re the experts here. Your input on this guide would be super valuable!
(@ktoso I saw you already made some comments on the PR, thank you! If you have anything else you’d like to add, please don’t hold back )
1 Like
I just did a pass through the document, thank you again @jatcwang for your work. It feels really good that the Community helps us to convey such important “teachings” to the Scala devs.
I think it would make sense to have such mechanism. We do have the precedence of API docs attribute. Here’s my proposal for abiVersion setting - https://github.com/sbt/sbt/issues/3562
We need to be a bit careful about “SemVer” though. If sbt assumed SemVer 2.0 for all libraries, then it will warn that X 0.6.1 and X 0.6.2 are not binary compatible, since it does not cover 0.y.z.
Also for lower level libraries, I think we should also consider Persistent Versioning (http://eed3si9n.com/persistent-versioning) so 1.x and 2.x can live side by side.
Potentially relevant is Java’s binary compatibility guide/reference. I suspect this level of documentation might be unattainable (Sun had technical writers), but all the info in here is relevant.
https://docs.oracle.com/javase/specs/jls/se7/html/jls-13.html
Regarding inlining, Lukas Rytz quoted Scalac 2.12 release notes (Translate a virtual call on sealed trait/class with a single concrete subclass to a virtual call on that subclass. by TomasMikula · Pull Request #6088 · scala/scala · GitHub):
When inlining code from the classpath, you must ensure that all dependencies have exactly the same versions at compile time and run time.
If you are building a library to publish on Maven Central, you should not inline code from dependencies. Users of your library might have different versions of those dependencies on the classpath, which breaks binary compatibility.
From same thread:
When the inliner is enabled, the default heuristics may (and does) pick methods for inlining that don’t have the @inline annotation. This means that you cannot control the ABI with the optimizer enabled (except for
-opt-inline-from).
Guidelines on how to use -opt-inline-from would help. I guess you can at least inline from the same library, but that might make more code affect the ABI.
EDIT: forgot to say, thanks @jatcwang for working on this!
2 Likes
Hello everyone, @jatcwang has done lots of progress on the binary compatibility guide front and I’d like to summon other experts to complement my reviews to his document. I feel this is a topic where we need a sweeping consensus (after all, his proposed scheme will be recommended in the official Scala website).
So if you want to see a growing Scala community that is more conscious of binary and source incompatibilities and uses a versioning scheme that forces us to reason about it, check his guide out: https://github.com/scala/docs.scala-lang/pull/881#pullrequestreview-72713752.