A Guide on Binary Compatibility - need your input!

Documentation
2

This is not the right thing to do, because it is useless. A much more useful version scheme is the following:

  • Bump minor version if the new release contains (likely) source incompatibilities
  • Bump major version if the new release contains binary incompatibilities

This versioning scheme assumes that binary incompatibilities are worse than source incompatibilities. Why is that? Because of the dependency diamond problem. Let me explain.

If you are the author of an application App, you don’t care about writing it with binary or source compatibility in mind, because no one depends on you.

If you are the author of a library A used by some applications Apps, binary compatibility is still irrelevant to you. If you publish a new version, then either an App upgrades and it will recompile with your new version, or it doesn’t upgrade. In no scenario will it compile with version x but link with version y. Of course, you want to care about source compatibility so that the Apps can easily upgrade to your new version, but it’s not a big deal if you break source compat. At the time of the upgrade, the App can adapt to your source incompatible changes. They can also not upgrade if they don’t want to.

Compatibility becomes critical if you are the author of a library X, used by several libraries A, B, etc., themselves used by applications Apps (or further libraries down the line). To make things concrete, consider the following scenario:

  • A version a1 depends on X version x1
  • B version b1 depends on X version x1
  • App depends on A v a1 and B v b1.

Now let’s say you publish a new version x2 of X. If this version is source incompatible with x1, but binary compatible, the following happens. A can upgrade to X version x2 and publish version a2. At this point, it might need to change its source code to account for the source incompat of X, but that’s fine. The important thing is that App can upgrade to A version a2. Now we have a funny thing: App depends on A a2 which depends on X x2, and App depends on B b1 which depends on X x1. Note that when upgrading App had to take into account source incompats of A, and possibly source incompats of X x2, assuming the source code of App directly references things from X. But the important thing is that it now has two versions of X on its linking classpath; of course x1 is evicted. That means that we link App, A a2, B b1 and X x2 together. Oh look! there’s B b1 which expected X x1 but it receives x2. That is not an issue, however, because X evolved in a binary compatible way (even if not source compatible).

What you need to see here, is that despite source incompatibilites, the path through A could perform upgrades, and App could depend on new versions of A and X, without B having to do anything.

Now what if the changes were binary incompatible. In that situation, we are stuck. App cannot upgrade A because of B! App is stuck with A a1 until a new release of B comes up that depends on X x2. What if, in the meantime, X published x3, and B’s maintainer decides to go straight to X x3?

In case of binary incompatibilities in the upper parts of the dependency graph can prevent from upgrading a branch of the graph without also upgrading unrelated other branches of the graph. This is dependency hell. Add a couple more libraries to the mix besides A and B, and App can never ever upgrade any of its libraries in a way that they all work together.

Source incompatibility does not have this unrelated branches issue. You can always upgrade any part of the graph, including App, to newer versions of parts of its dependency graph, without being impacted by other parts of your dependency graph.

This is a very, very important point. Binary compatibility is more important than source compatibility! That is because binary incompatibilities lead to dependency hell, while source incompatibilities do not.

Therefore, breaking binary compatibility should be something extreme that requires a major version bump. Breaking source compatibility is not that big a deal, and only deserves a minor version bump. From a technical point of view, this is the only versioning scheme that makes sense for Scala libraries.

8 Likes