Bloop is a Scala build server, whose main purpose is to provide fast and reliable compilation for other build tools and editors. It can be used by Metals, Intellij, and Scala CLI or via a standalone CLI. It has proven to be a useful tool for making developers productive, however, because of the complicated build of its code base it has fallen by the wayside recently.
Bloop has a lot of great custom solutions for a number of issues such as shadowing or doing additional release tasks, however it is also a bit complicated and hard to understand. These can also cause a lot of maintenance burden and scare away potential contributors with consequences such as:
- slow start of sbt due to additional configuration to be generated
- complex build definition, with a lot of custom solutions that might break easily when modified
- no automatic updates for dependencies, mostly connected to the previous point
There has recently been work done to extract the main functionality of Bloop to a separate repository for Scala CLI which is being used since the 0.1.0 version. However, the situation with the fork will also not help the Bloop maintenance, so we should aim to have a single core repository of Bloop, where it interacts with Zinc and contains the most important parts plus the ability to run tests and benchmarks.
We are currently working on adopting the mainline Zinc instead of a Bloop custom fork, which would make sure that we get all the most recent improvements and avoid any unpleasant surprises such as with Scala 2.13.7 that needed a separate fix. We will also be able to drop support for older Zinc versions in Scala 3 and get all recent improvements and fixes. We will later integrate pipelining supported by Zinc, which helps us to reduce some custom logic we previously had for it.
Some of the intended work to be done:
- remove custom shading plugin and use something like GitHub - coursier/sbt-shading instead.
- move integrations to separate repositories, where they can be released independently. Configuration has been stable for the last 2 years, so there is no need to keep it all there.
- create a community suite (most likely some modification of buildpress) that would test out some of the more popular repositories.
- make benchmarks easier to run and use.
- rewrite the generate test projects to be created during tests
- migrate from sbt-release-early to sbt-ci-release
- add Scala steward
These changes should simplify working with Bloop and ease the maintenance burden on any current and future contributors. Later we would also be able to synchronize with the Scala CLI fork and make sure that we only have a single Bloop fork.
We are also considering to ship Bloop with its own JDK 17 (this is currently being tested by the Scala CLI fork), which has better support for named sockets and would allow more secure and much less flaky connection. A change like that might be seen as controversial, however: dealing with multiple Java versions, detecting the Java home and making sure that everything works on each possible version takes a toll on the amount of time used for maintenance without providing clear advantages.
We would welcome any comments or suggestions, nothing is 100% decided yet.