Definitely not my answer. I was always very skeptical of SBT’s approach and remain so.
4 Likes
I closely monitor for bloop
I think when bloob has a good intellij idea integration, I will prefer to use gradle or maven because of Occam’s razor 
1 Like
Hey there, I’m one of the main maintainers of Zinc and one of the creators of bloop. The rationale behind bloop is just that: create a build server easy to integrate with and that enables any build tool to provide the best Scala developer experience out of the box. The bloop website linked above is rich in details.
We decided to do this after realizing how badly integrated Zinc was in many popular build tools and how it lacked many performance improvements that could be fixed and be provided as a service for every build tool, improving the experience for every Scala developer no matter what build tool they use. For example, Bloop has pioneered an implementation of pipelined compilation that intends to compile your build faster.
So far, bloop is only missing one feature (safe and performant handling of concurrent compilations for the same project; which I have already partly implemented) before I start to gradually improve our build plugins in Maven, gradle and sbt to integrate with bloop via BSP.
The main requirement of these improved integrations is that end users will now know that they are using bloop under the hood – they will only benefit from all the improvements bloop offers (reduced memory consumption, faster compiles, etc) without learning another tool or paying a UX cost.
I believe it’s a matter of days before we can see these build tools using bloop together with IDEs. For the moment, Metals, fury and IntelliJ are already integrating with bloop. I’ll keep you updated @eyalroth if you’re interested in more developments.
A good way to stay up-to-date is to watch the repo or use our scalacenter/bloop gitter channel.
3 Likes
sbt 0.3.2, the first public release was released December 18, 2008, about ten years ago to workaround idiosyncrasy of Scala and its ecosystem at the time, like slow compilation and lack of IDE.
When Scala 2.9.0 implemented binary compatibility across patch versions, sbt evolved together to implement _2.10 name mangling via %% and cross building support. It’s not uncommon for modern Scala libraries to support not only multiple Scala 2.x versions, but Scala.js cross building as well.
It also set the common interface for test libraries so things like testOnly and testQuick works across ScalaTest, Specs2, and ScalaCheck. In the hindsight we can also question whether Scala needed its own test frameworks too, but sbt provided utility for many users in this regard.
Also there’s plugin ecosystem, which is directly connected to Scala library ecosystem as well. sbt 0.13.x releases held binary compatibility for four years, but there was a demand from the plugin authors for sbt 1.x that uses Scala 2.12.x.
One of the first thing I’ve done was to split Zinc out into its own repository, precisely so other build tools like Maven, Gradle, Bazel and its cousins would have better Scala compilation, and we can evolve Zinc together. But I disagree with the idea that incremental compilation was never the integral part of sbt. Incrementality and parallel processing of tasks is what enable things like ~testQuick across many subprojects. (testQuick reruns test for affected code)
I know people poke at “simple” in sbt, but the build can be simple. Just make a blank build.sbt and run sbt console to get Scala REPL. Throw a few libraryDependencies in and you get something similar to what Maven gives you.
As a sbt maintainer I am a fan of sbt, but as Lightbend Tooling team, I wish for a vibrant Scala ecosystem. For that I have huge respect for folks like Chris Vogt, Li Haoyi and Jon Pretty exploring the build tooling space by putting their ideas to the test; as well as other build tools like Gradle. Where possible, I’d be picking on their ideas to improve sbt since I think there are many things we can improve to get to better existing user’s experience.
Happy hacking.
19 Likes
@eed3si9n - You mean 2008, not 2018. Might want to fix that, though I think given the link and the statement it’s pretty clear that it’s a typo.
2 Likes
Yes, thanks. I fixed it.
A bit of a tangent here, whenever there’s a new language, it seems that one of the first things someone does is make a build tool for that language in that language. It gains popularity because if you know the language, the build tool tends to use the same design patterns and so is easy to use and hack. Over time the build tool gains features to support the users of the language that don’t exist anywhere else. Would you rather write a build tool plugin for your Scala project in Scala for SBT or in Clojure for Leiningen? Or heck, we could build Scala projects with Make.
I guess then you could make the same argument about Maven. Why make Maven when there’s already Make? It’s the nature of art for people to take ideas and make them their own, and that’s OK.
2 Likes
Yea, I gave a talk with Josh Suereth in 2015 that included a brief history of build tools (video, slides), which might be relevant to this discussion here since it does highlight the differences between the tools.
1 Like
2c - being able to inject where necessary a Scala expression to define a settings value is a massive win in my opinion. That’s why Maven, Gradle etc. will never have any use for me. Perhaps they work in 80% of my projects, but not in the remaining 20%. Of course, this advantage also exists for the newer build tools that also use Scala as a definition language. Here sbt’s major advantage is a plethora of well working plugins.
I think people forget the history of SBT. SBT was designed in a time where IDEs were terrible and build integrations really slow. It evolved with scala and kind of pushed Scala to be more IDE friendly.
SBT’s experience of streaming-builds, interactivity and fine-grained dependency management at a class-compilation level are all things Scala came to rely on. Most of these other build tools evolved in a completely different direction (focus on hermetic builds first, then work towards a nice in-dev experience). I think what you’re seeing now is both:
(a) Proof that starting from the dev experience angle of a build tool and working back toward hermeticity is a far better approach than trying to bootstrap interactive features on a tool solely designed around hermetic/stand-alone builds.
(b) SBT falling under the weight of its own architecture to achieve interactive features.
Basically SBT 0.10.x took a huge risk in its attempt to get interactivity BOTH at the code level and the build level. From what I understand of working w/ Mark, it’s based loosely on some papers and ideas he had that weren’t fully proven out. It has a LOT of cool things in it, but I think we’ve found a few limitations which will always limit sbt to small->medium sized builds which are coordinating artifacts via Ivy/Maven/etc. That said, I still find it the BEST build tool for that use case, albeit I too wish the surface syntax was less arcane.
Having used Gradle/Bazel in anger for a while now, I can confirm that neither of these gets the user experience for a project as good as I had it in sbt. Just the amount of code SBT has to ensure that you can recompile a project with native-library linking and reload the REPL without massive JVM breakages is impressive. These are things we take for granted in SBT, but when you’re doing OpenGL or JLine work in Scala, you very much appreciate the thought and care that went into the classloader architecture alone. Effectively those other tools have architecture that are geared towards their primary languages. SBT has been solely shaped by what makes Scala best. It’s hard to get those other tools up to the same standard.
Does Scala need its own build tool? no
Would Scala have been as successful without sbt? Probably not
Can we do better than sbt? yes. We can also do better than Maven (for sure), and Gradle.
I think it’s up to each project to determine what’s important to them. I also don’t think there’s huge issues, if your components are divided well, running different build systems with clear README’s for how to develop each component. I don’t see as much of a need for centralized tooling, as even with centralized tooling you have large divergence in other aspects of development. (e.g. in Gradle Scala there’s a lot of nuances you won’t see in a Gradle Java.)
16 Likes
@jvican This is great news! I am already subscribed to a few tickets regarding build tool integrations with bloop (therefore I recognize you 
), but despite that I wasn’t aware of this progress. I wonder if we could improve the process of updating users about these endeavors in a centralized location. I believe this topic was discussed a bit as part of the discussion on the preliminary developer survey results.
I wonder if there’s a connection between Bloop and the SBT server @eed3si9n and Josh Suereth were talking about in their talk from 2015? Is the so called “server” is in fact Zinc? The talk presented the server project as a very interesting one, and is quite a good talk overall! It’s one of the few I actually enjoyed watching in (close to) normal speed.
In no way I am trying to claim that incremental compilation was not an integral part of SBT, but rather that it is no longer has to be. Identifying and removing task execution redundancy is and will ever be an integral part of SBT, but that is a different feature than incremental compulation, and in my opinion could be separated from the core build tool. It is obviously a part of the build process – much like the compiler is – but I’d rather see it as an external component.
As for testOnly and other build tasks related to tests, AFAIK there are similar alternatives in other build tools, and indeed I wonder if there’s a necessity for Scala’s own test frameworks. I very much agree with the notions described in Li Haoyi’s blog post regarding essential testing and his utest framework.
Moreover, glancing at SBT’s current status, it seems to be lacking a few major test related features in comparison with the other tools. It seems to be falling behind the other tools on other aspects as well, and especially in popularity, community size and the scope of contribution (I’ve described this in more details in the Scala Users topic).
But then again, I am not denying the importance of SBT and the major part it took in Scala’s history, nor am I requesting for its development to stop - as long as it gives purpose and enjoyment to its developers and contributors, they should keep on with their work and efforts.
What I am asking is whether this should be considered the major build tool for Scala. Should we aspire to see it used in the leading Scala projects? Should we recommend it to Scala newcomers (from Java or not) over other tools? if so, why?
I believe these questions should be addressed publicly and presented formally on SBT’s website.
1 Like
Currently I think yes, it is the major build tool, but may be it’s a matter of days.
To say the truth, I do not understand why sbt should advertise other build tools 
I think any such website should declare the product advantages and disadvantages and that is enough.
I think if some website publishes something like: “we are not major we are minor”, it will be honestly but, I am not sure it will be healthy 
In the end there are scala website for such recommendations.
1 Like
I should do a better job at advertising the changes. Maybe I should just start posting important bloop releases in this forum so that everyone is up-to-date 
Eugene is in a better position of answering this question. As I see it, the sbt server is greater in scope than just zinc (and, by extension, bloop). Bloop only tries to get right three things: compiling, testing and running Scala applications, whereas the sbt server would try to be something like the Gradle background daemon.
I like sbt and I would like to see sbt improving in the future. I believe a compilation build server can help improve sbt down the road in numerous ways and it complements the sbt server. It’s one of the many approaches people are taking to improve the tool and I think it’s got potential.
What was the reasoning behind creating bloop as wrapper for zinc instead of expanding the surface area of zinc?
One of the common reasons for creating a wrapper is not having enough of influence on the development on the wrapped project (not being able to merge changes fast enough) but i’m not sure it applies here.
3 Likes
I’d say bloop is the spiritual successor to sbt server. The idea was you split apart the “components of good developer experience” and let all tools leverage them within their own idioms.
Bloop appears to be a less-aggressive attempt that will likely be more successful
4 Likes
Experimentation was indeed one of the main reasons why it started out as an independent project. Bloop has been the testbed of BSP, compiler performance improvements and other innovations when it comes to client integrations. The cost of experimenting in the zinc repository was too high.
However, the main reason why it was separate is a matter of principle. In my view, a build server is much more than an incremental compiler. Yes, Zinc currently powers bloop, but who knows how we will be doing compilation in the future? Who knows if we find better approaches to incremental compilation than zinc? Or break internals of the test framework APIs?
Bloop only marries itself to its configuration file format, the CLI commands and the BSP protocol and these are independent from the backend. If rsc or a new compiler comes along, integrating it with all the existing build clients (numerous IDEs and build tools) is simpler.
I elaborate on some of these points in What is Bloop · Bloop.
1 Like
The notion of “server” is pretty much the same as Language Server concept espoused by VS Code’s Language Server Protocol, which is why I implemented a VS Code extension with sbt 1.1 to demonstrate the point.
Any tooling that lives natively with this rich user experience, I would say is the successor of the “server” concept. That includes
- Metals + Bloop
- Dotty’s IDE support. See ‘Integrating IDEs with Dotty the experimental Scala compiler’ by EPFL’s Guillaume Martres (video, slides)
One of the limitations of building rich IDE on top of Scala 2.x has been unstable support for tab completion (presentation compiler), and it’s exciting that Scala 3 was designed from the beginning to support IDE interactions.
3 Likes
If you think you would like to forever stop using SBT in favor of your own maven/gradle integration, just do it. If other people are impressed then great, if not who cares: you still get the build system you want to use that is better than what was there before.
I haven’t used SBT in over a year now. Work is on Bazel, personal projects are Mill. Old projects still on SBT get ported to Mill the next time I touch them. If you want to use SBT, that’s entirely your choice. If you want to use something else, you can do that. If you want other people who are paid to work on SBT to instead work for free on the thing you yourself find absurd to work on, I don’t think anyone can really help you lol.
8 Likes
I’ve used a bunch of build tools. I tend to use sbt for scala because to be honest it’s the least pain option right now for me. I don’t like SBT. But I hate other build tools more. Perhaps I should look at Mill. I find SBT a horror show to extend with new functionality. But at the end of the day, scala the language sits to the side of the build tool used.
1 Like
From what I’ve seen so far, Mill’s syntax is waa…aay better. I tried it a little, but there are still a few issues that prevented me from migrating. Currently I’m busy with more crucial parts of my project, or I would have contributed to fix what I can. I think a good tipping point will be when IntelliJ will directly support Mill and will auto-import from it, because right now IntelliJ-Mill support status is similar to when I started using Scala and SBT ~4 years ago.
1 Like