Asking for your feedback on sbt + Scala Center announcement

Tooling
36

I’m not hacking the SBT internals or a plugin myself, but seems like sbt needs to support a better integration test framework than scripted. Might the rework in 1.0 and server both benefit and make possible a richer test framework? A framework that would be less tricky but also more performant. This would seem like a sound investment for the plugin ecosystem as well as the sbt core.

3 Likes
37

I’ve suggested putting effort on a better test framework. Seems like now is a good time given the 1.0 and server development.

38

I agree with @fommil; The most important thing for sbt is migration to Scala 2.12 (or Scala 2.11). As we will be late for supporting the latest version of Scala, the more sbt plugin authors will lose the opportunity to migrate their sbt plugins, because we will forget about the implementation details as time goes. Cross-compilation with Scala 2.10 is an option, but I think it’s acceptable even if sbt only supports Scala 2.12, since we are already bearing with sbt that can only use Scala 2.10 plugins.

Another thing I need to see the improvement is reloading time of project configurations; when I modify a library dependency, it usually takes 30 sec or more until my IDE (e.g., IntelliJ IDEA) can start re-configure the project because it needs to wait for the completion of sbt reload task. sbt-coursier plugin helps improving the download speed of dependencies, but this long turn-around time of sbt makes difficult to configure projects.

2 Likes
39

I can’t stand the new feature in 0.13.15 that throws up warnings if you want to run SBT without keeping that VM around permanently. I’m not aware of any other build tool that won’t let you execute a build from the command line without a mandatory, permanently enabled warning that you should start up the build system in daemon mode instead.

I understand that you want to nudge people toward running the console to avoid the extra costs of spinning up a JVM. IMHO, the current mechanism is a repeated, aggressive shove and not a nudge. Putting message like this in the docs, or showing this message once the very first time you use sbt on a machine would be reasonable. Showing it every single time with no configuration option to disable it will ensure that I stay on 0.13.13.

6 Likes
40

My main pain points with SBT

  • Underlying SBT model of scopes/configs/settings/tasks seems to be overly complex and abstract
  • Many of these concepts also tend to be replacements of Scala’s already existing language features (i.e. why have Settings when we can use plain val, why are we dealing with initialization order when we already have lazy in place). Why do we have Task when Scala already has an existing construct (i.e. def)
  • Its incredibly difficult (as an external contributor) to try and contribute to SBT because the design is so complicated
  • SBT has created what is a considered a mistake/abomination in most other language communities, which is some pseudo version of the language which isn’t really the language. In SBT’s case, build scripts are written in an (almost) copy of legitimate Scala code, however in an attempt to “simplify” things there is now a huge amount of magic in how settings are loaded in .sbt scripts
  • The design seems to have not been thought off from scratch, instead it seems to have sought of been based off maven/ivy (a lot of design for stuff like config is based off how maven does things)
  • SBT’s design also means that if you do something that slightly falls outside of what SBT provides, you need hacks like sbt-doge to do non trivial builds (i.e. support Scala 2.12 in a multi project setup, but not for each submodule since not all submodules support 2.12)
  • As a build tool, SBT’s design causes it to use huge amounts of memory (and sometimes CPU). May not be a problem when working on a monolith, but definitely an issue if you are working on modular projects
  • I don’t really see what a purely immutable model with a very convoluted way of creating copies of this immutable representation has brought any tangible benefits. Build tools are all about mutation and state change, it might have been a better idea to use a more mutable model with well thought out API?
  • Features such as runtime adjustments of settings are questionable, its very arguable that the proper way to address this is to have fast iteration cycle (i.e. change source code -> reload project -> run command). This is how CBT solves the problem
  • Performance/usability of SBT in general has issues
    • Although SBT may have high throughput (not sure if this is true?), its pretty irrelevant for most cases due to incremental compilation. I also don’t think that running tasks in parallel provides that much performance improvement, especially with the Scala compiler being as taxing as it is
  • The startup time for most trivial projects in SBT is pretty atrocious, and it also isn’t designed with usability in mind (i.e. SBT has to check all dependencies of your project when it loads, even though its not required a lot of the time, i.e. if you want to run in Scalafmt)
  • Also hard to integrate SBT’s current design in CI servers due to its high load time
  • Resolution with Ivy is really slow, coursier is fixing this but I would rather they make coursier the default resolution mechanism. If it breaks with some weird maven/ivy set up repos, then SBT should claim that it doesn’t support these sought of repo types by default. User experience for 95% of users shouldn’t be hindered because of some really weird repo setups
  • The above also makes developing SBT plugins a real pain, especially with regards to how settings and tasks are defined
  • In general, SBT seems to be good for very trivial builds (i.e. just libraryDependencies and some set versions) or incredibly complex builds usually created by authors who understand sbt very well (i.e. https://github.com/scala-android/sbt-android), but for the middle ground (where most users fall) it falls short in many areas
  • This is also in mind that a build tool is something that every user needs to understand (at some level) because they are exposed to it in every day in life. Its almost always a better idea to keep things simple at the cost of being less expressive or less capable
  • Sticking to Scala 2.10 for so long has probably done a lot more damage than good, especially when taking into account things like macros and reflect
  • Other quality of life improvements
  • SBT doesn’t provide sensible defaults for compiler flags (which are also only exposed as strings instead of proper enums). i.e. if you are publishing, it makes sense to use -Yclosure-elim if you happen to be using 2.11 but if you are using 2.12 then opt:l:project makes sense. Having default flags for the current project with the current Scala version context in mind would prevent needing stuff like https://github.com/knutwalker/sbt-knutwalker
  • Plugins like sbt-mima should probably be included in core sbt as a module (these are pretty much required for any type of sane library)
  • SBT documentation appears to have been written for someone who already understands SBT instead of someone who is completely new to SBT. Not sure how much of this is an issue due to SBT’s already complex design (i.e. its very difficult to do non trivial stuff in SBT without understanding the underlying model which is very complex in design)

At the risk of sounding sensationalist, at least where I work SBT has gotten so bad that coworkers have either given up working on it (which usually means the SBT related work gets delegated to someone who has some SBT experience) or they do stuff like copy and paste code around which creates entirely different set of problems. I haven’t really seen this kind of problem with other build tools. Its gotten bad enough, that even for fairly simple use cases (such as a code formatter) have resorted to using SBT for only its interface, i.e. see http://scalameta.org/scalafmt/#sbt

I don’t really have high hopes of SBT fixing the issues I have described because they appear to be very fundamental, in fact CTB seems to address almost all of the issues. I do like the approach of modularizing SBT more, because at least it makes it much easier for alternative build tools to arise and fix said problems.

4 Likes
41

These are the most significant issues I have with SBT:

  • In a large project with -30 modules, watching sources (eg. ~compile) even in just one of the modules often freezes for up to 20 seconds, doing nothing before responding to changes. Other tasks like testOnly can also suffer from this but usually with <10 sec of lag.
  • Sometimes an incremental build takes twice as long as a clean build. Hard to provide any info about when or why.
  • Scopes of keys are a nightmare. They’re great when you know exactly what’s needed but when you don’t know which scope you need it’s a nightmare to discover; similarly configuring a setting with the wrong scope never yields any warnings or errors. You just have to patiently run through all the related tasks and see if it works. Using inspect on tasks doesn’t help either because it’s usually 8 nodes away in the big task & setting DAG.
  • SBT doesn’t seem to care about UX. The UX is terrible across the board. When presenting compilation errors, it’s insanely hard to decipher huge blogs of monotonous text. SBT should use different colours for filename and line number at the very least. Even at the end it says “n errors” detected but if it’s ≤ 4 it says “four errors”, “three errors”. How ridiculous! This isn’t a damn novel. We aren’t reading all the output from top-to-bottom; we’re scanning for information we know is there. Now this is surely due to scalac not SBT but as far as I’m concerned SBT is the interactive tool here and it should be held accountable for it’s UI and UX; users new to Scala and SBT certainly aren’t going to bother with the distinction. Clippy is an SBT plugin that can be used to apply syntax highlighting to Scala snippets in compilation errors (still doesn’t colour filename and line numbers) but it’s 1) not enough, 2) should be part of SBT proper (or whichever SBT component does Scala compilation).
  • From what I can see on the outside, the SBT team seems too cautious, moving so slowly and scared to break anything for anyone (cough cough Ivy). SBT isn’t solid enough or well designed enough to improve in a reasonable timeframe with tiny, safe changes (which is probably why CBT has surfaced and is gaining momentum). When you look at all the deficiencies and desired features and ask how long it will take to fix/add, at the current rate it will take another 5 years or more. Are people really going to stick around? A strong backwards compatibility story won’t be worth much if, for example, 80% of the community abandons the tool.

Also worth mentioning:

  • It unclear how loose settings floating around in the build.sbt DSL apply in a multi-module project. The whole DSL/Scala dichotomy really feels like a big mess. Why not just Scala? It’s just adding complexity and knowledge curve for no benefit as far as I can tell.

I’ve also good to say about SBT. It used to be hell 4, 5 years ago but these days, with all the improvements and simplifications that’ve been made over the years, it’s a really good tool. It’s trivial to add new modules to a project, to have interdependencies between them, using the .value macros it’s super simple now to customise builds and add task dependencies, it’s not a big deal to add custom commands or settings. I think SBT has a lot going for it and, while I really wish the rate of improvement was faster, I do appreciate the work that has been done to improve it from the monster it was years ago.

4 Likes
42

Since this is (or at least was at some point) a conscious decision and considered a feature, I haven’t submitted a PR to change it, but I agree with the sentiment. Using digits in all cases would make it easier to locate the information on the screen.

43

Here are my thoughts about SBT:

  • SBT API is very complex and contains a lot of unnecessary stuff. I believe public API should be as simple as possible. It should be reduced to a few basic conceptions (project, task, setting, key) represented by a few classes with a few methods. Do we really need Def or Initialize in the public API? Recent changes (like replacing symbolic operators with := and .value) is pretty good, but I think there are a lot of places where we could make API simpler.
  • Scaladoc. There are a lot of code inside SBT without comments, including some widely used APIs.
  • Type safety issues. For example, absence of .value produce runtime error, but it should be compile time error IMO.
  • Too much macro magic. It’s almost impossible for a newcomer to understand what’s going on inside SBT.

Some points are already discussed above, so I will not repeat them. Personally, I think that SBT is moving in the right way, but this moving is too slow.

1 Like
44

I would love to see this discussion move back to “immediately” actionable request for SBT improvements.

The discussion about conceptual issues is legitimate, but I think it should be part of a different thread.

6 Likes
45

it is super-hard to understand find and use settings/keys/scopes correctly without investing huge amount of time.
IntelliJ helps a bit, but not that much.
Problem gets even bigger if one wants to use few plugins.
some examples:

  • how to inspect flags (-Dxxx) passed to test in subproject ?
  • how to execute task in subproject ?
  • where should I put “my” settings - like ( setting in Scope) - what Scope should i use? how to check It, do I have to think about it?
    all such little things requires thinking and remembering details, which even in medium projects can be tedious

Compared to maven which I used before I think it is kind of problem of power and flexibility. Sbt is very powerful and flexible compared to maven, but taming that power requires serious investment of effort.
Maven on the other hand is much more limited, anything non-trivial requires a plugin. and even plugins don’t have the same flexibility that sbt offers out of box via settings/keys/scopes.

46

Overall, I would say I am really happy with sbt. I think it’s the best build tool available for the JVM; if I was for some reason to start a Java project, I would probably write its build in sbt.

I like the way that settings work and that you can explore their interdependencies so well with inspect. It’s nice that you can just guess at scoping, and get compiler or setting-solver errors if you screwed it up. Where inspect or show failed, I’ve found it easy enough to browse Keys.scala and Defaults.scala to figure out how things are being computed and where the correct place to insert a customization is.

It’s also wonderful that so many things are type-checked, and if not that caught by the setting interdependency solver. Having to wait until you’re trying to release to get basic errors that could have been caught with types is really annoying; sbt heads that off pretty well. It also makes it even faster to do exotic things, since you don’t even have to run any builds to catch most errors. I don’t really have any strong feelings about DSL versus regular Scala source; they are both fine to me.

I also had a pretty easy time working out how to publish to Sonatype, and doing so, just following the manual. This was back before sbt-sonatype existed, even, so it is probably easier now. Maybe not having trouble with this makes me an outlier.

In particular, we would like to listen to your ideas and suggestions to improve the build tool. How could we make sbt easier for your day-to-day job? What current interactions with the tool you would like to see optimized?

My only real ask would be a built-in version of sbt-adhoc-subprojects-plugin’s core feature that actually worked with normal projects. Being able to treat whole other projects as subprojects is really nice; I used it all the time with Leiningen when I used Clojure. My hacky plugin is not nearly good enough for real library sbt builds, though.

My experience may be atypical, however. Maybe most users do not care about patching libraries while working downstream.

4 Likes
47

Hello, I think this is a great initiative.

I’ve been using SBT professionally but never learned it. I think

  • It has a complicated syntax
  • Too many files
  • It’s really hard to find documentation of the simplest things for your version. There are so much weird documentation which mixes Build.sbt and Build.scala (I think)
  • Eclipse have trouble using SBT (last time I checked)

The only thing I want is

  • Simple format, JSON?
  • Work with Eclipse
  • Set meta data and dependencies, and then use IDE to make full project setup. Automatically download dependencies is the main reason I use a building tool
1 Like
48

I agree with that point, the syntax makes it hard to learn it. I love sbt for one specific task. Creating an Uber jar for my Spark project with the assembly plugin.

Moving files around to be able to create the final deployment package shoud be trivial.

It would be nice to be able to use regular Scala to perform simple task without having to google it.

49

For me the most awaited feature of sbt-1.0 is the sbt server. Is there any documentation or example code of integration with it?

50

Not sure if this is something I can fix myself, but after using sbt a while I get:

java.lang.OutOfMemoryError: Metaspace

Which requires me to restart SBT

1 Like
51

I can only echo many of the points that have been put forward in this thread already. Please take the criticisms here with a grain of salt because I think SBT is a great tool, I just use it all the time and encounter many of the rougher edges.

Pros:

  • It’s still (probably) the best build tool I have used on the JVM
  • The user interface used by SBT - an interactive shell which covers most common tasks - is great
  • For trivial projects it’s brilliant - set a couple of keys, put your source code in the right place and off you go

Cons:

  • Beyond the most trivial projects, it’s incredibly hard to understand for beginners due to the amount of magic involved
  • Has a strange DSL that blows away all your intuitions about how Scala code works - e.g. if (bool) task1.value else task2.value always evaluates both tasks
  • Documented for somebody who already understands SBT
  • Documentation steers people toward overly complicated usage of SBT (e.g. multi-project builds when they’re not necessary) even for trivial projects
  • Requires odd file layouts that are hard to explain to beginners, e.g. why do I need to add plugins to my build in a different directory to my build file? Why is the version of SBT set in a properties file when everything else is in the build?
  • Related: the recursive build thing is cute but it’s basically just more boilerplate to maintain that I would rather was in the build.sbt or supporting .scala files
  • Incredibly resource-intensive
  • Even slower on Windows which is unfortunately ubiquitous
  • Really slow dependency resolution that makes you regret using multiple projects and configurations
  • Requires detailed understanding of its inner workings to achieve anything that falls outside its built-in functionality
  • The above is exacerbated by the poorly documented source code and dense, difficult to understand APIs
  • The reliance on plugins that are not maintained by the SBT team for functionality that is almost necessary due to SBT’s complexity (e.g. sbt-dependency-graph, sbt-release) is very awkward
  • I have inherited a build that uses multiple configurations to enforce a layered architecture and getting many plugins to work is a terrible experience akin to a Satanic ritual. I have given up on getting Scoverage to work, for example.
  • Doing anything non-trivial with artifact publishing like creating a .zip bundle of several projects is really difficult and I usually give up and use sbt.IO to do it myself because at least I can understand that

I frequently find myself staring at a terse, one-sentence explanation of a key in SBT’s source code trying to make sense of it when I am trying to understand an obscure error. Often the sentence essentially repeats the name of the key in long-form.

I have been forced to add a section to our onboarding page for Scala developers at work describing how to set JVM options to increase the amount of memory allocated to SBT using environment variables. It would be great if this could be configured in a less annoying place and if it had more realistic defaults given how terribly resource-intensive SBT is for multi-module projects.

It’s baffling to me that SBT continues to use Ivy even though it’s really slow, has a global lock that makes it very difficult to use in CI without simply using a different Ivy home for each build and has lots of outstanding bugs that have not been addressed for many years (like the one where it fails to publish large artifacts because it keeps the whole artifact in memory).

Overall I get the impression that SBT was designed to demonstrate an idea for a clever task engine and day to day usage of SBT was not considered until much later when it was a huge, out-of-control project.

Everything about SBT has improved substantially since I first started using it but I feel there is still a long way to go and I frequently think about the simplicity of CBT enviously while maintaining our build.

3 Likes
52

I mostly agree with what has been said but I wanted to bring a small light of hope on a couple points made by DavidGregory084 :slight_smile:

That’s really sad becasue this is mostly a documentation issue : you can create a .sbtopts file at the root of your project for project specific sbt launch options including JVM configuration. I was unable to find a mention of this in the official sbt documentation. It is mentionned when you run sbt -help however.
This feature was added when the official sbt run script adopted sbt-extras. It seems the official script lives at sbt-launcher-package/src/universal/bin/sbt at master · sbt/sbt-launcher-package · GitHub

You can find mention of this feature here and there on the internet.

I have yet to find an non contrived use case for that.
However I find it very easy to explain why you have to declare plugins in a specific directory at the top level.

Because at the moment you can only have one version of a plugin in a build. If you were able to define the plugins directly in a module’s build file, it would become very easy to declare different versions of plugins in two different modules within a multi module build. How would that behave ?
Maven actually had this issue if you don’t use pluginManagement and used to more or less randomly select a version of the plugin (it would select the version of the first module built and use that for all other modules regardless of the declared version, imagine how easy this can be to understand the first time it hits you) not sure if it has been fixed or not.

So I do like that some settings are for the whole build (such as plugin definitions) and some settings are for each modules.

1 Like
53

to 2) found minimal .gitignore for a single build project at https://github.com/typesafehub/activator-minimal-scala/blob/master/.gitignore
it contains the details in directory project

54

@Fristi Do you add this flag -XX:MaxMetaspaceSize= to your sbt? If you don’t add this flag, I thought it would be hard to solve by yourself.

55

Jumping into the discussion: I do have that flag set. Doesn’t help – I still get Metaspace OutOfMemoryErrors after running my application a few times. This has been going on for years now, and I have yet to figure out how to fix it…

(I should note that I don’t know if the issue is sbt, Play, Akka or what – it’s a somewhat complex application.)