Reply to your feedback on sbt

Hello everyone,

First of all, I would like to thank you all for your feedback on sbt. The result of this initiative has exceeded my expectations and I am happy to see that so many of you have gone out of your way to reply to the call of this thread. I appreciate that – we’re all busy these days.

Your insights have helped me to understand how sbt is perceived and understood, and which are the flaws that most affect you. This information has been invaluable during the past month to prioritize my work on sbt. We have changed our plan according to your feedback with the spirit of optimizing for impact.

As a result of this effort, I created the following mind map:

Note: if your browser does not render the PNG file correctly or you cannot read it well, you can download the PDF version here.

The mind map sums up this thread and splits your concerns in several major topics. These major topics represent fields on which sbt could do better. We will dive into them later.

Most of you have expressed concerns about a large array of different sbt problems. I guess this variety depends on the different uses of the build tool and the professional environments in which it is leveraged. However, there have been several recurring themes across all your comments:

  1. Documentation has to improve.
  2. Sbt should be easier to use.
  3. Sbt 1.0 on Scala 2.12 should be out as soon as possible.

These problems have led my contributions to sbt in the past month. However I have also focused on some of the other remaining topics, contributing to topics like incremental compilation, dependency management, new features and performance.

Contributions

These are the contributions that the Scala Center has done for the next sbt milestone (1.0.0-M6).

  1. Contributing experience:
  1. Usability and Learning experience:
  1. Performance:
  1. New features:
  • Work in progress 90%: Addition of the dependency lock file. The dependency lock file has three benefits:
    • Brings reproducibility to the dependency management of your build: transitive dependencies are stored in a file named dependency.json.
    • Brings performance improvements: several modes in ivy are enabled to increase performance of resolution. It allows both sbt and ivy to assume certain properties about resolution.
    • Allows Coursier or other dependency managers to resolve and download dependencies independently of sbt.
    • NOTE: This feature will be available in a binary-compatible experimental sbt 1.0 version in the next days. As it’s a feature difficult to get right that required big changes in the sbt-ivy integration, I preferred to wait a little bit more and battle-test it with common builds.
  1. Current misbehaviours:
  1. Dependency management:
  1. Zinc, the incremental compiler:
  1. Migration from 0.13.x to 1.0.0x:
  • Created an sbt migration tool to help migrate plugins and builds. This tool has been created to ease migration between 0.13.x and 1.0.0x.

Almost all of this work has been done in the last month and is available now in the sbt 1.0.0-M6 release, published to Maven Central.

The Lightbend tooling team will continue cross-publishing core plugins for it, and if you’re a plugin author you’re encouraged to do so as well! To the best of our knowledge, projects like Coursier and Scalafmt already cross-publish to both sbt 1.0 and 0.13. Use our migrator and see how to migrate to 1.0.x here and how to cross-publish here.

A last, positive remark

I was not familiar with sbt internals before all these contributions. I used to think sbt was too slow of a build tool and excessively complex. I have had some kind of love-hate relationship with sbt during my past year, having developed several sbt plugins and making an intensive use of it. Some of you have expressed similar concerns in this thread and outside of it.

However, this opportunity has given me a lot of insight about how sbt works. It has debunked lots of thoughts and opinions I had on sbt. This new understanding of the build tool has helped me to analyze sbt more objectively.

If I have learned something during the previous month, it’s that most of sbt complexity and slowness is accidental. There is room to address almost all of the complaints you have had in this thread: usability issues, difficulty to reason about scoping, unexpected behaviours. I do not believe there is something inherent to the sbt model that makes it impossible for people to learn it. The lack of hands-on documentation and tools to reason about sbt have not helped in this regard, but this could change, and the Scala Center has done a first step towards this direction.

With the help of the community, we can polish the rough edges of sbt. Help us! You can participate in the development of sbt 1.0.x by filing issues, helping in the Gitter channel and opening PRs. If speed bothers you, file an issue pinpointing what you think could be improved. If you think the docs for a concrete issue are not good enough, head over the issue tracker and let us know. The more actionable something is, the more likely is to happen.

I am optimistic about the future of sbt and hope to see it grow in the next months.

See you in the issue tracker

All in all, I am really enthusiastic about the outcome of this thread! The Lightbend team and the Scala Center agreed to repeat this experiment for the next sbt 1.0 releases. We plan to create a Scala Contributors thread to get your feedback on every release, and we hope that you help us steer the boat towards a direction we all feel comfortable with.

I would like to thank the Lightbend tooling team (@eed3si9n and @dwijnand) for all their help. I have been working with them elbow to elbow to ease migration to sbt 1.0 and advance its development for an earlier release. As announced by Eugene, sbt 1.0.0-M6 is already out and the next release will be the release candidate, where sbt API will freeze and will focus on stability. This new release is scheduled to happen in the beginning of July.

Have a good day,

23 Likes

Thanks for the summary writeup, Jorge!

2 Likes

If this post were read at a live venue there’d be a standing ovation.
Much appreciation and thanks to those that contributed to these changes, especially Jorge!!

8 Likes