Build Server Protocol in sbt

Scala Center Updates
1

( The Scala Center team is dedicated to providing regular and transparent community updates about project plans & progress. In this forum we created a new topic category to allow quicker orientation going forward: “Scala Center Updates”. Even though all feedback is welcome, we keep the right to make executive decisions about projects we lead. Overview of all our activities can always be found at https://scala.epfl.ch/records.html )

This project was initiated by the community members and voted on during the Scala Center Q1 2020 Advisory Board as per Scala Center Proposal 023

Identified Problem

The integration of IDEs (Metals and IntelliJ) with sbt rely on the injection of custom plugins into the user project configuration. This approach is burdened by a number of drawbacks:

  • the user experience is not optimal
  • the custom plugin can have bad interactions with other plugins
  • the build requests from the IDE do not run the full sbt task graph (including source generation) which sometimes cause the IDE to be out-of-sync with sbt.

Example: running sbt clean will cause Metals to go off the rails.

  • the maintenance effort is multiplied by the number of plugins

Proposed Solution

The proposed solution is to implement BSP in sbt server. This solution would solve the identified problems. Additionally it will push the adoption of BSP in the Scala ecosystem one step further.

Identified Risks

The synchronous nature of sbt could deteriorate the IDE user experience:

  • It would be impossible for the IDE to compile when the sbt shell is running another task
  • Running the sbt watch command would block the IDE indefinitely. In other words, it won’t be possible to use the watch command while working in a IDE.

Third parties (in collaboration with)

The third parties involved in this project are:

  • sbt maintainers
  • Metals maintainers
  • BSP and Bloop maintainers
  • JetBrains and the IntelliJ Scala plugin team

The Scala Center team is actively involved in collaborating with all interested parties during the development of this project.

Scala Center proposed roadmap

May 2020

  • Take over @eed3si9n initial draft for BSP support in sbt and implement the minimum viable set of BSP endpoints in sbt core. This implementation will be delivered to the community in an experimental sbt version.

  • Elaborate on the integration of the sbt BSP server in Metals and/or IntelliJ Idea to assess the viability of the solution and to collect community feedback.

June 2020

  • Implement the BSP server discovery standard in sbt so that the integration in Metals and IntelliJ become trivial.

  • Depending on the community feedback the Scala Center will push forward to ship the full BSP implementation in sbt 1.4.0.

July 2020

  • Implement in Metals an option to start the sbt server if it is not yet started. This option will be an alternative solution to using Bloop with sbt-bloop.

  • In case of a poor user-experience, elaborate on a better integration of Bloop in sbt via delegation of compile, test and run requests. In this solution, sbt would be a BSP server but the actual compilation and execution would be serviced by Bloop.

From August 2020

  • Maintain the BSP implementation in sbt and support the eventual BSP evolution.

  • Support the community in integrating other tools with the sbt server via BSP.

17 Likes
2

Thanks for the update, this is really exciting! I’m excited to see what the team comes up with, and I’m hopeful it will improve the user experience for both Metals and IntelliJ users.

Looking back at the initial proposal, there were three different strategies outlined. It seems that the team has chosen the second, although it looks like there were some pretty large concerns about it in the conversation.

To copy a couple of them from the conversation:

Implementing BSP in sbt requires locking down sbt to execute every task/command so it doesn’t support concurrent clients (e.g. while your terminal is running a command, Metals cannot compile your project). This problem is solved in Bloop.

Another problem is that sbt needs to be always running in the background. So, if you have three different projects open in VS Code, you need to have three sbt sessions running, with the tax on memory consumption that comes associated with that.

Are you able to elaborate a bit on the reasoning for this choice over the third option of which this was said:

Strategy 3 seems to offer the best of both worlds, and seems to be favored in the comments so far. It should provide the user-visible benefits of both, as well as reusing a significant amount of existing implementations (though less than Strategy 1).

Thanks!

2 Likes
3

Hello Chris,

Thanks for asking! The goal of the ongoing effort is to evaluate if “strategy 2” is viable. You can see in the roadmap that if this solution doesn’t work we plan to investigate “strategy 3” (in July 2020). We started with “strategy 2” because it seemed to be a quite low-hanging fruit, and it’s anyways a first step towards building “strategy 3” (if needed).

5 Likes
4

Thanks, Chris & Julien!

I would like to add that the fact we are now able to fully collaborate with @eed3si9n (which was not an option back when the discussions were taking place) was also an added opportunity to go forward with “strategy 2”.

The intention of presenting the road-map is to notify the community that we a) took all sides in the consideration b) made a decision & back-up strategies c) will move forward in a certain time-frame and d) will continue to communicate about our findings rather than making a statement “one solution fits them all”.

3 Likes
5

I’m very happy about this development and hope to be working together closely to improve the sbt experience in IntelliJ through BSP!

9 Likes
6

Thank you everyone in the community for bringing this up.
The sbt shell improves a lot and this kind of integration will make it great.

3 Likes
7

How is viability determined? I appreciate that there’s an “Identified Risks” section in this report but there’s no information on how these risks will be mitigated. Concretely, is the plan to try to make the sbt task engine fully asynchronous? If not, then how do you plan to avoid blocking?

Strategy 2 is “Implement BSP in sbt core (probably through sbt server), directly using the task graph” doing that well seems super hard to me, I’m impressed you consider it low-hanging :grin:. I would have thought that integrating sbt-bloop into sbt would be lower-hanging, since it’s basically all there already and we know this sort of thing is possible after sbt integrated the coursier plugin in 1.3.

7 Likes
8

@smarter what exactly are you trying to say?

How can we do better?

9

Not sure what else to add:

Basically I’m looking for more details on what will be tried.

2 Likes
10

Thanks for being more precise.

Re more details on what will be tried: we will keep you and the community posted. This post (first of many) is not to disclose all the internal scenarios we are looking into (which you are welcome to be a part of), but to ask you (the community) to give us your solutions so we can enrich our work.

So, please share with us what you think we should try?

11

Integrate sbt-bloop first and see if that addresses the concerns? (Maybe that’s an option 4?)

12

This option doesn’t address the following concern:

Unless I missed something?

13

IIRC, the there is a keynote that sbt is Event/Command driven by @eed3si9n, so why can’t it be blocking? or the current BSP can’t support bi-direction communication?

14

Right, I was a bit too quick to characterize this as “low-hanging”, sorry. It seems that a proper integration between sbt and bloop which keeps them in sync at all times will require to do much more than sbt-bloop currently does.

1 Like
15

Blocking means that SBT will only be read to respond to another request after it has completed responding to the previous request. Of course, the communication is bi-drectional (request - response). It means that when SBT is working on a long-running request, other things have to wait. For many tasks, users expect a quick response and will be frustrated to wait.

3 Likes
16

Thanks for you reply, very clear:)

17

Thank you for sharing this update! I’m excited about this project and I’m looking forward to try it out when it’s ready.

Identified Problem

Another issue that users end up re-compiling their codebase twice to run non-compile/test workflows such as publishLocal, docker and sbt-revolver.

Identified Risks

Here are some additional challenges you may face,:

  • clearing diagnostics with low latency, we discovered that if you wait until bytecode generation to clear diagnostics then users get slow feedback when they have fixed a compile error (reported issue Feedback very slow in some cases · Issue #438 · scalameta/metals · GitHub). See https://youtu.be/MRQMylDxBJ8?t=709 for an explanation of the solution that Bloop uses to clear diagnostics sooner.
  • enable the SemanticDB compiler plugin for the user without requiring users to update build.sbt. The SemanticDB compiler plugin is required for several Metals features to work correctly (example: rename symbol, find references). Bloop has custom support for Metals to enable SemanticDB even if it’s not declared in the original sbt build. Maybe it’s OK to require users to update build.sbt to enable SemanticDB, but I am concerned this would make it harder for beginners to use Metals.
  • compile progress notifications, Metals relies on BSP compile progress notifications (https://build-server-protocol.github.io/docs/specification.html#compile-notifications) in order to know when to update indexes, restart the presentation compiler, display progress bars for the user in the status bar and tree views.
  • sbt console output such as “Compiling N sources to …” for that commands that are triggered via sbt server print out to process that started sbt server. This means that sbt shell console output looks different if the user has an open sbt shell before Metals connects to sbt server, or if Metals starts a new sbt server process. This may not be a big issue, but it may be confusing for some users.
8 Likes
18

Thanks @ckipp01 and @smarter to share your concerns about this implementation because it gives me the opportunity to elaborate on this decision.

I would classify BSP endpoints in two categories. The first category is about retrieving information and most endpoints fall into it: workspace/buildTargets, buildTarget/sources, buildTarget/scalacOptions and more… The second category is about processing some task: buildTarget/compile, buildTarget/test, buildTarget/run. I would even put buildTarget/scalaMainClasses and buildTarget/scalaTestClasses in the second category because they require compilation. This is a bit too simplistic but it helps me to elaborate further.

Our concerns about BSP are perfectly legitimate but they are mostly restricted to the second category. I am convinced that the first category is much useful because it gives you the ability to extract information from sbt programmatically, with a standard protocol (no plugin to maintain and build tool agnostic).

So basically it would be possible for IntelliJ our Metals to extract the information they need from the BSP server (whether it is sbt or another one). Assuming the sources are generated, they would then be able to choose how to compile, run and tests:

  • they can do it by themselves
  • they can delegate to Bloop
  • they can delegate to the build tool

(The “assuming the sources are generated” is problematic in the first 2 options and I am starting to think that they may be an endpoint missing for that in BSP, not sure)

We can even assume that Bloop will be able to import the build from sbt by itself. So basically Metals would start Bloop and then Bloop will connect to sbt through BSP and configure itself. This would work with sbt or Mill or Fury or any other BSP compatible build tool. sbt wouldn’t know about Metals nor Bloop. Theoretically, every component would be interchangeable (Metals by IntelliJ, sbt by Mill for instance), or you can choose to use Bloop or not.

The primary goal of implementing BSP in sbt is to reduce the coupling between sbt and its clients (Bloop, Metals, IntelliJ). This is not the silver bullet but it has the benefit of opening up the possibilities, so that one can make the right decision, depending on its own requirements.

There is a third category, which is the category of notifications. It will be really nice to have those in sbt because it will keep its BSP clients updated:

  • when sbt run reload it sends a buildTarget/didChange notification to all clients so that they can re-import the build
  • when sbt run compile it sends a build/publishDiagnostics notification to all clients so that they can refresh the error highlighting and the semantics
  • similarly when testing, sbt could send test reports as notifications

So even if sbt is blocking it will keep its clients up-to-date about what it is doing.

Back to our concerns, I don’t have the perfect solution for all people. Still I think that using sbt as a BSP backend for IntelliJ or Metals is a serious candidate, depending on the developer experience you are looking for and/or the project you are working one.

Delegating to Bloop inside sbt is definitely something to consider but it is, in my opinion, a different story.

At the moment it’s too early to tell how useful it will be to have BSP in sbt. But I am convinced it will be useful to some extent for some tool. It might be Metals, IntelliJ, Bloop or another tool.

You can check out the progress in this PR. Apart from the generated sources, it is about 300 lines of code and we are already able to use sbt as a BSP backend inside Metals. Much work is yet to be done to handle all the corner cases though

3 Likes
19

Thank you Olaf for those input that are very helpful.

clearing diagnostics with low latency, we discovered that if you wait until bytecode generation to clear diagnostics then users get slow feedback when they have fixed a compile error (reported issue Feedback very slow in some cases · Issue #438 · scalameta/metals · GitHub). See https://youtu.be/MRQMylDxBJ8?t=709 for an explanation of the solution that Bloop uses to clear diagnostics sooner.

We are using an implementation of the Zinc Reporter interface to publish the diagnostics. When Zinc call resetPrevious we send a empty diagnostic notification. And when Zinc reports an error we aggregate all the errors seen so far and we publish a diagnostic. I don’t know much about Zinc internals, but I would expect that to be fast. It is verbose but fast.

enable the SemanticDB compiler plugin for the user without requiring users to update build.sbt

At the moment I do configure the build manually but in the end it should not be hard to implement the custom support for Metals. The easiest solution I can think of is to set the compiler plugin settings globally (in ThisBuild) as soon as Metals initialize the connection with sbt. So even if another client triggers a compilation it would generate the semanticDB files for Metals.

compile progress notifications, Metals relies on BSP compile progress notifications

this is nice but not required so I will work on it later

sbt console output such as “Compiling N sources to …” for that commands that are triggered via sbt server print out to process that started sbt server

Our current strategy is to send logging notifications to every client and to log everything to the shell as well. The idea is not to be silent when sbt is blocking because of doing something else.

20

The compile finished notification is required for Metals to be able to properly restart the presentation compiler (which influences the correctness of completions among other things)