Build Server Protocol in sbt

smarter · May 7, 2020, 6:10pm

Right, I was a bit too quick to characterize this as “low-hanging”, sorry. It seems that a proper integration between sbt and bloop which keeps them in sync at all times will require to do much more than sbt-bloop currently does.

curoli · May 7, 2020, 6:37pm

Blocking means that SBT will only be read to respond to another request after it has completed responding to the previous request. Of course, the communication is bi-drectional (request - response). It means that when SBT is working on a long-running request, other things have to wait. For many tasks, users expect a quick response and will be frustrated to wait.

hepin1989 · May 8, 2020, 7:51am

Thanks for you reply, very clear:)

olafurpg · May 8, 2020, 9:39am

Thank you for sharing this update! I’m excited about this project and I’m looking forward to try it out when it’s ready.

Identified Problem

Another issue that users end up re-compiling their codebase twice to run non-compile/test workflows such as publishLocal, docker and sbt-revolver.

Identified Risks

Here are some additional challenges you may face,:

clearing diagnostics with low latency, we discovered that if you wait until bytecode generation to clear diagnostics then users get slow feedback when they have fixed a compile error (reported issue https://github.com/scalameta/metals/issues/438). See https://youtu.be/MRQMylDxBJ8?t=709 for an explanation of the solution that Bloop uses to clear diagnostics sooner.
enable the SemanticDB compiler plugin for the user without requiring users to update build.sbt. The SemanticDB compiler plugin is required for several Metals features to work correctly (example: rename symbol, find references). Bloop has custom support for Metals to enable SemanticDB even if it’s not declared in the original sbt build. Maybe it’s OK to require users to update build.sbt to enable SemanticDB, but I am concerned this would make it harder for beginners to use Metals.
compile progress notifications, Metals relies on BSP compile progress notifications (https://build-server-protocol.github.io/docs/specification.html#compile-notifications) in order to know when to update indexes, restart the presentation compiler, display progress bars for the user in the status bar and tree views.
sbt console output such as “Compiling N sources to …” for that commands that are triggered via sbt server print out to process that started sbt server. This means that sbt shell console output looks different if the user has an open sbt shell before Metals connects to sbt server, or if Metals starts a new sbt server process. This may not be a big issue, but it may be confusing for some users.

adpi2 · May 8, 2020, 12:08pm

Thanks @ckipp01 and @smarter to share your concerns about this implementation because it gives me the opportunity to elaborate on this decision.

I would classify BSP endpoints in two categories. The first category is about retrieving information and most endpoints fall into it: workspace/buildTargets, buildTarget/sources, buildTarget/scalacOptions and more… The second category is about processing some task: buildTarget/compile, buildTarget/test, buildTarget/run. I would even put buildTarget/scalaMainClasses and buildTarget/scalaTestClasses in the second category because they require compilation. This is a bit too simplistic but it helps me to elaborate further.

Our concerns about BSP are perfectly legitimate but they are mostly restricted to the second category. I am convinced that the first category is much useful because it gives you the ability to extract information from sbt programmatically, with a standard protocol (no plugin to maintain and build tool agnostic).

So basically it would be possible for IntelliJ our Metals to extract the information they need from the BSP server (whether it is sbt or another one). Assuming the sources are generated, they would then be able to choose how to compile, run and tests:

they can do it by themselves
they can delegate to Bloop
they can delegate to the build tool

(The “assuming the sources are generated” is problematic in the first 2 options and I am starting to think that they may be an endpoint missing for that in BSP, not sure)

We can even assume that Bloop will be able to import the build from sbt by itself. So basically Metals would start Bloop and then Bloop will connect to sbt through BSP and configure itself. This would work with sbt or Mill or Fury or any other BSP compatible build tool. sbt wouldn’t know about Metals nor Bloop. Theoretically, every component would be interchangeable (Metals by IntelliJ, sbt by Mill for instance), or you can choose to use Bloop or not.

The primary goal of implementing BSP in sbt is to reduce the coupling between sbt and its clients (Bloop, Metals, IntelliJ). This is not the silver bullet but it has the benefit of opening up the possibilities, so that one can make the right decision, depending on its own requirements.

There is a third category, which is the category of notifications. It will be really nice to have those in sbt because it will keep its BSP clients updated:

when sbt run reload it sends a buildTarget/didChange notification to all clients so that they can re-import the build
when sbt run compile it sends a build/publishDiagnostics notification to all clients so that they can refresh the error highlighting and the semantics
similarly when testing, sbt could send test reports as notifications

So even if sbt is blocking it will keep its clients up-to-date about what it is doing.

Back to our concerns, I don’t have the perfect solution for all people. Still I think that using sbt as a BSP backend for IntelliJ or Metals is a serious candidate, depending on the developer experience you are looking for and/or the project you are working one.

Delegating to Bloop inside sbt is definitely something to consider but it is, in my opinion, a different story.

At the moment it’s too early to tell how useful it will be to have BSP in sbt. But I am convinced it will be useful to some extent for some tool. It might be Metals, IntelliJ, Bloop or another tool.

You can check out the progress in this PR. Apart from the generated sources, it is about 300 lines of code and we are already able to use sbt as a BSP backend inside Metals. Much work is yet to be done to handle all the corner cases though

adpi2 · May 8, 2020, 12:42pm

Thank you Olaf for those input that are very helpful.

clearing diagnostics with low latency, we discovered that if you wait until bytecode generation to clear diagnostics then users get slow feedback when they have fixed a compile error (reported issue https://github.com/scalameta/metals/issues/438). See https://youtu.be/MRQMylDxBJ8?t=709 for an explanation of the solution that Bloop uses to clear diagnostics sooner.

We are using an implementation of the Zinc Reporter interface to publish the diagnostics. When Zinc call resetPrevious we send a empty diagnostic notification. And when Zinc reports an error we aggregate all the errors seen so far and we publish a diagnostic. I don’t know much about Zinc internals, but I would expect that to be fast. It is verbose but fast.

enable the SemanticDB compiler plugin for the user without requiring users to update build.sbt

At the moment I do configure the build manually but in the end it should not be hard to implement the custom support for Metals. The easiest solution I can think of is to set the compiler plugin settings globally (in ThisBuild) as soon as Metals initialize the connection with sbt. So even if another client triggers a compilation it would generate the semanticDB files for Metals.

compile progress notifications, Metals relies on BSP compile progress notifications

this is nice but not required so I will work on it later

sbt console output such as “Compiling N sources to …” for that commands that are triggered via sbt server print out to process that started sbt server

Our current strategy is to send logging notifications to every client and to log everything to the shell as well. The idea is not to be silent when sbt is blocking because of doing something else.

olafurpg · May 8, 2020, 12:51pm

The compile finished notification is required for Metals to be able to properly restart the presentation compiler (which influences the correctness of completions among other things)

RichType · May 8, 2020, 1:02pm

I don’t know if this the right place to put this, but one thing I’ve wondered for a while, is if it would be possible to do an incremental build and and a build from scratch in parallel at the same time. I’m sure these days most developers have got idle cores sitting around nearly all of the time.

adpi2 · May 12, 2020, 9:38am

The compile finished notification is required for Metals to be able to properly restart the presentation compiler (which influences the correctness of completions among other things)

I have tried to reproduce the limitation you are mentioning but without success. Maybe the json-rpc response of the buildTarget/compile do restart the presentation compiler.

Do you have in mind a scenario in which the completion would not be correct?

olafurpg · May 12, 2020, 10:25am

The relevant code in Metals is here

github.com

scalameta/metals/blob/c5c3a90f2594b72c720813dfff4b3f5306d76a8a/metals/src/main/scala/scala/meta/internal/metals/ForwardingMetalsBuildClient.scala#L144




@JsonNotification("build/taskFinish")
def buildTaskFinish(params: TaskFinishParams): Unit = {
  params.getDataKind match {
    case TaskDataKind.COMPILE_REPORT =>
      for {
        report <- params.asCompileReport
        compilation <- compilations.remove(report.getTarget)
      } {
        diagnostics.onFinishCompileBuildTarget(report.getTarget)
        didCompile(report)
        val target = report.getTarget
        compilation.promise.trySuccess(report)
        val name = buildTargets.info(report.getTarget) match {
          case Some(i) => i.getDisplayName
          case None => report.getTarget.getUri
        }
        val isSuccess = report.getErrors == 0
        val icon =
          if (isSuccess) clientConfig.initialConfig.icons.check
          else clientConfig.initialConfig.icons.alert

github.com

scalameta/metals/blob/c5c3a90f2594b72c720813dfff4b3f5306d76a8a/metals/src/main/scala/scala/meta/internal/metals/Compilers.scala#L202


def didCompile(report: CompileReport): Unit = {
  if (report.getErrors > 0) {
    cache.get(report.getTarget).foreach(_.restart())
  } else {
    // Restart PC for all build targets that depend on this target since the classfiles
    // may have changed.
    for {
      target <- buildTargets.allInverseDependencies(report.getTarget)
      compiler <- cache.get(target)
    } {
      compiler.restart()
    }
  }
}


def completionItemResolve(
    item: CompletionItem,
    token: CancelToken
): Future[CompletionItem] = {
  for {
    data <- item.data

Here’s an example scenario where you may be able to reproduce an issue without task finish notifications.

Step 1. Trigger completion in B.scala, which is in a different project than A.scala (for example main and test)

// src/main/scala/a/A.scala
package a
object A {
  def foo: Int = 42
}
// src/test/scala/b/B.scala
package b
object B {
  def bar = a.A.foo // completion here shows type Int
}

Step 2, save the following diff, let the compilation succeed

// src/main/scala/a/A.scala
package a
object A {
-  def foo: Int = 42
+ def foo: String = ""
}

Step 3, trigger completion in B.scala again. It may still show Int when it should show the updated String signature. It’s important that B.scala is in a different project than A.scala because otherwise there’s a chance that the presentation compiler may already have updated the signature for a.A.foo because Metals reuses the compiler instance within the same project.

I’m not 100% sure this scenario reproduces the issue, but in general Metals needs to compile finished notification to tell the presentation compiler to load fresh symbols from the classpath. Otherwise users risk seeing stale signatures in completions, parameter hints, type at point, etc.

adpi2 · May 13, 2020, 4:03pm

Thank you Olaf for the detailed explanation! It behaves exactly the way you described.

olafurpg · May 13, 2020, 5:16pm

Do you mean that you’re able to reproduce the issue or the completions work as expected (show the new String signature)?

adpi2 · May 25, 2020, 12:50pm

Hello everyone,

You will find a detailed status of the BSP implementation in sbt in this post.

The implemented endpoints are:

build/initialize
workspace/buildTargets
buildTarget/sources
buildTarget/dependencySources
buildTarget/scalacOptions
buildTarget/compile

The compiler diagnostic notifications, as well as the compilation taskStart and taskFinish notifications are also implemented.

adpi2 · June 25, 2020, 12:14pm

Partial support for the BSP protocol has been shipped in sbt milestone 1.4.0-M1. And so it is now possible for any BSP client, Metals and IntelliJ for instance, to connect to sbt server.

Both Metals and IntelliJ are used to relying on Bloop for importing an sbt project. Bloop is a build server that manages compilation, test and run of several workspaces concurrently. It supports BSP and much more.

The combination of Bloop and IDEs has proved to be reliable and efficient. However, the integration with sbt is still burdened by the injection of a plugin (sbt-bloop). For this reason we decided to implement BSP in sbt.

In this post I propose you try to import your project in Metals or IntelliJ by connecting to the sbt server directly, using BSP.

The good aspect of it is that the IDE uses the exact same compiler than you do when you run sbt compile, thereby:

you are immune to inconsistencies (false error reports)
you avoid duplicated compilation which consume resources
when you run sbt test or sbt run your code is already compiled

Metals

Metals is a BSP client which means that it relies on a BSP server to delegate compilation, run and test.

By default Metals connects to Bloop. Here we want to prevent that because we want Metals to connect to sbt server instead of Bloop.

Close VS Code and remove the .bloop folder, which contains the Bloop configuration files. You can optionally remove the project/metals.sbt file that is not needed anymore.
Open VS Code again. Metals will propose you to import the build. Deny it by clicking on Not now.
Set sbt version to 1.4.0-M1 in the project/build.properties file.
You are good to start the sbt shell.

Once sbt is loaded you should see a log saying that sbt server started. You can also check that a .bsp/sbt.json file has been created. This file contains the information that Metals needs to connect to sbt.

In the Metals panel click the Connect to build server button and see the magic happens.

Metals sends several requests to sbt to extract the build structure out of it. You can see that in the sbt shell as well as in the Metals log (View > Output then choose Metals in the dropdown list).

Open a source file and start coding.

Each time you save a file, Metals sends a compilation request to sbt. Every compile error or warning is reported by sbt to Metals so that it can highlight them to you.

About code navigation

Code Navigation in Metals relies on the semanticdb files that are generated by the semanticdb compiler plugin. You can enable this plugin by setting semanticdbEnabled := true in your build.sbt file.

Do not forget to reload sbt and to restart Metals.

BSP discovery

Since the .bsp/sbt.json file exists, Metals will always try to connect to sbt. This is not very convenient because it starts an sbt process in the background that you cannot interact with. I recommend to always start sbt in a shell before opening VS Code.

IntelliJ IDEA

The IntelliJ Scala plugin offers the ability to import a Scala project using BSP. Like Metals, it is used to relying on Bloop and that’s why it creates the Bloop configuration file.

In the following tutorial I use IntelliJ IDEA version 2020.1.2.

Set sbt version to 1.4.0-M1 in the project/build.properties file.
Start the sbt shell.

Once sbt is loaded you should see a log saying that sbt server started. You can also check that a .bsp/sbt.json file has been created. This file contains the information that IntelliJ needs to connect to sbt. We are ready to go.

We need to import the project from scratch, so remove the .idea folder.
Open IntelliJ in the project folder (in Linux or Mac OS you can run idea . command) and choose BSP.
Don’t bother if it creates the Bloop configuration files, it will then connect to sbt and not Bloop.

In the sbt shell you should see the requests that IntelliJ sent to extract the build structure. After that you are ready to start coding in IntelliJ.

You can improve the experience by customizing the BSP settings. Go to File > Settings > Build, Execution, Deployment > Build Tools > BSP. Uncheck export sbt projects to Bloop before import and check build automatically on file save.

On file save, IntelliJ will use sbt to compile the changes and it will report the compiler diagnostics sent back by sbt.

BSP discovery

Since the .bsp/sbt.json file exists, IntelliJ will always try to connect to sbt. This is not very convenient because it starts an sbt process in the background that you cannot interact with. I recommend to always start sbt in a shell before opening your IntelliJ project.

Jasper-M · June 25, 2020, 1:40pm

That’s not very convenient. If I run sbt for a project in VS Code I usually do so in a shell in VS Code.

tgodzik · June 25, 2020, 2:04pm

The exact user experience will most likely be fixed. There is still some more work to be done on that front.

adpi2 · June 25, 2020, 2:47pm

That’s not very convenient. If I run sbt for a project in VS Code I usually do so in a shell in VS Code.

I agree this is not convenient at all and we have to work on it.

One solution would be to delegate the task of lauching the sbt shell to the IDE. But I don’t think this is good solution for the long term. My expectations are for the thin client mode to become the solution. So that you can have one sbt server running in a terminal or in the background and that you can connect to it from a terminal in your IDE.

ckipp01 · June 25, 2020, 6:35pm

Really neat to see the progress on this. Great work!

eyalroth · June 26, 2020, 12:18pm

This is great to hear!

The work here can also provide the much needed reference for more build tools on how to integrate with BSP.

jasticE · June 26, 2020, 6:08pm

Thanks for this writeup! First tests will IntelliJ look good!
We’ll soon be working on a better sbt integration in the IntelliJ Scala plugin based on BSP!