In an effort to improve the getting started with Scala experience and also enhance the average developer’s workflow while using Metals we’ve decided to spend some time and re-evaluate the decision Metals makes to default to Bloop as your build server.
A little history
Historically Bloop has been a fantastic choice for Metals enabling support for multiple build tools like sbt, Mill, Gradle, and Maven. That paired with compile-times that have yet to be matched, made it an easy choice to default to it. Plus, it was the first build server implementing BSP. It’s impact and importance can’t be understated in proving that BSP can be an effective protocol for building, testing, and running your code.
The current state of things
With that being said, BSP is sort of an implementation detail for the user and one that an average developer shouldn’t have to concern themselves with. Over the years in Metals we’ve seen a myriad of issues about being unable to connect to the build server, issues getting gnarly sbt builds to export to Bloop, or general confusion about the entire flow. We’d like to further lean into the idea that BSP should be an implementation detail and one that users don’t often need to think about, while still allowing users to make power decisions like switching their build server.
As we’ve seen more BSP implementation pop up, like the work done in sbt (Build Server Protocol in sbt) and also in other tools like Mill, we’ve seen the confusion level rise around switching build servers as it becomes more popular. For example, details like BSP discovery start surfacing due to the nature of how Metals defaults to Bloop without following the rules of discovery. While very informal and anecdotal, there’s a non-insignificant percentage of people that seems to be automatically switching to their build tool to serve as their build server (mainly sbt and Mill). This is backed up by the very common recommendation we see of “have you tried with sbt as your build server” when things go awry. This can be due to wanting a closer picture of your actual build for things like source generation or custom configs, or better editor support in your build file, which your build tool serving as a BSP server often is better at than Bloop. Because of this we’ve decided to roughly address this topic as follows.
Roadmap
Do some initial research about the current state of BSP servers and their level of support when using Metals. This comparison has actually already been done and you can view the results here. In this process we’ve also started to create issues in sbt/Mill to address any shortcomings.
After creating the issues and understanding the shortcomings better we’ll see if we’re able to tackle them along with the maintainers of the various tools ensuring that the default level of support in Metals for users doesn’t change when we switch the defaults.
We’ll make the necessary changes in Metals to update the default choice and better follow BSP discovery. In reality what this will mean is that when an sbt user opens an sbt project that has an existing .bsp/sbt.json file, it will just default to using sbt as the build server.
While diving into the above we’ll also have a couple other goals in mind.
When utilizing BSP, how can we get someone from opening a new project to being able to code as fast as possible. There are some noticiable differences here with build servers, so we’d like to address this and ensure that you get an accurate picture of your code as quickly as possible.
Metals isn’t alone in this problem of various ways to get started editing your project as IntelliJ users also are often faced with the choice of how to import their project, as an sbt project, as a Bloop BSP project, or a sbt BSP project. Our hope is that the work we do here solidifying the BSP choice and defaults will not only benefit metals, but also other tools utilizing BSP.
Our hope is to complete this work in Q1 of 2023.
As always, don’t hesitate to reach out to us with questions or comments. They are more than welcome.
Yep, I confirm I have trust issues with bloop/metals, and for complex projects I gave up and just used sbt in a separate terminal. For medium projects I resort to deleting .bloop and .metals folders every now and then. When I’m giving projects to my students I say to them "if you’re getting weird errors in the editor, first make sure to run sbt compile in the terminal before attempting a long debugging session. This really needs to be resolved.
Yep, I confirm I have trust issues with bloop/metals
The first step is acceptance @soronpo, so we’re making progress.
Jokes aside, do you remember what sort of issues you were having? Did you by chance create any issues when this happened that I could take a look at?
I gave up and just used sbt in a separate terminal
This is for sure the type of thing we hope to avoid, so if you hit on this again, please do create an issue or even just ping me and we can try to hunt down what is causing everything to get out of wack.
OK, I must admit it has been a while since I last tried it. If my students run into this again, I’ll request they zip the entire directory structure and open an issue with it. For my own developments, I’ll also try to give it a second go and see how long before it breaks.
I am afraid i have to second soronpo. I use Sublime+Metals for development but the final build is done by SBT from the command line. I don’t know if the actual final/clean build by Metals would be bad, but i occasionally see:
errors marked in the code by the editor that are not true errors (compile by sbt is fine)
bsp claims the code is error free when it is not (but it notices it when the window containing the error is opened)
LSP | finding references is incomplete, so i always check using ‘find in all files’
without internet connection, LSP is completely dead
sometimes it just refuses to compile and you need to completely clear all temporary files (it’s not that often, but often enough to have a script for that)
So my trust is not big enough to let BSP make the final build. It could be of course that i make mistakes in the way the system should be used; but to just have sbt along side gives me the ease at mind i need to focus on my work.
Don’t get me wrong however, i am very pleased indeed with the support bloob/metals gives in Sublime. Despite these small annoyances, it’s great to work with. A very big “thank you” to all the developers.
In general I’m very happy with metals and neovim. It is mostly stable though I can see often troubles if the build changes. It would be nice to get a notification when build.sbt is broken be because there are syntax errors or dependencies are not available.
One extra thing that keeps me attached to sbt is that (I think) via metals/bsp it is not possible to produce a scala.js output. We are doing our UIs with scala.js and I need to produce the js output very often.
In general I’m very happy though it maybe nice to simplify the experience for first time users. it can be confusing to learn about sbt/bsp/bloop/metals etc…
My hope is that if sbt becomes the default for sbt builds, then the situation both of you mention will become less. Instead of relying on sbt for you final compile, you’ll have just been relying on sbt the entire time. Hopefully this will provide a smoother experience.
Could you provide some more information on this? Is this something you can reproduce and minimalize to create an issue out of? Even if it’s just something you can reproduce in an open source repo we can totally take a look at it.
This totally shouldn’t be the case. If everything has been resolved beforehand, there should be no issue using Metals when offline. Can you provide some more details for this and create an issue?
This is actually something that is being worked on in Bloop and that we’re aware of in sbt. This is a good reminder to try and get these across the board.
In your specific scenario have you been hitting on it when using sbt as your build server or Bloop?
I think the experience here is nicer when using sbt as your build server as you’re actually tied into the task graph, so depending on your build, this should actually happen using using sbt via BSP. On the Bloop side, this is way trickier. We’d need a way to export these types of things via sourceGenerators from your build definition to Bloop, but I’ve personally found this quite clunky when having an sbt or Mill driven build, and therefore it’s not implemented.
I think that would be great in theory, but in practice this can be a source of troubles since if I run some tasks like console in sbt, the server will stop responding to messages. So https://github.com/sbt/sbt/pull/6074 would need to be revived at least.
Yes, I tried to make it clear in the comparison that this is one of the biggest weakpoints imo of the sbt BSP implementation. Unfortunately it’s also probably harder to solve than any of the other weaknesses listed in the BSP tools.
While it’s for sure a weakness and people are aware of it, it’s surprisingly not one on the Metals side that we often hear complaints about. At this point, for me anyways, that issue isn’t enough in itself to block making sbt the default. I think we’d want to hear that concern echoed more widely before we viewed it that way.
Default Bloop i guess. Sorry, but I don’t really know what “sbt via BSP” means. I see the whole tool a bit as a black box that “just works” (most of the time at least). I installed Metals on Sublime Text and thats it. Bloop appeared. I installed SBT separately (and long before).
I must be more precise here, i should have said: "LSP | finding references is sometimes incomplete. Must of the time it is just fine. But it made me a bit nervous, since i cannot predict when this happens. Therefore, when i need to be certain, i verify with ‘find in all files’. Now, this behaviour is very hard to reproduce and disappears if I open the file that contains the missing parts. So I have the feeling it is a caching problem of some kind.
This totally shouldn’t be the case. If everything has been resolved …
Yes everything is resolved (at least i think so). Here is an example: First screenshot Sublime on some code with internet connection (after ‘indexing is complete’):
As you can see, no LSP menu present any more. LSP logpanel is empty. Now, i did not yet do any research on this problem, maybe it is just some setting or so. So i am not saying it is a bug, it’s just ‘an experience’
In my case I will be using sbt no matter what IDE I use, because it’s a companion tool that describes my project. I’ll use it to have custom commands and tasks that make sense for my projects, as well as running tests.
Having the terminal window embedded in the IDE-like window (vscode, intellij, what have you) is just a no-go, since it’s a small rectangle inside that window and doesn’t play nicely with multiple monitors setup. This is not a problem metals can solve, the terminal belongs in its own thing (in my case yakuake).
So with all these considerations, having the IDE spend time compiling only to then also have to compile on sbt to run tests and the like, is a waste of time and also a big waste of RAM. This last bit is not negligible, as with 32GB of ram I can barely run two metals projects at the same time with little to spare.
(I’m aware that in a way, if I were to not use SBT, then I’d have just one bloop server for all my metal projects, and that would save RAM, but again, using SBT is non negotiable)
I’ll reach out to the person that has spearheaded the Sublime work and ask about this since it’s for sure not a pure Metals thing. Testing in both Neovim and VS Code, there is no issues using Metals offline.
Actually I think we’re talking about something similar, but just in different ways. I’t apparent that using sbt as your source of truth is important, since as you mention, it ties in your task system, is the source of truth for your project, etc. When using sbt as your build server while using Metals, you’d be benifiting from this exact thing. When using Bloop you’re not always 100% sure that it will behave the same way as a sbt compile whereas when using sbt as your build server, you can be pretty dang sure that the Metals compile will be the same as your sbt compile as it will be using sbt.
So with all these considerations, having the IDE spend time compiling only to then also have to compile on sbt to run tests and the like, is a waste of time and also a big waste of RAM.
Again, in the outlined situation of using sbt as your build server there would be no need to recompile. If you project is fully compiled with Metals while using sbt as your build server, running sbt compile would actually be a noop compile since you’d be sharing cache.
close Sublime, disconnect internet, restart Sublime
It is because the plugin looks for the latest stable version by default in https://scalameta.org/metals/latests.json
The quickest fix is to set server_version in the plugin settings manually to avoid the http request. It was probably a bad idea to make this the default behavior, I will fix that.
bsp claims the code is error free when it is not (but it notices it when the window containing the error is opened)
This is a limitation from sublime LSP plugin, it can only report about opened files due to some sublime API limitation I believe. It probably also make sense to report the issue to discuss possible fixes with the maintainers.
