Unification of sbt shell notation and build.sbt DSL

[sjrd]

@eed3si9n

A setting key or a task key can hang under any levels. That doesn’t necessarily mean that the axes aren’t hierarchical: it just means some of the values can be set to Zero. For example, key in Compile is short for key in (<current project>, Compile, Zero).

A setting key or task key can also hang somewhere that can hardly be called a level. For example key in (<current project>, Zero, compile). How do you reconcile that with the hierarchical model? Is it thisProject / Zero / compile / key?

[eed3si9n]

@sjrd

For example key in (<current project>, Zero, compile). How do you reconcile that with the hierarchical model? Is it thisProject / Zero / compile / key?

Yes, that would be the absolute coordinate of a subproject-and-task scoped key, which does occur with plugin settings. In reality, the user would write something like:

assembly / assemblyJarName := "foo.jar"

or type in:

> assembly/assemblyJarName

in the shell. I can deduce this because I am checking that “assembly” in the first segment is not a subproject id.

[nadavwr]

@eed3si9n The ‘/:’ comment was in response to @OlegYch. I meant it as a drop in replacement for ‘in’. In second thought it’s a horrible idea—describing hierarchy in reverse with ‘/:’ won’t make any visual sense.

Do people still use ‘/:’ for foldLeft? I think of it as a relic.

[jducoeur]

@nadavwr

Do people still use ‘/:’ for foldLeft?

All the time, yes. I’m always a bit mystified why it gets so much hate – I’ve always found it intuitive…

[hohonuuli]

+1 for /: hate.

Seriously, SBT has been full of these weird arcane symbols. These are thankfully being deprecated, but I vote for getting rid of symbolic method names as much as possible. After years, I still have build files with <<= in them and I have no real idea what that means.

[lihaoyi]

@Daxten

As some here already argumented, let us stop reinventing the wheel. There is already a syntax for mutable variables in scala, so why not use that?

In many ways, lots of SBT is re-inventing the wheel: re-inventing values, variables, scopes, assignments, etc… I don’t like that either. But even if we wanted dot-notation to work, I don’t see a clean way to implement it in the build files given the constraints of the current SBT implementation, and “fixing” those constraints to avoid SBT re-inventing all those things is probably a huge undertaking outside the scope of this discussion.

Chris Vogt’s CBT (Video) does tons of interesting work in trying to not re-invent stuff, in a direction you’d probably be happy with, but if you watch his talk it’s a total ground-up re-think and re-write of what a build-tool is. We should encourage such efforts, but they can happen in parallel and shouldn’t get in the way of this kind of incremental quality-of-life improvement to SBT: collapsing two distinct, idiosyncratic, equivalent syntaxes into one.

[muuki88]

Thanks @eed3si9n for your work on this.

At this point I favor the new / syntax if I get the direction of this proposal correctly.

(1) No Nonsensical scopes

@eed3si9n

Having said that, I generally recommend using in with tuple because you could write nonsensical scopes like scalacOptions in Compile in Test or something order dependent:

From my understanding the / syntax doesn’t allow arbitrary combinations of scopes, but only a well defined set. E.g.

// okay
root / Compile / compile := ...
Compile / compile := ...

// fails to compile
compile / Compile := ...

Which will get rid of various typos and confusing errors.

(2) Readability & Usability

Tab completion works more naturally IMHO and when all axis are spelled out things get really clear
where settings are applied.

I’m not sure if this is a good idea, but things like this would be possible

// scope multiple keys to a scope
Compile / Seq(
   name := ...,
   test := ...
)

But which is nice for more nested settings, e.g.

Debian / assembly / Seq (
   name := ...,
   assemblyJar := ...
)

(3) Visualizing == Using

Display all values for a key would look as natural as writing it. E.g. with some kind of “inspect scopes” command

$ sbt
> inspectScopes compile
root / Compile / compile := Defined at Defaults.scala:271
root / compile := Defined at Defaults.scala:271
root / Test / compile := Defined at Defaults.scala:271

IMHO this would look a bit strange and unreadable with the in notation.

Looking forward to see this UX improvement in SBT. Thanks a lot

[jvican]

Hey guys,

I am really enthusiastic about these improvements to the sbt UX .

I lean towards the / syntax because:

• Autocompletion works flawlessly and accomodates all the use cases that the current syntax supports now.

  The in syntax does not allow you to inspect / autocomplete keys for a subproject. For example, if you have a key in a subproject foo, you cannot do foo/<TAB> and autocomplete like you would do now – which means users need to know beforehand what keys exist in all the subproject, which is definitely not something that they know. I believe this is a very common use case, so that’s why I think in is not the candidate we’re after.

  Note that the same happens not only for subprojects but for any scope component (that is, configurations like Compile and tasks that can have other keys defined inside). In my opinion, merging the in unification syntax into sbt will render the tool unusable for most of the users since they are not aware of what’s inside what and they rely heavily on sbt to answer their questions.

• Scopes are vertically aligned, which makes them easier to read (in both sbt files and the shell output like the inspect command):

  Compile / compileInputs := { ... }
  Compile / compile := { ... }
  Test / test := { ... }
  Test / cacheDirectory := { ... }
  Test / target := { ... } 
  

• The / syntax gives scope keys the importance they deserve by being placed first. In my opinion, the in syntax makes scopes more difficult to spot and feels less natural. Making scopes more obvious is a win for clarity given their major role in the use of the build tool.

• Adding an extra scope to an existing key is easier. If we have project / key, we just place the cursor before key and type Compile /. With the in syntax, you first have key in project and to modify it we type key in (Compile, project), which is a little bit harder. The tuple is the most common format to represent complex scopes and feels more awkward than project / Compile / key.

What are the disadvantages of the / syntax? Well, it breaks 0.13.x syntax and the way people write sbt code now. If we’re to pick this one, we must ask ourselves: how are we going to improve sbt users’ life without forcing them to rewrite all their code?

To answer this question, I am working on a migration tool called sbtfix that I believe can migrate all or at least most of the uses of in to the new / syntax, both in sbt files and plugins (note that the README is outdated). I will show in the next days a rewrite to prove this claim.

So, this leaves us with one last inconvenience: people need to get used to the new syntax. I think this is the price to pay to improve the usability of any tool.

However, the good thing is that if they get it wrong and write in instead, they will not get a compile error, but a deprecation error. And if they add sbtfix to their global plugins, sbtfix will rewrite the in syntax for free. Of course, existing 0.13 sbt code in docs and StackOverflow questions will still work!

We always complain sbt is hard. This is our chance to make it easier. Sbt 1.0.x can go down in history as the release that significantly simplified the use of the tool. I say let’s be bold and take this chance.

[scf37]

IMO, best sbt devs can do is to:

• separate build.sbt DSL classes from implementation
• document them well using scaladoc

I was able to use sbt freely only after spending several days understanding object model and how it is glued together.

P.S. still have no idea how to get ProjectRef by Project and vice versa.

[eed3si9n]

Thanks everyone for your inputs.

It seems like there are stronger preference towards the slash syntax, but there are some concerns around backward compatibility of existing builds. I think we can address backward compatibility by keeping around the old syntax for the duration of sbt 1.x series.

There’s also been concerns around introducing this change late in the game, so after discussing with Dale, we decided to introduce this as a form of a plugin. This way you can start playing around with it without waiting for sbt 1, and get better feedback. https://github.com/sbt/sbt-slash

[jvican]

Hey guys,

Just wanted to drop by to say that I’m enjoying sbt-slash a lot. I highly encourage you to give it a try. Not having to deal with the shell syntax makes the sbt experience much nicer.

Try it and let us know your feedback!

[eed3si9n]

eugene yokota 🥙 (eed3si9n)

The PR for slash syntax is now merged. Here’s a sneak preview:

[jvican]

Congratulations on the bold move. I’m optimistic this will make the use of sbt much simpler.