Feedback sought: Optional Braces

Language Design
1

This thread is for feedback of what seems to be Scala 3’s most controversial feature: optional braces.

History

I first seriously considered optional braces about 18 months ago after having played with the idea for several years. At that time it was experimental; we wanted to have something that we could try out on people. The immediate setting for the experiment was the standard 2nd year course on functional programming at EPFL. During that course and after, we experimented with several versions of the syntax. We got the feedback from students, but more important for me was the feedback from my own work: Do the slides and tutorial texts look natural and as simple as possible? Is it easy to explain? Are there hidden traps? Does it have a clean specification and implementation?

In the Spring 2020 the rules were solidified, and we continued to get feedback from users. I also used it now in all of my own work. All my commits from late 2019 onwards use indentation syntax. By that time we had refined the system so that indentation and braces could be freely mixed. Consequently, the name of the feature changed from “indentation syntax” to “optional braces”. Mixing old code and indented code felt surprisingly natural. We had first expected that there would be a clash but that has not turned out to be the case in our experience. Mixing both styles had two other advantages: No global rewrite was necessary, so we could keep the git history clean. And, we got a simple unobtrusive marker whether code was new or old.

My personal experience working with optional braces was overwhelmingly positive. I encountered none of the shortcomings and problems that were predicted. And that holds over a very large range of usage scenarios: from a single tutorial slide to multi-thousand line pull requests. I can say confidently that using optional braces is the single most important productivity boost for me when switching to Scala-3. Of course, this is subjective, not everyone will feel the same. Programming styles are a matter of personal
taste and preference. But I nevertheless like to believe that when I experience a huge advantage in a way of doing things, there will be others that appreciate it as well, and this was born out by informal feedback I got. I explained this view, and the expectation that Scala 3 will have optional braces, in multiple conferences starting with ScalaSphere 2019.

I was so convinced that optional braces are the way forward that I re-recorded my two MOOCs with the new syntax. That amounted to about 20 hours of material overall. This was a large investment of my time (about 3 months overall) and even more time from several others helping me with the editing. If we had stuck with braces, we could probably have kept a large part of the old material, saving us a lot of time.

Technical Design

The rules for optional braces are described here.

To get a feeling what the code looks like, you could take a look at the Scala 3 compiler. In many places old and new are mixed, but some parts were written exclusively with indentation syntax.

Variants

Optional braces for control constructs were quite uncontroversial and did not change much.

We had several design iterations for optional braces in classes. We started with : as the marker for an indentation block, then tried (1) nothing at all, i.e. every indentation after a class signature opens a block
(2) where, (3) with. Every trial went through a complete cycle where the spec and implementation was changed, course material was changed, and we tried it out in teaching and in our own work. After extensive experimentation we decided that : was the best choice, after all. So we stuck with that.

We also had many discussions about avoiding braces around function arguments. The original design used again :. Many alternatives were proposed and debated. In the end we decided that it would be premature to settle on a solution now. A revised scheme that allows to drop braces after : is available under a command-line setting -Yindent-colons. We expect that this will be resolved at some point past 3.0, once the community has gained more experience with the existing rules. We need to debate what the best solution is here, but I’d prefer to keep that out of this thread.

By contrast, we never considered to drop braces altogether. Optional braces will stay optional, just like semicolons are still optional in Scala after 15 years. This means if you don’t like any of this, you have the option to just always write braces where Scala-2 would demand them, and this can be enforced with a linter or the compiler’s -noindent setting.

Possible Objections

If you are against indentation syntax because you had bad experiences with Yaml or Python, you might still come to like the way it’s done in Scala 3. In fact, we have solved many of the problematic aspects of indentation syntax by careful design. For instance:

  • Spaces vs tabs: solved by a new interpretation of indentation widths. Spaces and tabs can be mixed, and at the same time the system still does not require an encoding of what a tab is.
  • An accidental space can change meaning: solved by careful rules that mean a single missing or additional space can never change the meaning of a program as well as a type system that catches in practice the great majority of multi-space errors. In fact I believe the system is more robust against accidental changes in meaning than what we had in Scala 2.
  • It’s hard to find what scope gets closed when a line closes multiple indentation levels: Solved by optional end markers.
  • It’s problematic to mix different styles: this was not observed in practice. Scala 2 already has optional braces in that braces are optional around single expressions. Some people write them, others don’t. The new scheme is mostly just more of the same.

Feedback Sought

It would be good to get feedback about concrete aspects of the proposal. How can it be improved? What problems did people encounter? Are there possible fixes that avoid the problems? We are particularly interested in reports based on actual experiences working with the current version of dotty/Scala 3.

Process

I know that optional braces ended up being handled differently than other new Scala-3 features. The discussion about them was always very controversial. It was the one change where I was relying on my authority as informal BDFL to push it through by convincing enough others to go along. So if things turn out badly, I’ll be the one to blame. But I am quite hopeful they won’t.

I also apologize for having sought feedback so late in the game. We dropped the ball on this one. It was because due to Covid all SIP activity was dormant since March 11th. We should have opened this thread over the summer, but nobody thought of it then.

42 Likes
2

Due to the : delimiter, comparing optional braces and optional semicolons is a bit misleading. It’s not a matter of being able to insert optional braces, it also involves removing the :. That makes it feel less like optional braces, and more like two different syntaxes. The name optional braces therefore sounds wrong to me and in fact disingenuous, as giving something a misleading name to shape opinion. I have the impression the name of the feature is chosen intentionally to lower objections. Especially when the braces aren’t really optional – you can’t just put them in without changing anything else – this gives me a feeling of dishonestly representing the feature, and “flying it under the radar”, which causes most of my unease towards the whole situation.

In that regard, I’m also happy you’re clear about pushing it trough as informal BDFL. Doing that above the table feels much less as being bullshitted – and I’m sorry to put it in such terms, but I can’t in good conscious put it in any other – than before, when it felt obfuscated it was being pushed through.

As for the feature itself, my main concerns are about the cost of getting used to it and with that the mix of different styles. Collaborating with some who use it and others who don’t where both have strong feelings about it may be harder than the situation where you didn’t experience problems, and mixed codebases are prone to raise the time newcomers get up to speed with a codebase written in the “other” bracket style than they’re used to.

5 Likes
3

I’ve written all the examples in Programming Scala, Third Edition using the new syntax (https://github.com/deanwampler/programming-scala-book-code-examples). When I first heard about it, I was opposed to the idea, as it seemed like an unnecessary change. It will trigger the usual complaints that Scala is too complex or has too many ways of doing things, justified or not.

However, since the book is focused squarely on Scala 3, I decided to embrace the syntax and I’ve come to really like it. The code looks even cleaner than Scala with braces, which was always a strength of Scala.

I’ve also been thinking about the “message” this change signals to the broader programming community. I believe Scala is at the point where the connection to the JVM is far less important than it was. We have probably won over almost all the Java developers who are “winnable”. Kotlin is taking the rest of them willing to change. Of course Scala is now a JavaScript and a native language, too.

Python is on the upswing (again) and optional braces will feel more familiar to them. There are a nontrivial number of data scientists (Python) collaborating with data engineers (JVM), so they will either be confused by the similar-but-different qualities of Python vs. Scala, or they will find the new syntax easier to embrace, as they work together. Also, I think fans of Haskell will also like this syntax better, for similar reasons.

Hence, I’m in favor of keeping this change.

19 Likes
4

Echoing @deanwampler, I was initially wary of this change but after using it, I’ve come to really like it. Thanks for summarizing the history of this change. I’m looking forward to it being included in Scala 3.

10 Likes
5

I appreciate your explicit acknowledgment of being the informal BDFL (Benevolent Dictator For Life). It’s proven to be an effective form of leadership, and as an effective peer to consensus building. Linus Torvald’s shepherding of Linux is a great example of how powerful it can be.

Additionally, I appreciate your willingness and courage to lead into spaces fraught with resistance. That’s how OOP was married to FP in Scala. Because you were relentless in both leading and pursuing that integration.

I’ve personally been quite resistant to this new syntax. And if you had sought consensus prior to exploring and experimenting, I don’t think this would have come into being. Just like you forged ahead with OOP+FP, you have forged ahead with this syntax change. And as a result, my resistance has fallen considerably. While I now have some small reservations, I am mostly anticipating trying it out given so many people who were also quite resistant have shifted to become advocates; Dean Wampler and Alvin Alexander.

In my personal experience, most breakthroughs don’t and won’t happen with consensus. So, kudos to you for having the courage to continue to take risks around growing Scala. I deeply appreciate your having done so. And your continuing to do so.

12 Likes
6

I was hesitant about the change when it was introduced, but I’ve switched to the Optional braces syntax since quite some time for all Dotty/Scala 3 projects I’m working on.

The only gotcha IMO is when a line or block of code is accidentally shifted enough so that it is considered to be in a separate scope. If the compiler would issue a warning in such suspicious cases, that would be very useful.

The initial version of the “Moving from Scala 2 to Scala 3” course used curly braces syntax, but we switched that completely to the new syntax.

Another thing people should keep in mind is the possibility using the compiler to rewrite code between different syntax choices (and that includes the new/old control structure syntax).

So, +1 for me for the optional braces syntax.

9 Likes
7

I’ve enjoyed the syntax of Haskell and F#. I’ve enjoyed how lightweight it felt and how it emphasized the important things and removed noise.

And for the same reasons, I’m looking forward to using whitespace and indentation to do syntax in Scala 3.

I believe programming languages should be primarily for humans, not machines, and braces are for parsers.

4 Likes
8

Great to see a request for feedback. I’m a bit confused, though, whether that feedback might actually be taken into account? Or does the “informal BDFL” reference mean it doesn’t matter? And what does it mean for the SIP process?

Two concerns mostly. First: type ascriptions are often written with colon at line and type in next line. In the new syntax, such type ascriptions look just like blocks, making the code harder to read.

Second concern: methods with multiple multi-line arguments, e.g.:

def m(f: A => B)(g: C => D)(h: E => F): R = ...

to be called thus:

m { a: A =>
...
}{ c: C =>
...
}{ e: E =>
...
}

I understand there is a syntax to write the above without braces, but it is behind a flag and it looks super-ugly and hard to read.

Suggestion to address both concerns:

Why not use { where the new syntax would use a :, keep the enclosing } optional, and for end markers, instead of end S use } S?

1 Like
9

Note as of latest 2.13 and 3 releases, parens are required on single param lambdas – i.e., m { (a: A) => ... }

10

Overall I think it’s a reasonable syntax. Historically, we can look at F# and Haskell to see that when given a choice, with both brace-delimited and indent-delimited syntaxes are available, people generally end up picking the indent-delimited version. Python’s success also speaks for itself; beginners certainly don’t pick Python because of performance, ease of installation, packaging, IDE support, or simplicity of the language’s runtime semantics!

I’m generally in favor of this indent-delimited Scala, as long as:

  1. The exact syntax has time to be bikeshedded over and argued about, so we don’t get stuck long term with a sub-optimal or warty design simply for lack of discussion

  2. We find a solution to avoiding braces when calling higher-order functions with multi-line lambdas. Higher-order functions are common enough that we cannot claim to have brace-free Scala unless they’re supported, and it’s generally accepted that indent-delimited blocks and multi-line lambdas are a challenging pair of requirements to satisfy https://stackoverflow.com/questions/1233448/no-multiline-lambda-in-python-why-not

  3. We have a best-effort backwards-compatibility plan and migration plan. As Scala 3 is a big release, I expect upgrading to be painful, but hopefully indent-delimited blocks do not make it more so

I’ve a ton of experience using indent-delimited syntax: I’ve used Python regularly for my entire career, I spent years working in Coffeescript, and have done some work in F#. But as much as I like indentation-syntax, there is definitely ways you can do indent-delimited syntax badly:

  1. Coffeescript ends up making too many delimiters optional (parens and commas, in addition to braces and curlies) resulting in code that is challenging to skim.

  2. Coffeescript also commonly used two space indents, which visually make distinguishing blocks much harder than code using four space indents, more so because of the lack of explicit delimiters

  3. F# has a pretty unusual/irregular syntax overall, so despite being indent-delimited I find it much less pleasant to read than curly-heavy C# or Scala

Thus it’s not sufficient to do indentation syntax in Scala, but we also have to do it right.

Looking at the linked ExtractSemanticDB.scala file, I have the following concrete feedback:

  1. I still find the two-space indentation makes it very hard to distinguish blocks when skimming. Normally I don’t discuss formatting concerns, but given that this whole effort is about improving Scala’s surface syntax, I think this is important. After spending years reading 2-space-indented Coffeescript, I don’t think this response to 2-space indents is going to change for me

  2. The code snippet has a distinct lack of multi-line lambdas being passed to higher-order functions. I understand that those would be the “worst case” scenario for the whitespace rules as currently specced and implemented

  3. The (inconsistently applied?) indentation of case blocks within a match statement seems extraneous. With curlies it looks a bit more reasonable, but without curlies it seems a lot more awkward to have two levels of indentation where an if/else if/else chain would only have one

Given that we’ll be stuck with this syntax for the long haul we need to make sure it’s the best syntax we can come up with. While the current set of rules that are specced out is very reasonable, I do think that arguing out these last few concerns (and any other concrete concerns that people bring up) is definitely worth doing before the spec and implementation are set in stone for the next decade.

19 Likes
11

I think it’s normal for developers who are already familiar with Scala 2 to be skeptical of optional braces, in much the same way that developers like myself were skeptical of optional semicolons.

But as Kotlin and Java move closer to Scala 2.x, I think optional braces really helps differentiate Scala 3 (like a fine coat of paint on a new automobile), and, moreover, represents a doubling-down on “Scala being Scala”. Optional braces is more of what we came to Scala for: concision and beauty, without duplication and boilerplate.

I myself am teaching Scala 3 without braces, and I think the braceless syntax will eventually win over the majority of Scala 2.x developers. This despite the fact that braceless syntax has an obvious drawback (copy/pasting code doesn’t work without adjusting indentation). Yet I personally find this drawback is compensated by the benefit (layout reflects semantics, without duplication!), and the more minimal look and feel really plays to the historical strengths of the Scala language.

As Python has demonstrated, whitespace significance is, at the very least, not an impediment to adoption, even among beginning programmers, despite the very real drawback it has with copy/paste.

So for all this reason, I’m in favor of optional braces, and encourage developers to give the syntax a spin, because, as I personally discovered long ago with optional semicolons, sometimes that’s all it takes to acquire a taste for a new way of writing code.

14 Likes
12

I mostly don’t (yet) care for the new syntax, but I am delighted to have it available to me for those cases where it makes code cleaner, visually more pleasant, and/or easier to understand.

Having these kinds of syntactic options available is one of the main reasons why I find Scala code appealing to read and write. Other examples include

  • Functions need not have braces when they are a simple expression
  • for's map/flatMap block can be in parens instead of braces
  • Method names can be symbolic or alphabetic
  • Methods can be called infix as well as dotted (Um…?)
  • Function return types are optional
  • Semicolons are optional
  • = is optional in purely side-effecting functions instead of requiring a return of Unit (Uh…)
  • return is optional
  • Results of simple expressions can be used without being named (e.g. 2 * 3 + 5 is okay, don’t need { val a = 2 * 3; a + 5 })

I am delighted to add optional braces to this list (and saddened to lose the ones we’re losing).

I do not like alternative syntax when the two options clash or where the alternative admits dangerous behavior (e.g. def foo(x: X) = println(x) which suggests that foo might return something worthwhile).

But for optional braces it’s almost entirely upside with very little chance for confusion once one has a modest amount of familiarity with the syntax. I can envision using it quite a bit in contexts where the block is very short, and the brace-to-content ratio is therefore currently very poor.

4 Likes
13

Thank you for the acknowledgement of the process issues this feature has experienced, for a while it felt a lot like gaslighting, and was a deeply disorienting and unsettling experience, and it’s helpful to clear the air.

My experience with this feature has been that, the more I used it, the less I liked it. This, admittedly, may be partially due to my background, as I’ve worked in Python and Ruby prior to Scala, and this feature runs headlong into some of the least pleasant parts of the syntax of both those languages.

I don’t consider this a complete feature. This may be due to coming in before the rebranding to “optional braces”, as when I was experimenting with this it was treating it as “significant whitespace” and the concerns I encountered while evaluating it under those criteria have stuck, mostly because I haven’t really seen anything that would resolve them.

When I was experimenting with this, it hit the sweet spot of covering enough cases to look good in most of the sample code, while repeatedly running into situations where the whitespace wasn’t sufficient. I found this really detrimental to the legibility of the code. The problem is that, at least for me, it’s much harder to see what isn’t there, so blocks delimited only by indentation tended to bleed together. This was worsened when some of the blocks had braces and some didn’t, because when looking for boundaries the braces made the whitespace differences even harder to spot.

Refactoring was unpleasant, as was having to manage indentation. These are familiar problems for me from working in Python, and tooling hasn’t really improved much over “select the region and hit tab until it lines up”. Compared to “paste and hit Ctrl-Opt-L” (note: that’s my scalafmt hotkey), moving code around was much slower. Given the popularity of Python, if tooling improvements haven’t materialized by now, Scala adopting significant whitespace is unlikely to change that. Bottom line is that this moves indentation and formatting from “stuff I don’t have to think about” to “stuff I have to actively manage”, which is not generally the direction of progress.

Similar to Ruby, even when it became possible to skip the braces for function calls with the appropriate compiler flag, the code is deeply ugly when multiple parameter blocks or even method chaining come into play. Both tend to come up in my code fairly often, which is a problem for me, due to the issues intermixing the styles I mentioned previously. As an aside, I don’t have the same issues with the cases where curly braces are optional now, because they only apply when the block would contain a single expression, and those tend to be visually distinct.

Given this irregularities, I don’t see how adding this is consistent with changes like removing infix flexibility. It adds way more variability and irregularity than infix syntax, and BDFL or not, I can’t shake the impression that this is getting in because it’s new and shiny, rather than on its own merits.

Call it out as an experiment and put it behind a feature or language import, if it’s really as compelling as you think, it’ll get adopted. If not, that sets up an easy way for people to try it out, and time to figure out how to fix the places it currently underperforms. I’d be behind that 100%, because the places where this really shines for me is code that doesn’t have the characteristics mentioned above: embedded DSLs, and it would be handy to be able to opt-in on a file-by-file basis (block by block would be better, but that’s probably an unreasonable ask).

3 Likes
14

As long as the lambda-containing thing is last, all you need is to double-down on infix syntax rather than abandon it:

xs map s =>
  val n = s.length
  if n < 5 then s * n
  else s.toUpperCase

and

xs foldLeft(0) (acc, x) =>
    acc + x

The problem comes when you have multiple parameter blocks or a lambda that is part of a parameter block, because you’re mixing styles: delimited parens, but non-delimited blocks. That’s already true even with stuff like multi-line if/else

def foo(p: Int => Boolean, q: Int => Boolean) = ???

foo(
  i =>
    if (i < 189571) bippy(i/2)
    else whatchamacallit(i, i + 1) < 893475,
 j =>
    if (lovely(j)) checkIfFlower(j)
    else false
)

This is about the best one can do and there’s still an awkwardness in finding that comma that separates the two.

Anyway, I just don’t buy that multi-line lambdas are hard at all. => can just be an end-of-line block opener. The problem is that the other syntax doesn’t support them. (You end up surrounding things with parens instead of braces, and the parens were already optional with braces, so you accomplish precisely nothing.)

P.S. Want chaining? No worries, just triple-down on infix syntax:

xs
  map s =>
    val x = foo(s)
    bar(x, s, quux(x))
  filter y =>
    val z = baz(y)
    isScrupulated(z, bippy(y, z))

Maybe we don’t want to write things that way because braces or parens make things more clear, but hey, that’s why braces are optional. Sometimes they make things more clear.

2 Likes
15

Note–the parsing rules for this are a bit hairy for both machines and people. You have to consider that it might be xs.foldLeft(0)(acc, x) in addition to allowing for infix syntax of a method plus all-but-the-last parameter block. It might be better to introduce a new argument-supplying operator for clarity. Something like

xs foldLeft(0) @ (acc, x) =>
  multiple
  lines

Or

xs foldLeft(0) $ (acc, x) =>
  multiple
  lines
2 Likes
16

Scala being “differentiated” or “Scala being something” shouldn’t be what the decision is based on. It should be what the developers favors and what boosts productivity and maintainability, etc.

While it doesn’t stop newcomers to learn Python, it does stop many people from even looking at at all. The huge benefit of using Python is due to its AI and scientific libraries and easy (almost naive) syntax, and that made people to make compromise on the indentation syntax and by no means it means all Python developers like it.

10 Likes
17

Or:

xs foldLeft(0) { (acc, x) =>
multiple
lines
18

Making } optional is an extremely bad idea. It would be confusing for all users and all tooling while bringing no benefit at all.

8 Likes
19

Thanks for the feedback!

  • Two space vs more: Three spaces looks nice, four is arguably already too much. But it will be very hard to get the community to shift. I believe the syntax will work for two spaces even if might not be optimal.
  • Multi-line lambdas: Will be worked out next, I hope. It could come in 3.1 or 3.2.
  • Match layout: Note that you can write
     s match
 case A => 
   ...
 case B => 
   ...
 case C => 
   ...
 somethingElse
    The syntax allows it. Most people are used to the double indentation however.
2 Likes
20

Regarding the multiline lambdas. It is already possible to write them without braces or awkward :

xs.map(
  s =>
    val n = s.length
    if n < 5 then s * n
    else s.toUpperCase
)

For some reason xs.map( s => on one line and a single indentation level doesn’t work yet, but I don’t see a fundamental reason why it shouldn’t.

4 Likes