One thing I found quite striking in the example files given was the inconsistency in using end markers. Some huge scopes did not have them, while some smaller ones did. This is a little unfortunate, and makes reading nested class/object definitions challenging.
Would it make sense to enforceend delimiters when nesting object or class definitions? For instance, forbid
class A:
...
object B:
...
// many definitions
...
def foo =
...
...
// are we still in B here? or in A?
...
but allow
class A:
...
object B:
...
// many definitions
...
end B
def foo =
...
...
// it's a bit easier to see the `end B` above (and we can look for it)
...
Nesting in expression method bodies is mostly fine though, and wouldn’t need enforced end marks IMO.
Alternatively, I think it’d be a good idea to simply require braces for object and class definitions (at least until we find a better way, in a future version). My impression is that these braces are not the ones that are most annoying to write. And it would help a lot when reading code.
It’d be great to see the goals of optional braces conveyed concisely. I wonder if you’ll get the feedback you seek given that the goal of optional braces isn’t quite apparent.
On a personal note though, I see optional braces as being the most significant feature of Scala 3. This single feature could help position Scala as a replacement to Python for data science; a significant source of developers. So, if attracting a significant number new developers is the goal, then I believe Scala 3 will attain it (TIOBE or other forms of quantification would also be useful).
Mechanically, it appears very simple, in that particular editor. For whatever reason, IntelliJ and Metals consistently mess up the indentation when pasting (the first line is generally the most egregious), across all modes. It’s not like these aren’t really skillful and creative developers, their work on implicits is proof enough of what they’re capable of, so if it doesn’t exist already, there’s probably a reason for it.
That reason might not be technical, and if that is the case, then if Python’s user base isn’t sufficient motivation to invest the needed resources to fix it, Scala’s user base won’t be - there just aren’t enough of us.
As to selecting the pasted text being the hard part, just selecting it won’t fix that, because the first line generally gets weird indentation. That means you still have to hit tab a bunch of times (annoying and tedious as, even with my keys remapped to relocates tab to the capslock, it’s still being hit by the weakest finger), then go back to the first line and correct that one. Compared to at most paste + reformat, it’s a very poor user experience.
Note that a major difference between Python and Scala 3 is that the latter allows mixing styles.
So the copy-pasting issue has an easy solution: when you want to paste a block of code, just open a brace (the editor will usually even close it back automatically after the cursor) and then paste your code:
class A:
def foo =
// want to paste here, open a pair of braces:
{}
// paste the code block:
{
val x = 1
println(x)}
Then, an automated formatter should be able to reindent everything appropriately, and even remove the braces if that’s in the formatter’s settings.
class A:
def foo =
val x = 1
println(x)
Note that this works best if you always copy your blocks starting from the previous line-break, which I always do anyways even in Scala 2, because it avoids messing up the indentation. If you don’t want to do that, you should insert a new-line after the { before pasting your code.
Overall a big +1 from me. Having the ability to add end markers is great. Being immersed in Haskell most of the time (and the occasional python), optional braces feel totally natural and reduce the overall “noise” in the code.
The 2 confusions I had came from:
extension methods, because : is optional if only one extension method is defined
I agree–it’s very important to reject code like this.
The rule should be that you remember the literal whitespace before the indentation line, need that literal plus more whitespace on the first indented line, which you then remember, and every following line in the same enclosure level must have the exact same whitespace. I can’t tell from a glance if that’s what the PR does.
Whenever you’re in a context where indentation is not used as a delimiter, I guess you should be able to do whatever you want? Or should the lack of indentation kill the block?
def example =
val a = foo(1)
if nice(a) then
println("I need to be indented")
bar(
"But indentation here isn't needed for parsing.",
"So is this okay? Or an abomination?"
) // What about close-parens?
println("This isn't in a block. Can you tell? Does it matter?")
You guess based on the indentation at the insertion point, and leave the block highlighted so the user can fix it with a tab or shift-tab if the guess is wrong.
So if you go
if fribble then
wabble
| <-- insertion point there, not helpful
then yes, it’s ambiguous, but either way you do it wrong
Great, thanks for the post and the level of transparency and honesty in it @odersky. Really appreciate that. Feature feedback:
Overall it seems nice and well thought-out but there are some inconsistencies that jump out at me. In most cases a block is opened by a trailing colon. It seems to be such in nearly all cases except given and extension. For extension it’s no colon, for given it’s with. I have no idea what the reasoning and I’m sure there are good arguments in isolation, but overall consistency is usually much more effective in UX. A consistent language is much easier to learn, remember and use as well because you just remember one simple rule. Could we change
given [T](using Ord[T]): Ord[List[T]] with
def compare(x: List[T], y: List[T]) = ???
extension (xs: List[Int])
def second: Int = xs.tail.head
to be
given [T](using Ord[T]): Ord[List[T]]:
def compare(x: List[T], y: List[T]) = ???
extension (xs: List[Int]):
def second: Int = xs.tail.head
The given case as it currently stands is doubly inconsistent because I see a case in the code links where a new refined class uses : too.
private val symsAtOffset = new mutable.HashMap[Int, Set[Symbol]]():
override def default(key: Int) = Set[Symbol]()
Indenting the blocks that follow match and catch is optional. I really think we should choose one or the other. (I’d prefer no additional indentation.) Or is it the plan to try both and then later settle on one?
Could we please get some more clarification around the then keyword in if statements, when it appears at the end of a line. In the doc I’m seeing
if x < 0
-x
else // error: `else` does not align correctly
x
where as in the code I’m seeing
if ctx.explicitNulls && ref.isNullableUnion then
if rhstp.isNullType || rhstp.isNullableUnion then
Is an EOL then optional? Like above I think it’d be valuable to only have a one way of doing it.
Love the optional end markers
The Dotty compiler can rewrite source code to indented code and back
^^ Very awesome
Also, I haven’t had much of a strong opinion on this feature either way but after reading and understanding the motivation and framing (which I’d misunderstood before), and after going through the doc properly, I’m convinced. I think this is going to be pretty good.
I wrote a quite big chunk of code in scala3 and can say, that now I semi-automatically delete extra braces in code before porting/reading.
Few notes:
-Yindent-colon is optional. But when somebody use this, and like this, he/she start to write chunks of code in githib gists, forums, etc … Now, when somebody trying to use this gist, will discover that code is not compiled. Ie. Cut & Paste for Scala code not works out of the box(!). This can be a source of frustration for novices. [need to know, that code is depends from compiler feature, and that enabling this feature will not break existing code].
When I use if as expression, I time to time want to write something like:
val myVal = if (condition)
firstVariant
else
secondVarian
(else is situated on the same line as if. But this is impossible, so I use braces).
Discovered, that I don’t like extra ‘then’ and prefer to write conditions in braces instead.
That’s because the editor’s auto-indent clashes with the literal-mindedness of paste.
I’m sure we can think of ways this could be addressed.
I’m also unconvinced that “because it doesn’t exist, it won’t, because sociology” is a compelling argument. Scala used to not have good syntax highlighting in any IDE. Something happened.
I’m not sure whether to take this comment seriously. You presumably use the shift key all the time, which is biomechanically similar. it’s got to be annoying and tedious to hit shift all the time.
def templateStatSeq(): (ValDef, List[Tree]) = checkNoEscapingPlaceholders {
var firstStatIndent: Option[IndentWidth] = None
....
while (!isStatSeqEnd && !exitOnError) {
setLastStatOffset()
val indent = in.indentWidth(in.offset)
if (firstStatIndent.isEmpty) firstStatIndent = Some(indent)
else if (strictIndent && firstStatIndent != Some(indent))
syntaxErrorOrIncomplete(s"unexpected indent: found $indent expected ${firstStatIndent.get}")
else ()
....
Since my PR does this at parsing time of templateStatSeq(), for now it’s “you can do whatever you want” within the expr. All it cares is that b in bar(... has 4 whitespaces before it like the line above; and two whitespaces before if and last println.
Python looks like it’s doing similar thing by distinguishing logical lines and physical lines.
Not completely because the compiler can’t read minds. When you paste at the end of a block, and your cursor isn’t indented on a fresh line to the level you mean to paste into, nobody can tell whether the paste belongs at the end of the indented block or at the beginning of the unindented part after the block. It’s just inherently ambiguous.
But an editor should be able to handle the paste if your cursor is properly indented to begin with. Adding this as a mode should be pretty easy even at the editor level. If you have Metals or something running to give you cleverer parsing, you could do better.
We could spec out the compiler-independent rules pretty easily:
A multi-line selection is an indented selection if the following conditions hold:
Every line following the first starts with the same leading whitespace as the first line (todo: decide whether empty lines are elided)
The selection start precedes the non-whitespace text on the first line
The selection end captures either all or none of the non-whitespace text on the last line.
An insertion point is an indentable insertion point if the following conditions hold:
The cursor has as much whitespace to the left of it as there is before the text on either the last non-whitespace line above it, or the first non-whitespace line below it.
The rest of the line is empty, or non-whitespace text starts immediately to the right of the cursor.
If an indented block is pasted into an indentable insertion point, then the common leading whitespace from every line in the block is replaced by the leading whitespace before the indentable insertion point. (Incomplete whitespace on the first line is okay–however much is there will be replaced by the whitespace before the insertion point.)
If an indented block is pasted into an empty line which is not an indentable insertion point, automatic indentation rules (from the preceding non-empty line) are used to guess an indentation and the indented block is re-indented to this new level, but this new insertion is left selected.
In any other case, the indented block is inserted literally with no changes. Optionally, a warning icon may register that an indented block has been pasted into a non-indentable context.
Following these rules will make indentation management really easy: you can grab a bunch of lines by dragging down anywhere in whitespace, and then paste them just by making sure the cursor is in a sensible position for an existing block.
If it’s not sensible, you get a highlighted block and can indent it yourself.
It is tempting to reason that since Python is popular, Scala will become more popular if it is more similar to Python, but I don’t think it works this way.
Typically, software engineers use languages like Scala, Java, C++, C and JavaScript while analysts use languages like Python, Perl and R.
Among these, JavaScript is an outlier: it is used, because it is the only thing that runs inside your browser.
Leaving JavaScript aside, a clear pattern emerges: software engineers typically use statically typed compiled languages, while analysts use languages dynamically typed interpreted languages where the “Hello, World!” program is one simple line.
Python has a clear distinction between statements and control structures. In Scala, it is considered a powerful feature that you can create new expressions that look just like native control structures.
Most analysts will feel that Scala is unnecessarily complex, because they just don’t need most of Scala’s features.
Actually, even some engineers use Scala for complex software and Python for smaller jobs. I know of one case where an engineer wrote a system that consists of small Python scripts that perform some data munging and software written in Scala to orchestrate the whole thing.
So, the idea that Scala will replace Python doesn’t seem quite likely.
It’s less “because sociology” and more that I trust that the folks at JetBrains are (at least) reasonably competent, and from the outside it appears there’s fairly compelling motivation for this to be easier already. That means that, more likely than not, there’s more going on than is visible from the outside, so I don’t think it’s reasonable to assume that a solution will appear just because the motivation for it becomes incrementally more compelling.
The shift key requires a slight curl down to reach from the home row, while tab is a slight stretch to the side (or a big one to the side and up, if you are using a vanilla keymap), which are not biomechanically similar.
I’ve flirted with RSI long enough to be very conscious of this sort of strain, and don’t particularly find, “consider paying money to make it comfortable to work with the new syntax” a particularly compelling argument
I agree with @etorreborre. I prefer the indentation syntax, and the option to provide end when a block gets long. I think it would be more consistent if the extension syntax always required a : rather than having two approaches, if possible:
It feels a little confusing to have two approaches for this. It feels weird asking for fewer options, but for whatever reason, it’s easier, especially in this case.
Enforce : for extension with only one extension method in the block to make it uniform
It could be even more important to educate users on recommended styles. Prescriptive examples in common contexts (short functions, long functions, nested blocks, single line, multi line lambdas, where curly braces are still the best, etc.) would help if communicated effectively and lead to a distinct and pleasing Scala visual style.
Indentation tokens are only inserted in regions where newline statement separators are also inferred: at the toplevel, inside braces {...} , but not inside parentheses (...) , patterns or types.
It prevents code like
xs.map( s =>
val n = s.length
if n < 5 then s * n
else s.toUpperCase
)
from being interpreted as
xs.map( s => {
val n = s.length
if n < 5 then s * n
else s.toUpperCase
})
Which one wouldn’t write with braces (you’d just use {} instead of () around the entire lambda), but in a world where braces are optional it seems perfectly reasonable. And without the above rule it should just work.
PS I’m not really sure which rule makes this compile in spite of the above rule:
xs.map(
s =>
val n = s.length
if n < 5 then s * n
else s.toUpperCase
)
Here are already some replies to specific remarks, Please keep them coming!
In fact the latest iteration drops the : entirely for extension methods. The thinking is that we should not have a choice here and collective extension methods are not like classes or objects since they do not open a nested scope. So there is a reason not to write a : at this point. And we can afford to not use the : since something must follow an extension parameter. That’s the main reason why we require : after objects and classes. If we would drop it then any indented code after a class A would become the class body. That means a single stray space changes the meaning of a program and it also makes it harder to see visually what is a class body and what isn’t.
Goot point, we should document that! For others who have not come across this before: self types would go on a new line:
class A:
this: B =>
...
Tooling concerns: I also have observed that tools could do auto-indentation better. I believe what I would want is this:
When inserting a block of code immediately after some non-empty line, follow the indentation on that line, or indent if it is a token like = or => that opens an indentation region.
When inserting a block of code after some empty line, keep the indentation width of the original block.
That avoids an annoying effect where I just want to copy methods around but the copied method
gets indented relative to the last statement of the previous method. Also, quadruple-clicking could select the currently indented block instead of the whole file. Others on this thread have worked out much more detailed recipes, which might work even better. I still have to wrap my head around them. Otherwise, I have not found editor support lacking.
For other tools like scalafmt: These get a token sequence that would include <indent> and <outdent> tokens instead of { and }. So they can work exactly as before.
I should mention that the files were not in any sense curated; I just picked two random samples. We certainly want to work out guidance when to use and end marker. My current recommendation, which is in the docs and also in my course, is to use an end marker if
the closed block has empty lines, or
the closed block is long, where what “long” means is flexible (somewhere between 10 and 20 lines), or
the closing indentation jump is at least 4.
Another requirement I toyed with is to always use an end marker for classes and objects. But all that is subjective, and others might have other preferences. So I believe we need more experience, and at least at first it’s better to put these things in formatters and style-checkers before we decree it in the language.
I believe that is problematic as well, since if you translate it back to braces
what you have is an abstract given of a refinement type. We had a long discussion about this issue in the PR that changed the given syntax to not use as. The decision was we need something heavier than just braces/:.
Could we please get some more clarification around the then keyword in if statements, when it appears at the end of a line. In the doc I’m seeing
I just realized that the published doc is not updated yet. There’s a pull request but that has not been merged yet. I’ll fix that ASAP and that should take care of it.
