Meanings of underscore (including wildcard imports)

[soronpo]

@sjrd

It’s worth pointing out that @lihaoyi’s suggested plan for 2021 is exactly what we have done … in 2020!

Nope. The SIPs are not documented anywhere and the community build is rather small in comparison to Scala 2.

[soronpo]

@odersky

We will improve the language after 3.0, but we will not be able to do fundamental syntax changes anymore, simply because all the docs and tutorials, including printed books and online courses have to stay relevant for a long time. So after 3.0 we are back to the more settled standard way of language evolution, which means:

• don’t break existing code on a large scale (except for things that we have announced for removal today, so they won’t be in the Scala 3 docs)
• add new capabilities only
• concentrate on more advanced and specialized use cases.

So if something is confusing today, we cannot decide to remove it once 3.0 has shipped. We can add stuff, but I fear that will not make anything less confusing.

I think there is another option. If there are things that you want to change post 3.0, just make a note in the books and courses that “syntax XXXX may be subject to changes in a later Scala version”. Languages evolve all the time. If the tutorials made for Scala 3.0 now will reflect “only” 95% of the language post 3.1, I don’t see this as a problem.

[martijnhoekstra]

I’m sympathetic to the pragmatic argument that backticking the * is cumbersome, especially in quick scripts.

But the workaround that’ll work in the vast majority of cases is not backticking it and allowing it to mean the wildcard including the *.

For those few cases where you may not import some other member but need the * so you must backtick, itll be marginally more annoying, bit its plausible we already spent more time discussing it than the total amount of time and effort it’ll cost to put in the back ticks for everyone who will hit this situation.

[odersky]

@Ichoran

I think it’s too late for this kind of change. It’s not even the worst of the _ weirdnesses that remain (the others are just harder to fix).

What are the others, in your opinion? (except : _*, which has already been addressed).

[mbloms]

@odersky

I would probably still keep “ as _ ” for an import exclusion. Having a separate except clause is nicer but also more complex, syntactically. It would give rise to new feature interactions, so it would be more risky to do, in particular at this late stage. So I’d rather go with

  import p.{a as _, *}

because its rules follow directly from the combination of renaming imports and wildcard imports.

Haskell uses the soft keyword hiding for this. It’s very intuitive. While I generally prefer to derive things like this from existing syntax, I think that syntax helps newcomers greatly! It’s not really necessary to remove the existing syntax. Both of these could be allowed:

import p.* hiding a
Import p.{a as _, *}

It would make things like this much easier to write and for code reviewers to understand what it means:

import p.* hiding {a, b, c}

[odersky]

hiding is nice. It’s worthwhile to consider but since it is an addition to existing syntax it can also come in after 3.0.

[Jasper-M]

IMHO none of these suggested changes to imports are objectively that much better than the status quo that it would be worth it to change this after >15 years. The only thing about imports in Scala that is kind of confusing is {name => _, _}. But that’s not extremely common anyway, and something like hiding could still be added in the future.

[odersky]

There’s now a PR for the new syntax.

Impressions:

Changing wildcard imports to * is visually a BIG change since it is so pervasive.

Advantages:

1. It aligns with the common convention in other languages and shell scripts and is therefore immediately understandable.
2. It removes one of the aspects where Scala is different, but for no good reason. Hopefully, there are very few of these left.
3. It removes confusion about what _ means. I agree that there’s a way to rationalize it as “fill in the blanks”, but then one should also have kept the wildcard in ListBuffer[_] to mean “could be any type”. We got rid of that so that _ in using position now always means “to be filled in by function argument”. Removing _ as a wildcard import strengthens that message.
4. It avoids the meaning clash in import p.{a => _, _}. There might be other ways to achieve that with hiding but that would have to come later (and there is quite a bit of work to do to consider all the interactions and syntax variations).

Disadvantages:

• A lot of program lines are affected, which will cause friction.
• Cost of re-learning syntax.
• Need backticks if * is imported as a name (but in practice that one is quite trivial as an annoyance).

By contrast, changing “=>” to “as” is easier to do, since renaming imports are less frequent than wildcard imports. The biggest win in practice is that we can now write renaming imports without braces. That turns out to make them much more pleasant to read and write.

import scala.annotation.threadUnsafe as tu
import scala.collection.mutable as mut

I believe that’s a great improvement, because we want to encourage renaming imports. They are a very effective way of combining concision and clarity when referencing external definitions. Renaming imports seem to be not used that much in Scala, maybe because the syntax was not that intuitive.

Could we have the braceless renaming syntax without changing => to as? I think that would be more problematic.

import scala.annotation.threadUnsafe => tu

looks a bit weird. Certainly not as self-explanatory as with as.

Summary: as is a clear win. But if we touch import syntax, maybe go all the way and do * as well?

Answering the question: can this come later? I think not. I am pretty sure that for the next 10 years there won’t be a chance to change something as common as wildcard imports. Add new stuff like hiding, yes. Change fundamental syntax, no way.

Unlike many other changes we have made, none of this is problematic in the sense that we would need more experience to decide. We know exactly what programs will look like, and they will behave in exactly the same way as before. Also, no code has to be rewritten until later in the 3.x series, since for now both syntaxes are supported (but the docs would make it clear what the official one is, and code can already be rewritten automatically).

[robstoll]

Can we add a deprecation warning to wildcard imports using _ and remove it in 3.1 or 3.2?

[LPTK]

While we’re at it, why not replace the cryptic x @ <pattern> pattern bindings syntax by <pattern> as x, as was discussed some time ago? Many people found it a lot more natural than the now defunct x as <pattern> experiment.

[odersky]

@robstoll

Can we add a deprecation warning to wildcard imports using _ and remove it in 3.1 or 3.2?

Eventually, we’ll do that, but only after 3.0. We need to be able to cross build between 3.0 and 2.13.

[odersky]

<pattern> as x makes more sense now, since it aligns with imports. But I think it can wait and must wait. People were not unanimous about what’s better. One good argument for x @ <pattern> was that it makes it easy to switch between x: C and x @ C(...). So I think we need to have more discussions.

Also, since pattern bindings are not that widely used, it should be possible to change it later in the 3.x series if we decide we want to do this.

[Ichoran]

@odersky

@Ichoran

think it’s too late for this kind of change. It’s not even the worst of the _ weirdnesses that remain (the others are just harder to fix).

What are the others, in your opinion? (except : _* , which has already been addressed).

My impression–maybe I’m wrong–is that partial function application is still done with _. So we still have f(x, _) just not f _. This is super confusing with anonymous lambda parameters: .map(_ * f(x, _)) isn’t what anyone thinks it is. If that has been addressed, then I don’t think anything else is more confusing.

Equally confusing, I think, is that we have “use the obvious” vs. “this isn’t named”. xs.map(_ => 5) and xs.map(_ + 5) both use _ but in completely different senses.

Then, I don’t think imports are weird mostly because of _. We have that import foo.{x => y, _} isn’t the same as import foo.{x => y}; import foo._. That thing in braces is weird when it has a _ in it, and it’s not _'s fault. It some sort of unique named-set-construction syntax, because that last _ doesn’t mean everything, it means everything that you haven’t already talked about. * doesn’t help this issue. This weirdness is orthogonal. It’s really more like a pattern match:

foo.foreach{
  case x => import y
  case _ => import _
}

except the parallel with a pattern match is better when you use _ than when you use *.

So, anyway, I don’t see that changing _ as the wildcard import is much of a win. It does change things in a weird corner of language syntax, and it does make it more like other languages, but even after the change it’s still weird. (Weirder in some ways.) Maybe it’s a little better, but I think it’s far from the only wrinkle left. If it was the only one, then maybe it’d be worth it.

[soronpo]

Is it possible to import a named given while renaming it?

[lihaoyi]

@odersky

By contrast, changing “ => ” to “ as ” is easier to do, since renaming imports are less frequent than wildcard imports. The biggest win in practice is that we can now write renaming imports without braces. That turns out to make them much more pleasant to read and write.

IMO this isn’t really worth the effort in renaming it: imports have always both had a someone special syntax (The .{ syntax for multi-imports, => binds a name on the right) and simultaneously just aren’t a point of confusion for people. Yes they are weird, no it’s not a problem in practice.

There are probably ways we could brainstorm to make it “better” or “clearer”, but if we open up the broader design space it would take more discussion to reach agreement as to what “better” or “clearer” mean to everyone, and as many have said we should be beyond the brainstorming-and-ideation phase at this point.

My original proposal (as you can see from the title of the thread) was to try and address the concrete issue that over-use of _ is a point of confusion. If we’re not going to do that then we should drop the idea for now and revisit these more exploratory discussions in some 3.x release as the language syntax inevitably evolves over time.

[odersky]

With Scala-3, we have tried everywhere else to be as clear as possible, and to break with existing syntax where it was necessary to do so. Even for simple things. I believe

import java.util as ju

is clearer and more legible than

import java.{util => ju}

And if we are making braces optional elsewhere, why not here?

[curoli]

Imports without braces? What about:

import a.b.c.{ d, e, f }

And I’m still very curious what people have against braces. The most clear and efficient way of delineating something is to put it into a matched pair of delimiters, i.e. parens, braces or brackets.

[odersky]

@lihaoyi

(The .{ syntax for multi-imports

That one actually makes a lot more sense with * since

x.*
x.{a, b, c}

are both well-known conventions from file specifiers. It’s just requiring {...} around some forms of single imports that makes it look strange.

[rjolly]

@Ichoran

and it does make it more like other languages

But again other languages do not necessarily allow importing symbols, so they do not suffer the symmetry breaking that we do.

[lihaoyi]

@odersky

And if we are making braces optional elsewhere, why not here?

Because this proposal is not really making import syntax better or more coherent, it is…

1. Changing the syntax only for renaming imports, which are probably <1% or so of all imports in a typical Scala codebase. Most imports are straightforward “import this under its own name”
2. Changing the syntax only for renaming imports which are alone, which is probably <10% of all renaming imports (a large number are import foo.{x => _, _} as discussed earlier, and another large bulk are going to be in a group of imports because you don’t need renaming unless you are in a large codebase with lots of names, and those typically import lots of stuff, with multiple names per import)
3. Changing the syntax for a part of the code nobody really has a problem with (how many times have people complained about single-identifier-renaming import syntax in the last decade? I’m quite active in the community, and have heard ~zero complaints)
4. Adding a special case for these very-rare single-identifier-renaming imports, making them less regular and less consistent (albeit more syntactically lightweight) with all the other imports which heretofore nobody has proposed changing

This proposal is to add a special case to make 0.1% of imports less consistent with the rest of the language, solving a problem nobody really has. Doesn’t seem like it’s a good ROI to me

We could come up with something to improve imports syntax in general, there are no shortage of things that can be tweaked to be more coherent (e.g. Why does it use curlies+commas? Nothing else in Scala uses curlies+commas together… it’s always curlies + inferable semicolons or parens + commas) . But that would take a longer discussion, what has been discussed in this thread isn’t it, and the Scala 3 release has a self-imposed deadline to meet.