Proposed syntax for _root_


#89

It seems that the Rust team has added the nested import groups to 1.25 that @MasseGuillaume was lobbying for.

// on one line
use std::{fs::File, io::Read, path::{Path, PathBuf}};

// with some more breathing room
use std::{
    fs::File,
    io::Read,
    path::{
        Path,
        PathBuf
    }
};

#90

seriously, how do I unsubscribe from this thread?


#91


#92

Hi, has anyone worked on a SIP?


#93

I’d be interested in discussing a SIP (as I missed the discussion the first time round).

I’d like to add the following points:

I’ve been by people working on Scalac and Dotty that the compile-time cost of supporting relative imports is disproportionate to their value. The reason, as briefly explained to me, modulo my recollection, is that in a language with only absolute imports, the process of naming is a quite trivial lookup in a limited number of imported scopes, all of which are fully-qualified, whereas permitting relative imports requires some amount of typechecking to be completed in order to resolve names, and naming and typechecking become much more interleaved.

I agree, therefore, with @olafurpg’s suggestion that absolute imports should be “the default” and relative imports should come with a “syntax cost”. The claim that there’s a compile-time cost for relative imports is very much consistent with the comprehension cost for the programmer.

I think that the syntax _root_ is very ugly, and should be removed from the language. There’s no other precedent anywhere in Scala for giving something a name surrounded by underscores, and it is an anomaly. It would be great to replace it with something more consistent with the ergonomics of Scala. Historically, IIRC, it was chosen so as to not require the introduction of a new special keyword (root) and to not be likely to clash with existing keywords in use. Dotty doesn’t have so many qualms about introducing new keywords, so I don’t think this applies any more.

My preference would be to deprecate _root_, and to make absolute imports default, and import _.foo seems like perfectly reasonable syntax for relative imports.

If we took this approach, there’s no reason why we couldn’t have a long transition period where both syntaxes are supported, before rewriting the compiler to take advantage of these promised compile-time speedups…

I broadly support the idea of being able to nest imports, not so much because I think it’s going to be really useful, but because the Principle of Least Surprise says they should be composable.

Tangentially, I heard a suggestion somewhere, some time ago, that Scala would be more understandable if imports did not introduce implicits into scope, unless prefixed with the implicit keyword. For example, you would require implicit import scala.concurrent.ExecutionContext.Implicits.global if you wanted to have it available in scope to use implicitly. I have no strong feelings about this. (I think it would be a radical change with both advantages and disadvantages, which would more or less balance out…) But I raise it here in case other people want to discuss it.

That’s all I’ve got.


#94

John Pretty Jun 1 '18

I think that the syntax _root_ is very ugly

I agree, and I think aesthetics may matter more than one may expect. For me the appearance of Scala programs was one of the reasons to start working in it.

My preference would be to deprecate _root_, and to make absolute imports default, and import _.foo seems like perfectly reasonable syntax for relative imports.

I agree; but since a lot of arguments have been given here pro and contra I give some quotes:

olafurpg Sep '17

I think absolute imports should be encouraged with lightweight syntax while relative imports should come with a syntax tax. Absolute imports are inherently easier to reason about.

lihaoyi Aug '17 and Sep '17:

If you use package.foo.Bar.doSomething(...), the package keyword makes it abundantly clear you’re not using a local symbol. In fact, it is clearer than having some import 1000 lines up at the top of the file and then trying to figure out which Bar you have in scope to call doSomething with.

I use libraries fully-qualified all the time

I agree with olafurpg and lihaoyi here. But nafg wrote on Sep '17:

(…) absolute vs. relative syntax should be consistent between import and expressions is not just convenience or elegance, it’s a lore more fundamental than that to the language. (Not saying it’s mandatary , but there’s a strong case for it.)

I think this case is not that strong, since import expressions are quite different from value expressions. As an import expression a.b.c would preferably denote an absolute path while as a value expression it would be allowed to be relative. In comparison, I find the case for keeping dots as infix operators in expression syntax much stronger.

So I prefer to have both

  • _. for relative imports
  • package. for absolute value expressions

About the roadmap:

pagoda_5b Mar 14 '18

Then what about going full-in and add the underscore for relative imports?

olafurpg Mar 14 '18

That would make it impossible to use relative imports in sources cross-built across 2.12-2.13/2.14. Ideally, any changes should be 100% backwards compatible and not prohibit cross-building.

Alternatively we could postpone the change to version 3.

John Pretty Jun 1 '18

If we took this approach, there’s no reason why we couldn’t have a long transition period where both syntaxes are supported, before rewriting the compiler to take advantage of these promised compile-time speedups…

IMO better: have a compiler switch for which the default value will change at some future version. This way the speedup would become early available.


#95

Thanks for the summary, Andre! Scala 3 has a “Scala 2 mode” which could be the means of transitioning between different syntaxes.

Also note that package objects are currently compiled to an object whose name is package, and it’s currently possible to refer to it with,

`package`

so something else might need to change to support this. I also don’t like the idea of using a keyword as an identifier. It’s already done with this and super, but at least in those cases, those keywords are not used in other positions.


#96

A few miscellaneous thoughts:

Are we talking cost in terms of compiler complexity, or in terms of speed? If there is a major compiler-speed win to be had here, that’s worth thinking about.

That said, while I think it’s plausible to make absolute the default for imports, I am much more skeptical about that being the case for values, and we seem to be overlooking that aspect. The language is currently consistent that I can do both:

import scala.collection.mutable
import mutable.Map

and:

import scala.collection.mutable
...
val myMap = mutable.Map

I can see the first going away, but really not the second. I suspect that the supporting the second would require keeping most of the compiler cost, but I’m open to being told otherwise. More generally, this is my problem with the idea of changing the syntax of relative imports: it makes them inconsistent with the syntax elsewhere.

Agreed. While I don’t think it’s terribly critical, it would be a nice enhancement to the language. (And would reduce the pain of getting rid of relative imports.)

I’m pretty skeptical that this actually makes much difference in terms of comprehensibility – implicits are a mental hurdle to get over, but this hasn’t come up often in my experience as an additional one. And given how important imported implicits are for typeclasses, I’m very leery of messing with it, unless we wind up with a typeclass approach that doesn’t require it…


#97

Hi Mark!

I should be cautious about passing along vague promises about compiler performance, as (at the very least) it would require some significant compiler refactoring to reap the supposed benefits, but the claim is that the compiler would be less complex, which would present itself as a performance enhancement.

Regarding values, it wouldn’t be necessary to stop supporting your second example in order to benefit from the reduced complexity in the compiler. The logic goes that every directly-referenced identifier in a given scope would have to resolve to an identifier in a (probably small) set of absolute imported scopes, none of which should require (much) typechecking to resolve, as compared to relative imports which might require typechecking of the symbol being imported from in order to know what’s inside it. In theory, that’s a tiny cost-saving which accumulates for every single directly-referenced identifier being resolved.

I think the benefit from my last point (I don’t want to call it a suggestion… :wink: ) is for those cases where you have an implicit being used, and you have no idea where it’s coming from. If you could reduce the number of imports which could be introducing it to just a couple, that could save a lot of time for checking. You could test it by removing the implicit prefix from one of them and seeing if it makes a difference, for example…


#98

I agree that val myMap = mutable.Map cannot go away. And it would be terrible if we invented different name resolution conventions just because an expression is used in an import. That would fly in the face of Scala’s language design principles. So, that sinks the whole proposal, I am afraid.


#99

@odersky Don’t we already have a disparity in that expressions can contain unstable paths, while we can’t import from them?


#100

So then one would have to write:

def foo(ctx: Context) = {
  import _.ctx._ 
  ???
}

This looks silly, but the main point I would like to make is: how would I intuitively parse the first _? The only other place where you would see this syntax is in lambdas (list.map(_.foo)). So is it a placeholder? A placeholder for what exactly? ctx is a “fully qualified name” here; it has no prefix. Or would that mean that import ctx._ is still legal? That only relative imports from non-local names receive the weird syntax? That would kind of defeat the purpose though, because then I can still for instance shadow package scala with a local val.


#101

@odersky Don’t we already have a disparity in that expressions can contain unstable paths, while we can’t import from them?

I don’t really see how this compares. Some expressions need to be stable – not just imports but also prefixes of type selections or pattern expressions or prefixes in a new. That’s a restriction. It’s not a change of the basic name resolution rules depending on what context you are in.


#102

@Jasper-M
Yes, that example does look odd, and I’m not particularly fond of it, though my suggestion was also to discourage relative imports. FWIW, I still prefer seeing this to _root_ anywhere, in purely aesthetic terms.

I’ve always thought of the underscore as meaning “everything, something, anything or nothing”, or “whatever”. Its interpretation is highly context-dependent and it’s certainly in keeping with Scala’s philosophy for _ to be go-to syntax for cases like this. Or, to argue that another way, adding another overloaded use of _ when it already has about eight makes less of a difference to the language than introducing a new keyword, or reusing an existing keyword which previously only had a single use… However, in this particular case, _.ctx._, you have two underscores, with different meanings, more or less juxtaposed, which (I think) is unprecedented, so it probably counts as a new level of confusion.

Anyway I don’t want to mislead anyone into thinking that I’ve got any particularly strong opinions on any of this. I’d like to see alternative experiments, particularly if they come with some other benefits like faster compile times.

@odersky
It’s a moot point. I like that I can glance at Scala code, and if I see a.b anywhere, I know invariably that it’s referring to a member called b inside an term-level entity called a. And yes, there’s definitely value in knowing that a resolves uniquely in the current scope. My example of the case where a is unstable was an example of somewhere we accept (because it’s sensible to do so) a slightly weaker global invariant: we can’t say that import a.b._ imports the members of the result of the expression a.b; we have to constrain the universal “context-free” interpretation to stable paths. But I’m not going to flog this dead horse any longer. :slight_smile:


#103

It seems like on the one hand, we want to encourage people to use absolute imports whenever possible, but on the other hand we don’t want to change the meaning of existing code, nor impose verbose changes everywhere.

It sounds to me like using a leading dot to denote an absolute path comes the closest to satisfying all the objectives. It’s not an eyesore, it’s intuitive because it’s analogous to how filesystem paths work, and it’s not hard to see people using it as the default everywhere (if we were to strongly encourage it). At the same time we don’t have to change the meaning of existing code; paths without a leading dot would continue to be relative.

Whether it should be possible to denote an absolute path in arbitrary expressions, other than import, is a separate question (although if anyone is using root outside of imports, then an important question). There most conservative approach would be to disallow it (for the time being anyway), the next most conservative approach would be to allow it but require parentheses around it so the dot doesn’t attach it to the previous word, and the least conservative approach would be to only require parentheses when there is ambiguity (without parentheses such cases would have to keep their meaning unchanged).


#104

Although this doesn’t need to be a deal breaker I think we should be aware that in python:

import .pkg

imports pkg relative to the containing package (leading dot means “parent” roughly). Where as,

import pkg

imports pkg as an absolute name (the default). We should be careful of having the exact opposite semantics in Scala.


#105
import .pkg

looks feasible. It’s a pity that it clashes with Python’s convention, but I also would not see it as a deal breaker. I’d in that case also be in favor of allowing leading . in parenthesized expressions. I.e.

(.scala.Predef)

instead of

_root_.scala.Predef

#106

There was already a summation for folks just back from Tahiti, but I had the pleasure of re-reading this thread this morning. I was disappointed the emoji proposal went nowhere.

Here is my summary. I have arbitrarily preferred package to dot for absolute paths, for reasons of Haoyi.

  1. selection from package denotes top-level package, a zero-length path prefix

  2. import selectors can be arbitrary stable paths, arbitrarily nested

  3. request imports from enclosing package using syntax package p with import

  4. brace selection is rewritten to import from stable path. Braces can be parens for simple expressions.

  5. scoped imports limit the import qualifier

Just the gist, ma’am.

The gist shows current and proposed syntax.

Besides absolute package, it includes optional lead underscore for relative, import[mypackage], import tree syntax, stable import expressions x.{ f(); y } and explicit import from enclosing package package p with import.

Apologies in advance if I left anyone out. These are also what I would go for, exclusive of emojis and the shrugging guy.

Bonus points for import[_] p._.

I don’t address implicit import, implicit val _, for (implicit _ <- g) or other valuable locutions.


#108

I would have used settings.{ language.enable(languageFeatures.postfixOps) } recently.


#109

After seeing all the suggestions here, I still think using a leading dot (e.g. import .scala.collection.immutable) for absolute imports is the simplest and most consistent way to solve the original problem. Using a keyword like package to me is less appealing than _root_ since package has other meanings in Scala and is more verbose.

The fact that it’s different from Python doesn’t bother me. Python chose to go against the standards set by DNS, file system paths, etc., and Scala is already rather different from Python in numerous other ways.