Relative scoping for hierarchical ADT arguments

I’m gonna chime in with my experience with this syntax in somewhat similar language. I use this syntax very frequently in Lean. It’s a pretty common case that I have two enums with cases of the same name. Without the syntax that Lean allows, I would have to add a prefix (like in haskell) or use the name of the qualified name. If you have an enum for each step of some data processing pipeline (like a compiler for example), you would want adjacent data models both in scope. Also, when you can have nested structures(again like in compilers) the noise really adds up. Here is some lean code that I recently wrote that would be much more verbose without this kind of syntax:

def num_head: Term → Term := λ t => .Lam (.Lam t)

def zero: Term := num_head $ .Var ⟨0⟩

def iterate (fn: α → α) (n: Nat): α → α := match n with
  | 0 => id
  | .succ n => fn ∘ iterate fn n

def five: Term := num_head $ iterate (.App (.Var ⟨1⟩)) 5 $ .Var ⟨0⟩

def Term.apply (f: Term) (args: List Term) := match args with
  | [] => f
  | .cons h t => (Term.App f h).apply t

def add: Term := iterate (.Lam) 4 $ (Term.Var ⟨3⟩).apply [
  .Var ⟨1⟩, (Term.Var ⟨2⟩).apply [
    .Var ⟨1⟩,
    .Var ⟨0⟩
  ]
]

def mul: Term := iterate (.Lam) 4 $ (Term.Var ⟨3⟩).apply [
  .App (.Var ⟨2⟩) (.Var ⟨1⟩),
  .Var ⟨0⟩
]

def let_chain (chain: List Term) (last : Term): Term := match chain with
  | [] => last
  | .cons h t => .Let h (let_chain t last)

def ex: Term := let_chain [
  five,
  add,
  mul,
  (Term.Var ⟨1⟩).apply [.Var ⟨2⟩, .Var ⟨2⟩],
  (Term.Var ⟨1⟩).apply [.Var ⟨0⟩, .Var ⟨0⟩],
  (Term.Var ⟨2⟩).apply [.Var ⟨1⟩, .Var ⟨0⟩]
] (.Var ⟨0⟩)

Hi,
I really like this feature — it lets us design a UI Component API in a style similar to SwiftUI, which feels both clean and elegant. I’ve been experimenting with an approach that comes very close. For example:

Button(variant = _.Primary, size = _.Large, icon = _.Home)

This gives us a SwiftUI-style API, and the best part is that after beta reduction all the lambdas disappear.

Don’t know if anyone has brought this up, but zig has had enum literals for ages, and got decl literals at the start of this year.

Enum literals are basically the same as what is suggested here.

no one mentioned the backslash, which should be quite accessible with a keyboard without any modifier keys:

val redCircle = Shape(\Circle, \Red)

could be ugly, not sure, but it is associated with “paths”.

Of course in ObjC and Swift, they also have KeyPath literals e.g. \.Foo which is basically a “selection literal” to access Foo member (think monocle in Scala).

so repurposing that in Scala:

val redCircle = Shape(\.Circle, \.Red)

Edit: I forgot that \ is a valid identifier so again probably ambiguous

I think this is extremely minor. It’s two key presses, which already is the case for many operations in Scala (” ??? < > + * _ & | ! () {} ), it’s already much less typing than writing out the names of enums which this replaces, and I would say that while we do want to pick a symbol that is easy to type occasionally, readability is at least 10x more important than writability.

2 Likes

On yesterday’s SIP-meeting we discussed this and @soronpo was encouraged to see if a soft modifier e.g. export on members of companion objects based on target type could provide an opt-in relative scoping that is not so surprising and does not require use-site extra syntax. This way it is up to the api-designer to unlock it. Such as:

trait Base
object Base:
  export def Available = new Base{}    // api designer unlocks relative scoping

hypothetic scala> def test: Base = Available     // no need to say Base.Available
def test: Base

hypothetic scala> test
val res0: Base = Base$$anon$1@32f2de5c

And as proposed by @odersky for enum the case members could have implicit export soft modifier so that the cases are directly available if the target type is the enum.

These are initial ideas from the meeting, but they need to be fleshed out and checked for corner-cases and show-stoppers…

I’m not sure the added ambiguity is worth it:

trait Base
object Base:
  export def Available = new Base{}    // api designer unlocks relative scoping

// Possibly in a different file or even project:

val Available = ???

def test: Base = Available // Is this the export or the val ?

Of course we can make it unambiguous to the compiler by prioritizing one over the other, but I think it would still lead to confusion

1 Like

The final SIP I intend to write will have a leading token. What will it be, we’ll see. Currently # is in the lead.

3 Likes

Well, if we should try to fix the issue with cases of enums requiring import/export then having a special unprecedented syntax may not be preferred, There is precedence for companion object members automatically getting into scope with givens values. With the export soft modifier it’s opt-in.

I’m think that a symbol such as # may be both aesthetically displeasing and cryptic. If we can find rules that makes it “obvious” and non-ambiguous based on the explicit expected type then I think less “symbol sallad” may be preferred, esp. to beginners.

1 Like

That’s not necessarily a good thing
Being opt-in also means it’s not always available !

The syntax available being dependent on the definition, not even of the method being called, but of each member of the companion to the parameter type seems like it would be confusing !

1 Like

It looks awfully jarring. I would suggest, since we’re not using this for block openings or ranges, ... The first dot, conceptually, means “yeah the thing that makes sense in this context”. The second dot and whatever follows is just normal indirection. So it’s equivalent to sugar of .Base or whatever.

Honestly, I avoided T#U notation in part because the hash was so striking, more than because it was unsound. Also, given that # is a common comment character, things after # take on a vague hint of being commenty. So in my direct-style error-handling, .? jumps out with error like Rust, but ?# "message" jumps out with a comment. This makes the comment-feeling work with the syntax rather than clashing.

Finally, there is no chance, I think, that we’ll need .. for capabilities. But I think # might be a good thing to be able to reach for, where it would feel a lot more natural as a “yes, it’s that type, but here’s a (compiler-enforced) comment about capabilities too”. Maybe ^ has got everything under control, but I don’t think the capabilities syntax is as nice as it ought to be yet.

3 Likes
  1. I think an opt-in keyword is a worse approach than a leading symbol that works for normal nested classes automatically as well as enums.
  • I will have to write export everywhere, which lessens the advantage of not having to write out namespace prefixes, you are replacing one tedious ceremony with another, albeit smaller, tedious ceremony.
  • even if export is automatic for enums, it won’t be for other encodings like nesting classes/objects within companion object hierarchies like the Shape.Geometry.Triangle example. This creates more inconsistencies and arbitrary trade offs between different constructs in the language: “I want inheritance, so I should use classes. want multiple inheritance, so I should use traits. I want value equality, so I should use case classes. I want easy lightweight construction, so I should use named tuples. I want to be able to loop through the cases at runtime, so I should use enums for .values. I want to split definitions across files, but I also want the types to be sealed for exhaustive pattern matching". Each construct offers capabilities that the others don’t, often for arbitrary historical reasons about the compiler/jvm. It becomes a game of choosing which construct sacrifices the least amount of ergonomics and boilerplate. This opt-in system adds one more conflicting desire to the list - “I want to get implicit member target assigning for free, so I should use enums.”
  • That inconsistency is going to make the language less guessable, especially for beginners. Many pre-existing APIs won’t have the export enabled, it won’t be obvious from callsites when you can and cannot use this notation, which means like the explicit infix modifier drama, many teams will commit to never using the feature if they cant rely on it to be consistently available.
  1. I think @Ichoran’s proposal of .. is great. It intuitively looks like ellipses, the dot makes sense because you are accessing a member which we use dot notation for already, and it is closer to Swift’s leading . notation so programmers familiar from that language will feel familiar with it here as well.
1 Like

Thanks for the list of trade-off. I see your point. And I definitely think .. looks better than # and I agree with the rationale for why it is more intuitive than hash and as it is illegal currently with double dots it should not affect semicolon inference and indentation syntax etc

So perhaps we should summarize the pros and cons of introducing relative scoping as a general feature without opt-in based on special syntax such as double leading dots.

The more conservative and limited option is to only make enum cases available in scope if the enum type is the target type.

I just deleted a post because a markdown file copy-pasted here does not work anymore. Syntax highlighting is completely broken for me. Does anyone have the same experience? Or, even better know how to fix this?

I am in favor of a special modifier and against a leading sigil, for the following reasons.

First, look at the common case. The pain that caused this discussion is enums, because they can define a great number of constants and constructors that need qualified access. Wildcard imports or exports of enum companions are not a good solution here since they import or export not only the enum cases but also everything else in the companion including compiler generated members such as values. So, here’s the status quo for a typical function:

def changeTrafficLight(current: Color): Color = current match
  case Color.Red => Color.Yellow
  case Color.Yellow => Color.Green
  case Color.Green => Color.Red

This is annoyingly repetitive and we want to fix that. That’s the main objective. Every solution has to be measured by how well it addresses this objective and everything else is a bonus feature. People lived without relative scoping until now and there were no complaints before enums were added to the language.

Now, let’s look at the proposed alternatives. Among all the proposed sigils I like .. best, but it’s still problematic:

def changeTrafficLight(current: Color): Color = current match
  case ..Red => ..Yellow
  case ..Yellow => ..Green
  case ..Green => ..Red

This is shorter but looks weird, particularly for someone who does not know the language. I would argue that a non-expert reader would prefer the first solution even though it is longer.

But there is an alternative that is clearly better than both:

def changeTrafficLight(current: Color): Color = current match
  case Red => Yellow
  case Yellow => Green
  case Green => Red

That’s exactly what one would expect, no? And all the counter arguments can be debunked or addressed:

  1. Isn’t this ambiguous? No it isn’t. We have many existing rules for name disambiguation: inner definitions beat outer ones, definitions beat imports, local definitions beat inherited ones, everything beats non-imported members of an enclosing package. This would add one more rule that says that enum cases are resolved with next to lowest priority. I still would make non-imported members of packages come last. This is consistent with the way we treat implicit scopes: givens that are provided by a type are only considered if there are no other givens already in scope. So in summary, I am confident we can find reasonable rules for disambiguation.

  2. But isn’t this too unspecific for readers? When you saw the changeTrafficLight example, did you have any doubts what Red or Green could refer to? Enums are usually like that: We pick suggestive names that can be used as literals. We take care to make these names unique.

    [Exception: In some rare cases we might have many related ADTs where we want to re-use the constructors. But that does not change the equation. If we see a constructor such as Var, and it is not qualified, it probably means “the Var of the expected type”. A sigil such as ..Var changes that only in the sense that I am now sure that it is not an imported Var. But that’s an edge case of an edge case that does not in itself support adding new syntax.]

  3. But there are things other than enums that should also profit from this treatment. That’s true. An example given in the SIP meeting that I found convincing was HTML elements. There’s a great number of them defined in a companion object of the HTML type, but we can’t make them an enum, since the type should be kept open for additions elsewhere. So we came up with the idea to introduce a new soft modifier to label the elements that would be eligible. It was proposed to re-use export for this label, but that’s just a first proposal that could be changed. Example:

    trait HTMLelement
    object HTMLelement:
      export class Body(contents: HTML): HTMLelement
      export class Heading(contents: HTML): HTMLelement
      export class Paragraph(contents: HTML): HTMLelement
    

    Enum cases are always eligible, no export is needed for them.

  4. But it’s tedious to write all these exports and I want to use the cool sigil idea with existing libraries! Well then you are out of luck. This attitude to “anything goes syntactically” was fashionable in early Scala, but we have come a long way since then. In my opinion which is very much shaped by experience it’s as much a job of a language to prevent abuses and to enforce common styles as it is to add cool new flexibility and features. If we allow to use scope injection with sigils everywhere we will get unnecessary diversity in coding styles and open the door to abuse. I can guarantee that people will start to write ..apply() instead of something like IndexedSeq() because it is shorter. And more reasonable people will still hesitate at every point whether it’s preferable to use .. or an explicit qualifier.

    Scala had this issue with methods used in infix position. Scala 3 introduced the requirement that methods used in infix position need to be declared as infix. And that was overall a very good thing for the community, even though it clashed with preferred coding styles of some users. Allowing sigils to skip prefixes is just the same story again. So, yes, writing export explicitly is a bit tedious but that’s the point! We want to restrict this feature to DSL-like API’s where the library designer planned for this possibility.

4 Likes

I’m fine with ..

1 Like

There are now two “modes”, markdown and rich text:
image
And by default the selected mode is rich text (once you change it, it should remember your preference)

2 Likes

Thanks! That was not obvious to me,

I’m honestly not convinced that the “identifiers pop out of nowhere” is more user friendly than “there are some weird dots”
The dots even if confusing have the upside of being explicit, there is something there to google
Disambiguation rules, on the other hand, are “invisible”, and therefore have to be perfectly intuitive

And I don’t have a good intuition for whether local members should shadow enum cases, or the other way around, both ways make equal sense to me
And I don’t even see a clear technical reason for which should be chosen, as I can imagine examples where either would be useful:

// Library

enum EquipmentType:
  case Armors, Weapon, Tools

def inHandSlot(et: EquipmentType) = et != EquipmentType.Armors

// User code

import library.*

object Tools:
  def my_util_method() = ???

inHandSlot(Tools) // Here we want the enum object, not the one in scope
// Library

enum Base:
  case Generation(number: Int, name: String)
  case Other

// Can even predate the introduction of this feature
def Generation(number: Int, name: String = "") = 
  val valid_name = if name.isEmpty() then number.toString() else name
  Base.Generation(number, valid_name)

// User code

import library.*

val g: Base = Generation(1) // We want the def in scope, not the enum object !

That is a very good point, it would even make more sense for ..() to be allowed, which is even shorter
I’m a bit torn on this, on one hand it feels too powerfull and might lead to terrible things
But on the other, it also feels like there would be a lot of cool usage for it !

MyClass.fromList(..(1, 2, 3)) // Pretty clear that it should be a list
distribute(..make("Distant Horizons", 1.5, true)) // What are we distributing and making here ???

But again, we can use .. and force the usage of export if we want !

1 Like

The idea of an export modifier is a disaster. Now every time you want to use this feature you need to remember which methods in the companion object are declared with export and which ones aren’t, and you’re completely at the mercy of library authors’ random whims regarding this. The guy who wrote the library you need to use doesn’t like the feature? Sucks to be you. Usage of this modifier will be wildly inconsistent across libraries, to the point where I’d probably forgo the feature entirely except where it’s guaranteed to work, i. e. enum cases. So why have that modifier in the first place?

The infix modifier that is presented as a model and success story here doesn’t convince me in the slightest. We can debate whether infix syntax for methods is useful or not (I never used it much), but not having an infix modifier and instead simply stating that symbolic methods can be used infix and alphanumeric ones can’t would have been the far better alternative because that’s what one can actually keep in one’s head.

I also find the HTML example completely unconvincing because when you’re writing HTML templates, import HTMLelement.* is really not a big deal. Or better yet: just declare those HTML elements at the package level so you get them by default when you import * from the package. What exactly is the upside of an export modifier over that?

I really only see two sane options here:

  1. go with a sigil (I like ' because it’s lighter on the eyes than # and it’s not required for symbol literals anymore)
  2. go without a sigil, expose only enum cases, rely on import for everything else.

Or maybe, maybe we could consider something like doing this kind of lookup only for identifiers that start with an uppercase letter.

7 Likes