Pre-SIP: a syntax for aggregate literals

Scala Improvement Process
95

As an additional data point, I would like to mention that the companion object members are not brought automatically into scope for the corresponding class, which I think is similar to the expected type situation, and at first suprised me, as companion members are like Java’s static fields, which are in a class’ scope, obviously.

// nok

class Foo:
  bar
  ^^^
  Not found: bar

object Foo:
 def bar = ???

// ok

import Foo.bar

class Foo:
  bar

object Foo:
 def bar = ???

Although surprising, I guess there is good reason for it, and that might hinder automatic import of companion members in the expected type situation as well.

96

At this point in the discussion (re-skimming the thread a bit) I am not quite sure what exactly the discussion is about exactly anymore, I see at least:

  1. new syntax, e.g. [a, b] being somehow automatically converted to the target expression (original proposal)
  2. a new idiom to use existing tuples as literals (e.g., with a FromLiteral typeclass that essentially acts like an always allowed implicit conversion) by soronpo, and mentioned by lihaoyi.
  3. “scope injection” of symbols defined in the companion object of the target type (I found a couple mentions, but not the original proposal)
  4. a symbol to kinda access the companion object of the target type.

I may also have overlooked some.

1&2 seem to be somewhat mutually exclusive, as seem 3&4.
But overall it seems unclear to me which to prefer.

I am actually with you on this one. Scala already has all the tools to win an obfuscated code challenge.
Moreover, I think that arbitrary “readability restrictions” make code harder to understand as it requires learning all the little exceptions where something is allowed and where not.

Side note, I think named tuples are an interesting example here, because they bring tuple syntax and parameter list syntax closer together, removing exceptions.

1 Like
97

Ah right, good point! I think it’s still technically unambiguous because expressions can’t be followed by a => token, whereas the type parameter list of a polymorphic function type must be. So once the parser finds the matching ] token, the next token can determine what kind of expression it is: if it’s => then it’s a polymorphic lambda, in any other case it’s an aggregate literal. But it’s still a mess that’s probably best avoided.

That’s an interesting idea… Trailing commas are not currently allowed in tuples though, so it’s still a syntax extension.
Python does it this way, but nevertheless, I find it looks a bit odd, and if we’re going to have to extend the syntax, then I think that I’d prefer something like your other proposal:

I thought of that too, and it certainly has advantages. I don’t think we can get away with any of {}, [] or (), so we’re going to need some sort of “decorated paren” thing. And since most characters can be used as identifiers in Scala, we don’t have that many left. We should also consider that we probably want to extend this syntax from expressions to patterns some day. If this code works…

val x: Seq[Int] = #(1,2,3)

…then so should this:

x match
  case #(a, b, c) => 42

Hence, syntax that works for expressions but not for patterns should probably be avoided. Specifically, :() and @() would be fine in an expression context, but could lead to confusion in a pattern context because those symbols are already used in patterns. An entirely separate symbol like # would avoid that for human readers. For non-human readers, any of :, @ and # would be fine because a pattern can’t start with any of those today.
The last option I can think of is .(), which is largely the same as #(), so I’d be fine with it, although I do prefer #() on a visual level.

Most other symbols are either obviously unsuitable, or a valid identifier, or Unicode (non-obvious to type).

98

Haha, I can’t blame you because we’ve explored many different paths from where we started. I think that’s actually really good and it has certainly provided me with new insights.

Going through your list in order, here’s how I think about the various options:

  1. That is the original idea. I don’t think of it as “conversion” though, rather it’s a kind of syntactic sugar that’ll fill in the correct companion object in front of a parameter list. So you can go from [1,2,3] to List(1,2,3), but also from [x=1, y=2] to Point(x = 1, y = 2).
  2. (tuple conversion) I don’t like this idea for a variety of reasons. It’s less flexible because there are things that just aren’t possible with tuples, like multiple parameter lists or using clauses or having some parameters named and others unnamed. It will also lead to terrible error messages and bad tooling, unless the tools grow specific support for these conversions. Moreover I don’t think it can be made to work for the case where such expressions are nested. I don’t think a good solution can be achieved this way.
  3. (scope injection) The original scope injection thread is here. At one point I thought it was a good idea to merge the two, but I’m no longer convinced because I think the issues are sufficiently distinct that more than one language feature is going to be required to solve them (sorry @soronpo, I’m still convinced that relative scoping of some form is required due to the reasons I’ve laid out in another comment, but I think it’s a largely separate issue, and I’m still prepared to help out with writing a proposal)
  4. (placeholder for companion object) That is quite similar to number 1 which proposes a syntax for companion object apply calls. It has the added benefit of also allowing things like #.of(1958, 9, 5) to create a LocalDate object (assuming that # means companion object). I think that’s a good solution.

Now that you brought up that last one again, I had some more thoughts about it. At one point I thought it would be nice to have a syntax to select members from the companion object, e. g. #of would select the of member of the companion object (or static method for Java classes). That would allow us to get rid of that ugly little dot in #.of(1958, 9, 5). But then I realized that maybe you don’t always want to select anything from the companion but rather just refer to the companion object itself. Notably, that is the case for collection conversions:

val foo = List("bar" -> 42)
def baz(m: Map[String, Int]) = ()

baz(foo.to(Map)) // using companion object here
                 
baz(foo.to(#))   // but could use a placeholder too!

So maybe that ugly little . in #.of(1958, 9, 5) is the price to pay to enable this use too.

Now that I’ve thought about it again and that @lihaoyi has demolished the [] idea, I think that this “companion object placeholder” idea is probably the best solution.

Absolutely, that is what I was trying to express with many more words before. Let’s make the language simple and orthogonal and have linters deal with “readability” for those that deem that necessary.

99

I wonder about this. Let me phrase it a bit differently than tuple conversions.

The current state of things is, that the syntax for method parameter lists at call site is effectively the syntax for literals in Scala.
The main problem discussed in this thread seems to be that importing and/or repeating the companion object is unnecessary boilerplate (there seems to be little disagreement about this).

A very direct way to address this seems to be to allow omitting that companion object definition. The remaining part would be a “parameter literal”. Are parameter literals typeable? Maybe not in general, but for many cases without other constraints their type would just be the corresponding tuple type.

Similar to how function literals can be converted to SAM types, a parameter literal could be converted to types marked by something (a type with an apply method on the companion, or marked with some annotation, or using some type class like FromLiteral, does not matter for now).

I think this is very close to many proposals that were made in this thread (it effectively looks like automatic import of apply methods, and implicit conversions for simpler cases). But I want to emphasize this here because it can explained by analogy to existing language constructs – the syntax exists, and conversions based on expected type exist.

To me it seems that this use of “just get the companion object” does not fit well with the way type inference works.
In my proposal for this variant: (companion.abc(xyz): T) it’s just the return type that needs to be inferred, and as my implementation shows that is actually possible today using implicit.
The above seems to either require some inverse inference (going from the outer type serveral levels deep inside) or heuristics on what the scope of # should be that are similar to how _ works for anonymous functions (which is to say, I don’t think that would work well).

2 Likes
100

Yes, exactly. That is pretty much point 1 in your list above, and my original proposal (using [] rather than ()).
And that’s definitely a viable proposal, we can make that work. But please do consider the issues that were brought up about this:

  1. It can’t be used when you want to use a method other than apply. For example, LocalDate objects aren’t created using LocalDate(y, m, d), they’re created using LocalDate.of(y, m, d). Or a cats.data.NonEmptyList, which isn’t created with NonEmptyList(a,b,c,d) but NonEmptyList.of(a,b,c,d). I don’t think this is absolutely crucial, but it’s nice to have.
  2. It doesn’t work when you have more than one parameter list or a using clause.
  3. If we go with a () syntax, then simply wrapping an expression in parens – which is so far always a no-op – can now cause a constructor to be called. I think this makes it way too easy to accidentally trigger construction of an object that you didn’t mean to (e. g. typesafe ID wrapper types)
  4. If we go with a [] syntax, then it complicates the parser as @lihaoyi has helpfully pointed out

So overall, this approach doesn’t address all the use cases that I would like it to, and both of the syntaxes that have been proposed have drawbacks that I’d rather avoid. That is why currently the “companion object placeholder” model looks best to me.

That’s right! And the good news is that we already do that today for other language constructs. For example, this works perfectly fine:

val f: Int => Int => Int =
  x => y => x + y

Neither x nor y need a type annotation here, so this recursive, incremental type inference thing is actually already happening, and it’s a proven approach.

Yes, absolutely, and the scoping issue is exactly what I was trying to get at in my previous comment.
But the good news is that, again, we have a set of proven rules on how that should work, and it’s the scoping of _ in lambda expressions. So that’s why my suggestion is to use the exact same rules also for the scope of the # placeholder. That would work for every reasonable example I can come up with:

val _: List[Int] = #(1, 2, 3)
val _: Duration = #.fromNanos(42)
val _: List[Int] = (1 to 10).to(#)
val _: Future[Unit] = #(println("Hello, ragnar!"))(using ExecutionContext.global)

And actually, we can experiment with that syntax today! We just need to place CompanionObject.type => in front of the expected type and then use an _ instead of # and squint a bit! All these compile:

val _: List.type => List[Int] = _(1, 2, 3)
val _: Duration.type => Duration = _.fromNanos(42)
val _: List.type => List[Int] = (1 to 10).to(_)
val _: Future.type => Future[Unit] = _(println("Hello, ragnar!"))(using ExecutionContext.global)

And here’s an extra cool one:

val _: List[List.type] = #(#) 

// to simulate the syntax in current Scala:
val _: (List.type, List.type) => List[List.type] = _(_)
1 Like
101

To me, introducing new characters vs mostly reusing existing things is quite a drastic difference in proposal. New characters run out quickly, and change the way a language looks by a lot.

Both the .of(a, b, c, d) and of(a, b, c, d) variants have been proposed and seem like they could work. There have been some arguments about ambiguity before, but the way I see it is that a “conversion” would happen only in places where there is a known expected result. Might still have issues, but would need to be explored further.

I don’t see why it could not work technically. Just have multiple parameter lists as in any of the other variants.

The example seems quite different, but I guess you view curried functions not as a single entity.

I did test it though, and came to the conclusion that baz(foo.to(summon)) works.

I guess it’s just unfortunate that it does not seem to work with my macro/dynamic hacks :neutral_face:.

Concluding remark. Anyway, as I said before, I think the “a shorthand for the companion object” variant is a good solution, and it does seem more plausible to add compared to making parameter lists first class (or even just second class …).

1 Like
102

I was working on a SIP draft when I realized that one of my motivating use cases, zio-k8s, won’t actually work with the approach as discussed so far :frowning:.

An example zio-k8s definition might look as follows:

val deployment =
  Deployment(
    metadata = ObjectMeta(name = "foo"))

Now ideally we should be able to write this like so:

val deployment =
  Deployment(
    metadata = #(name = "foo"))

But this doesn’t work, because the type of the metadata field isn’t actually ObjectMeta, it is Optional[ObjectMeta], where Optional is from the ZIO prelude. The first example still compiles because there is an implicit conversion from A to Optional[A]. But in the the second example, #(name = "foo") would be desugared to Optional(name = "foo"), which doesn’t work.

I have an idea for how to fix it, but I’d be interested in hearing the community’s thoughts about this issue first.

103

I think the expected type mechanism is too weak, and we must rely on (chained) implicit conversions. Suppose the # notation creates an object of type Something. Then we could have:

import language.implicitConversions
case class Something(name: String)
case class ObjectMeta(name: String)
def f(x: Option[ObjectMeta]) = x
given [T, U](using Conversion[U, T]): Conversion[U, Option[T]] = Some(_)
given Conversion[Something, ObjectMeta] = x => new ObjectMeta(x.name)
f(new Something("abc"))
104

Add apply method to Optional (PRs welcomed) and then this would work, no?

val deployment: Deployment =
  #(
    metadata = #(#(name = "foo")))
105

It does not seem too bad to just not apply in that case.

It also does not seem to bad to spell out the optional:
metadata = Optional(#(name = "foo"))

It would be nice if it were easy to add a generic forwarding method on optional, something like:

object Optional {
  def apply[T](args: Parameters): Optional[T] = Optional( T.apply(args) )
}

You can make that work via macros, but it’s not worth the effort.
And unfortunately, parameter lists are not first class values :face_exhaling:.

106

How could we potentially make parameter lists first class values? Perhaps there could be some rule or adaptation to convert back and forth to a named tuple? Perhaps using a special name args(i) gives a named tuple of args for param list i or similar? If so, that would seem to give opportunities of a whole new area of abstraction if viable in an ergonomic and regular way.

107

I think if we allow partially named tuples, that’s all we need.

After all, what is a parameter list? It’s a mix of named and positional values. Current named tuples are all or nothing w.r.t. naming, but there doesnt seem to be any fundamental reason why they need to be like that.

108

I built library-level partially-named tuples and they work fine, except that it’s already a lot of boilerplate the way I did it, and allowing an un-named prefix and named later elements quadratically complicates things when labeled elements have no subtyping relationship to unlabeled elements (which is how I did it, because I wanted labels for safety).

109

Hey @sideeffffect,

Yes, that would work. However while metadata = #(#(name = "foo")) is ok-ish, it’s still messier than I would like it to be. So here’s the idea I had to deal with this issue and allow the nice metadata = #(name = "foo") thing:

We could add a feature to customize which object the # stands for. To do this, you would add a type alias called # to the companion object of a type (such as Optional). That type alias takes the same number and kinds of type parameters as the type (Optional) itself, and the right hand side of the type alias is the type that will be substituted for the # placeholder.

In case it isn’t clear (it probably isn’t), I’ll give an example, namely the Optional type. In a context where a value of type Optional[A] is expected, you don’t want the # character to stand for Optional, you want it to stand for A. So you add the type alias to the companion object:

object Optional[+A] {
  type #[+A] = A
}

Now, when the compiler sees an expression like #(name = "foo") in a position where type Optional[ObjectMeta] is expected, it will check the Optional companion object and see the # type alias, which tells it that the # symbol in that expression is supposed to stand for something other than Optional, namely A. In this case, A is ObjectMeta, so #(name = "foo") becomes ObjectMeta(name = "foo") rather than Optional(name = "foo"). To keep things simple, and because this feature specifically solves a problem related to implicit conversions, I would not allow chaining these, because implicit conversions also don’t chain.

Now granted, this feels a bit bolted on :sweat_smile: But it does solve the problem, and it would be the kind of feature that not very many people would even need to know about. The authors of Optional would add that line to their companion object, and the users of Optional would probably not even realize it’s there because it just behaves the way you would expect it to.

The question is: is there any use case for this feature beyond Optional? Because if there isn’t, then maybe it’s better to have a dedicated language feature for optional function parameters. I think that would be justifiable because the existing options all suck: With Option, you need wrap Some around the parameter when you do specify it. ZIO’s Optional avoids that using implicit conversions, but those are kind of deprecated. You can simply set a default, but then you can’t tell if the parameter was supplied by the user or not. @lihaoyi uses null defaults in some of his libraries to avoid the need for Some wrapping, but ideally we’d be moving towards explicit null typing, and because the type A | Null doesn’t have a companion object, it wouldn’t work with the # syntax. So there might be room for a dedicated feature for optional parameters.
Perhaps we should just special case A | Null types and treat them the same as non-nullable A for the purposes of # expressions. That might be a better idea because it’s simpler, but at the same time, it would give | Null types a special status and thus encourage the use of null, which maybe we don’t want to do. So overall, I like the “# type alias” idea best for now.

So much for my brain dump for today.

110

tbh I can’t follow your idea here, @rjolly. You say that the “expected type mechanism is too weak”, but we need to get the information about what to create from somewhere…

fwiw, I really would prefer not to solve any of this with implicit conversions or first-class parameter lists because I think it’s essentially impossible to get this to play nicely with tooling. Your IDE needs to understand that you’re trying to invoke a companion object’s apply method in order to give you things like parameter assistance. It cannot understand that when a parameter list is its own thing that can have basically any shape at all. And it can’t be used to solve the “LocalDate problem” either, which I think is important.

111

You shouldn’t need partially named tuples because the compiler already maps these parameters to an ordered set of names, regardless of how they’re arraigned at the call site.

1 Like
112

Exactly.

There is no such thing as unnamed parameters. Parameters have always names.

It’s just the call side where you may leave out the names for brevity. So all that’s needed is to implement the mapping between positions and labels on the call side. But the compiler does this already for every call that uses unlabeled parameters… So that is nothing new.

The much bigger, but likely needed conceptional change would be to make every function accept only one parameter of type ParameterList per parameter list. That’s a quite fundamental change to the language, I think.

Applying a subset (does it need to be a sub-type also?) of a ParameterList to a function would need to yield again a function (partial application). That’s also something that does not exist currently as partial application works differently afaik, and does not involve the notion of constructing “sub-ParameterLists” as “real things”.

Nevertheless I think having a language that has a notion of ParameterLists would be really nice. Finally again one step closer to reach the conceptional simplicity, regularity and flexibility of LISP, the mother of functional programming. (Most likely this would also make manipulating code by code, and also code gen simpler as you had less special constructs, like hardcoded parameter list handling in the language. So this would look like a great win imho! I also bet Odersky will hate the idea for exactly the reason of similarity to LISP… :wink:)

113

Back on topic: I very much like the idea of literals for anonymous objects.

That’s one of the great features I miss most from JS!

But this here is taking a questionable turn currently.

This # thingy doesn’t look like object literals any more. It looks like some arbitrary magic syntax that allow to write extremely confusing code like this here:

https://www.reddit.com/r/programminghorror/comments/1ebwknk/maybe_i_should_use_type_names_for_constructors/

1 Like
114

Hi Mateusz,
thanks for engaging with the topic, and thank you for your support.

Numerous syntax variants have been discussed in this thread and were found to be unsatisfactory. Namely:

  • the original idea of square brackets, [].
    • is hard to disambiguate from polymorphic lambda syntax, which also starts with [
    • doesn’t allow invocation of other constructor methods like of (for LocalDate)
  • round parens, i. e. ()
    • incompatible because it’s used for tuples today
    • highly error-prone because () is also used for grouping expressions
  • braces, i. e. {}: incompatible because they’re used for blocks, and unlike in JS, blocks are expressions in Scala
  • other tokens like @ or :: used in pattern syntax today, and I’d like to choose a syntax that could potentially be extended to pattern matching in the future
  • most other tokens, e, g, <angle brackets> are valid identifiers today, and thus incompatible
  • .. was suggested, but it looks downright disgusting to me and I don’t want to propose syntax this ugly
  • $ is used for splices today

So no, the choice of # is in fact far from arbitrary, it has in fact been been discussed in this thread at length. As has the potential for abuse, to which my response is: if your problem is that “it hurts when I do this”, then don’t do that.