Pre-SIP: a syntax for aggregate literals

Scala Improvement Process
102

I was working on a SIP draft when I realized that one of my motivating use cases, zio-k8s, won’t actually work with the approach as discussed so far :frowning:.

An example zio-k8s definition might look as follows:

val deployment =
  Deployment(
    metadata = ObjectMeta(name = "foo"))

Now ideally we should be able to write this like so:

val deployment =
  Deployment(
    metadata = #(name = "foo"))

But this doesn’t work, because the type of the metadata field isn’t actually ObjectMeta, it is Optional[ObjectMeta], where Optional is from the ZIO prelude. The first example still compiles because there is an implicit conversion from A to Optional[A]. But in the the second example, #(name = "foo") would be desugared to Optional(name = "foo"), which doesn’t work.

I have an idea for how to fix it, but I’d be interested in hearing the community’s thoughts about this issue first.

103

I think the expected type mechanism is too weak, and we must rely on (chained) implicit conversions. Suppose the # notation creates an object of type Something. Then we could have:

import language.implicitConversions
case class Something(name: String)
case class ObjectMeta(name: String)
def f(x: Option[ObjectMeta]) = x
given [T, U](using Conversion[U, T]): Conversion[U, Option[T]] = Some(_)
given Conversion[Something, ObjectMeta] = x => new ObjectMeta(x.name)
f(new Something("abc"))
104

Add apply method to Optional (PRs welcomed) and then this would work, no?

val deployment: Deployment =
  #(
    metadata = #(#(name = "foo")))
105

It does not seem too bad to just not apply in that case.

It also does not seem to bad to spell out the optional:
metadata = Optional(#(name = "foo"))

It would be nice if it were easy to add a generic forwarding method on optional, something like:

object Optional {
  def apply[T](args: Parameters): Optional[T] = Optional( T.apply(args) )
}

You can make that work via macros, but it’s not worth the effort.
And unfortunately, parameter lists are not first class values :face_exhaling:.

106

How could we potentially make parameter lists first class values? Perhaps there could be some rule or adaptation to convert back and forth to a named tuple? Perhaps using a special name args(i) gives a named tuple of args for param list i or similar? If so, that would seem to give opportunities of a whole new area of abstraction if viable in an ergonomic and regular way.

107

I think if we allow partially named tuples, that’s all we need.

After all, what is a parameter list? It’s a mix of named and positional values. Current named tuples are all or nothing w.r.t. naming, but there doesnt seem to be any fundamental reason why they need to be like that.

108

I built library-level partially-named tuples and they work fine, except that it’s already a lot of boilerplate the way I did it, and allowing an un-named prefix and named later elements quadratically complicates things when labeled elements have no subtyping relationship to unlabeled elements (which is how I did it, because I wanted labels for safety).

109

Hey @sideeffffect,

Yes, that would work. However while metadata = #(#(name = "foo")) is ok-ish, it’s still messier than I would like it to be. So here’s the idea I had to deal with this issue and allow the nice metadata = #(name = "foo") thing:

We could add a feature to customize which object the # stands for. To do this, you would add a type alias called # to the companion object of a type (such as Optional). That type alias takes the same number and kinds of type parameters as the type (Optional) itself, and the right hand side of the type alias is the type that will be substituted for the # placeholder.

In case it isn’t clear (it probably isn’t), I’ll give an example, namely the Optional type. In a context where a value of type Optional[A] is expected, you don’t want the # character to stand for Optional, you want it to stand for A. So you add the type alias to the companion object:

object Optional[+A] {
  type #[+A] = A
}

Now, when the compiler sees an expression like #(name = "foo") in a position where type Optional[ObjectMeta] is expected, it will check the Optional companion object and see the # type alias, which tells it that the # symbol in that expression is supposed to stand for something other than Optional, namely A. In this case, A is ObjectMeta, so #(name = "foo") becomes ObjectMeta(name = "foo") rather than Optional(name = "foo"). To keep things simple, and because this feature specifically solves a problem related to implicit conversions, I would not allow chaining these, because implicit conversions also don’t chain.

Now granted, this feels a bit bolted on :sweat_smile: But it does solve the problem, and it would be the kind of feature that not very many people would even need to know about. The authors of Optional would add that line to their companion object, and the users of Optional would probably not even realize it’s there because it just behaves the way you would expect it to.

The question is: is there any use case for this feature beyond Optional? Because if there isn’t, then maybe it’s better to have a dedicated language feature for optional function parameters. I think that would be justifiable because the existing options all suck: With Option, you need wrap Some around the parameter when you do specify it. ZIO’s Optional avoids that using implicit conversions, but those are kind of deprecated. You can simply set a default, but then you can’t tell if the parameter was supplied by the user or not. @lihaoyi uses null defaults in some of his libraries to avoid the need for Some wrapping, but ideally we’d be moving towards explicit null typing, and because the type A | Null doesn’t have a companion object, it wouldn’t work with the # syntax. So there might be room for a dedicated feature for optional parameters.
Perhaps we should just special case A | Null types and treat them the same as non-nullable A for the purposes of # expressions. That might be a better idea because it’s simpler, but at the same time, it would give | Null types a special status and thus encourage the use of null, which maybe we don’t want to do. So overall, I like the “# type alias” idea best for now.

So much for my brain dump for today.

110

tbh I can’t follow your idea here, @rjolly. You say that the “expected type mechanism is too weak”, but we need to get the information about what to create from somewhere…

fwiw, I really would prefer not to solve any of this with implicit conversions or first-class parameter lists because I think it’s essentially impossible to get this to play nicely with tooling. Your IDE needs to understand that you’re trying to invoke a companion object’s apply method in order to give you things like parameter assistance. It cannot understand that when a parameter list is its own thing that can have basically any shape at all. And it can’t be used to solve the “LocalDate problem” either, which I think is important.

111

You shouldn’t need partially named tuples because the compiler already maps these parameters to an ordered set of names, regardless of how they’re arraigned at the call site.

1 Like
112

Exactly.

There is no such thing as unnamed parameters. Parameters have always names.

It’s just the call side where you may leave out the names for brevity. So all that’s needed is to implement the mapping between positions and labels on the call side. But the compiler does this already for every call that uses unlabeled parameters… So that is nothing new.

The much bigger, but likely needed conceptional change would be to make every function accept only one parameter of type ParameterList per parameter list. That’s a quite fundamental change to the language, I think.

Applying a subset (does it need to be a sub-type also?) of a ParameterList to a function would need to yield again a function (partial application). That’s also something that does not exist currently as partial application works differently afaik, and does not involve the notion of constructing “sub-ParameterLists” as “real things”.

Nevertheless I think having a language that has a notion of ParameterLists would be really nice. Finally again one step closer to reach the conceptional simplicity, regularity and flexibility of LISP, the mother of functional programming. (Most likely this would also make manipulating code by code, and also code gen simpler as you had less special constructs, like hardcoded parameter list handling in the language. So this would look like a great win imho! I also bet Odersky will hate the idea for exactly the reason of similarity to LISP… :wink:)

113

Back on topic: I very much like the idea of literals for anonymous objects.

That’s one of the great features I miss most from JS!

But this here is taking a questionable turn currently.

This # thingy doesn’t look like object literals any more. It looks like some arbitrary magic syntax that allow to write extremely confusing code like this here:

https://www.reddit.com/r/programminghorror/comments/1ebwknk/maybe_i_should_use_type_names_for_constructors/

1 Like
114

Hi Mateusz,
thanks for engaging with the topic, and thank you for your support.

Numerous syntax variants have been discussed in this thread and were found to be unsatisfactory. Namely:

  • the original idea of square brackets, [].
    • is hard to disambiguate from polymorphic lambda syntax, which also starts with [
    • doesn’t allow invocation of other constructor methods like of (for LocalDate)
  • round parens, i. e. ()
    • incompatible because it’s used for tuples today
    • highly error-prone because () is also used for grouping expressions
  • braces, i. e. {}: incompatible because they’re used for blocks, and unlike in JS, blocks are expressions in Scala
  • other tokens like @ or :: used in pattern syntax today, and I’d like to choose a syntax that could potentially be extended to pattern matching in the future
  • most other tokens, e, g, <angle brackets> are valid identifiers today, and thus incompatible
  • .. was suggested, but it looks downright disgusting to me and I don’t want to propose syntax this ugly
  • $ is used for splices today

So no, the choice of # is in fact far from arbitrary, it has in fact been been discussed in this thread at length. As has the potential for abuse, to which my response is: if your problem is that “it hurts when I do this”, then don’t do that.

115

It’s not about the syntax.

The first, and actually still unanswered question is the one about the semantics of this feature. My point was: The core is not even defined, but you start to add arbitrary rules.

Designing a language feature is generally coming up with some rules. So the first step would be to define the rules which say how the core of this should work. The syntax is at first unimportant and can be seen as kind of dummy. So whatever it is that’s fine for now.

My question would be: What’s the current state of the idea as such? What should this fictive syntax “expand” to? What does it even mean to write out an “anonymous object literal”? How do the rules for type checking look like?

The thread contains several ideas. Loosely intermixed. The last few posts started to add random stuff on top of the random ideas… I’m not sure this leads anywhere. (Even I for example like the general idea of having first class ParameterLists, and the idea of this “dot-scope-operator”, with maybe even the feature to allowing to leave out apply when using the dot. But none of these things explains in detail what “object literals” in Scala should mean).

As this would be quite a core feature the semantics need to be simple and general. So some rules with special cases are imho to avoid.

The only really overreaching concept I’ve seen here so far would be actually to unify parameter lists with named tuples with “anonymous objects”. But I don’t know how this could look like in detail… (At least you would likely get this feature here for free as all that would be needed is some syntax to trigger target typing guided conversion of named tuples to concrete, nominal types, which would yield the desired “anonymous object literals” feature).

3 Likes
116

Welp :person_shrugging: That’s just the structure of how the data is defined. I’d rather accept that this is what it is.
It’s at least simple, straightforward and regular.
Anything more clever would be too much magic and too confusing.

Yep. null and Null are already special, so I don’t see much of a problem in giving them a special treatment for this feature we’re discussing.
A | Null calls A.apply. Any other union types are disallowed.

val x: A | B = #(...) // not clear whose `apply` to call, prohibited
val y: A | Null = #(...) // `A.apply` is the only sensible option, so no problem
3 Likes
117

mberndt as the thread started seems to have put forth and communicated pretty clear proposal though:

A symbol that refers to the companion object of whatever type is expected as the result of the “current expression” where current expression is the same as for _ (as in, the _ used for shorthand anonymous functions)

The current question seems more to be about if that leads to the desired results.

1 Like
118

Hi Mateusz,

The feature is in fact very well defined by now: # is a placeholder for the companion object of the type expected where the surrounding expression occurs. The “surrounding expression” is defined in the same way as for lambda expressions that use the _ placeholder syntax. And this was all spelled out quite explicitly in this post.

Numerous examples of what this syntax expands (or desugars) to were given in this thread, including, but not limited to, here, here and here.

You are the only one here who used the term “anonymous object”.

Yes, it’s almost as if several people looked at the problem from different angles and discussed the various alternatives for solving it. How weird is that!
One could almost think that that’s what Pre-SIPs are for…

119

I have been keeping silent so far, since I wanted to see where the discussion would go. My main issue with the proposal comes from two observations, which I take as axioms.

  1. Names are usually really valuable. I know more code bases that use not enough names, or use bad names, than code bases that are too verbose. And missing names are much more of a problem than redundant ones.
  2. Programmers will very often use the shortest form available to them for expressing a concept. There are exceptions, but in general this is true.

So a proposal that allows programmers to omit names by replacing them with a special symbol is problematic, and has to be seen through a critical lens of possible abuses (which will be common, see point 2 above).

I was mildly positive about the original proposal to have special forms for collection literals. Here, it’s true that in many cases it does not matter what precise type of collection we create as long as it is a sequence or a map. So one could argue that the added requirement to specify the type as in List(...), or HashMap(....) is annoying if all one wants is the language default for this kind of collection, or, alternatively the type that’s expected. In any case, the syntactic form of the expression would tell us “this is a sequence” or “this is a map” without needing to look up the context.

But the use cases I have seen here for generalizing the proposal don’t have that property. They need a lot of context to get a sense of the meaning of the code. So they leave me skeptical. At a casual glance many of the examples looks like symbol salad to me.

10 Likes
120

Hey @odersky ,

Thank you for commenting. Regarding your point 1, I feel that you haven’t given enough consideration to the fact that this feature is specifically engineered as syntactic sugar (as opposed to something like first class parameter lists or implicit conversions) because that allows editors to show what the # symbol stands for when that is desired.

As for the potential for abuse, I’ve made my position clear on that more than once in this thread, so I won’t repeat myself here. Needless to say, I’m disappointed by this outcome, but I feel that at this point too many SIP committee members have expressed opposition to this idea for it to succeed.

121

I’m a SIP committee member and I’m still interested in seeing this idea develop. :wink: I don’t have a strong opinion on which flavor is the most promising so far, though, which is why I’ve been fairly silent as well.