SIP: Curried varargs

Scala Improvement Process
23

Separating the indices from the values could be done, if I understand things correctly, though it’d be a bit tedious.

class CurriedDynamic extends Dynamic {
  def updateDynamic: Dynamic0 = new CurriedDynamic.DynamicApply(this)
}
object CurriedDynamic {
  class DynamicApply(instance: CurriedDynamic) extends Curried {
    def applyBegin: DynamicBegin = this
    def applyEnd: String = s"$instance.updateDynamic called"
    def applyNext(method: String): Dynamic0 = new Dynamic0(instance, method)
  }

  class Dynamic0(instance: CurriedDynamic, method: String) extends Curried {
    def applyEnd: String = s"$instance.$method() called"
    def applyNext(arg0: Int): Dynamic1 = new Dynamic1(instance, method, arg0)
  }

  class Dynamic1(instance: CurriedDynamic, method: String, arg0: Int) extends Curried {
    def applyEnd: String = s"$instance.$method($arg0) called"
    def applyNext(arg1: String): Dynamic2 = new Dynamic2(instance, method, arg0, arg1)
  }

  class Dynamic2(instance: CurriedDynamic, method: String, arg0: Int, arg1: String) extends Curried {
    def applyEnd: String = s"$instance.$method($arg0, $arg1, $arg2) called"
  }
}

On the plus side, I believe this may limit what can and cannot compile:

val a = new CurriedDynamic()
a()          // "a.apply() called"
a.z()        // "a.z() called"
a.y(1)       // "a.y(1) called"
a.x(1, "Hi") // "a.x(1, \"Hi\") called" 

// I don't think these would compile:
a.f(1, 2)
a.f(1, "Hi", 2)
24

The last bit brought up an interesting thought: how would we expect something like this to behave?

object Calculate extends Curried {
  def applyBegin: Root = Root

  object Root extends Curried {
    def applyNext(name: String): Operation = name match {
      case "const" => new Operation.WaitingForConst
      case "sum" => new Operation.Sum(0)      
      case unrecognized => new Const(Left(s"Unrecognized operation: $unrecognized"))
    }
  }

  sealed trait Operation extends Curried
  object Operation {
    class WaitingForConst(val ignore: Boolean = true) extends AnyVal with Operation {
      def applyNext(value: Int): Result = new Const(Right(value))
    }
    class Sum(val accum: Int) extends AnyVal with Operation {
      def applyNext(next: Int): Sum = new Sum(accum + next)
      def applySeq(next: Seq[Int]): Sum = new Sum(accum + next.sum)
      def applyEnd: Left(accum)
    }
  }

  class Const(value: Either[String,Int]) extends AnyVal with Curried {
    def applyEnd: Either[String, Int] = value
  }
}

While the intent of the code is for something like this to work:

Calculate("const", 4) == Right(4)
Calculate("sum", 1, 2, 3) == Right(6)
Calculate("pi") == Left("Unrecognized operation: pi")

How this would actually happen is unclear, or what would happen in these cases:

Calculate("const")
Calculate("const", 1, 3)
Calculate("sqrt", 8)

Which leads me to believe that traits extending Curried should probably be disallowed

25

I don’t think the problem in the above example is the trait part, but instead that you require that the translation be performed at runtime.

More specifically, Curried as I have understood it, like Dynamic, is structural in it’s behavior (you only implement the parts you want to support, instead of overriding methods from the Curried trait).

If so, then it becomes clear that none of the bad examples you provided would compile, as it can’t find the method to execute.

The trait Operation has no applyEnd or applyNext method, so the curried calls become compile time errors.

You could probably do something like you want above if we had Dotty’s inline defs.

1 Like
26

The trait just makes the ambiguous fan-out easier to pull off. Returning Curried would probably have worked as well.

If I’m understanding things correctly, if you convert Calculate("const", 1, 2), you’ll get this:

/* 0 */ Calculate
/* 1 */  .applyBegin
/* 2 */  .applyNext("const")
/* 3 */  .applyNext(1)
/* 4 */  .applyNext(2)
/* 5 */  .applyEnd

Until line 2, the compiler knows the exact type which will be returned, but from line 2 on, the return value is just Operation, which extends Curried.

On line 3, either the compiler looks and sees that Operation doesn’t implement these directly, and fails to compile, or trusts that one of it’s subtypes will implement it (though it doesn’t know which one) and the code triggers a NoSuchMethodError at runtime.

So, I guess the problem isn’t that Operation extends Curried, it’s that the return type is dependent on the value passed. Overloading applyNext should work, because the compiler can trace the types, but I don’t know if there’s a way to do this for runtime values (or a good way to prevent this from happening).

1 Like
27

If named-parameter arguments are to be supported, I think it makes a lot more sense to translate e.g. f(x = 1, y = "q") to f.applyBegin.applyNamed.x(1).applyNamed.y("q").applyEnd than it does to translate it to f.applyBegin.applyNamed("x", 1).applyNamed("y", "q").applyEnd. Making named parameters into selections allows you to have different types for different named arguments (otherwise you are right back to all the problems with the existing varargs). If you wanted the stringish behavior, you could easily recover it by making your applyNamed extend Dynamic.

2 Likes
28

Using literal types is another option for named arguments.

29

But it’s subsumed by @clhodapp’s proposal, right? If you want literal types, you can always use Dynamic with a def applyDynamic[S <: String with Singleton](name: S)(...) or something like that, no?

30

I just realized that special treatment for named arguments or repeated arguments in this proposal is unnecessary. We can just pass through those arguments like this.

f(a, s: _*, k = v)

Translated to:

f.applyBegin
  .applyNext(a)
  .applyNext(s: _*)
  .applyNext(k = v)
.applyEnd

Then the library author is able to handle the parameter name k at runtime with help of a Dynamic, and handle s with the help of ordinary sequence arguments.

How do you think?

1 Like
31

I think that, for named arguments, this is a good idea. It’s low-effort, and provides maximum flexibility as the author is able to either restrict the permissible named arguments by explicitly defining applyNext (which would appear to be the default), or accepting anything by mixing Dynamic with Curried.

I don’t think we can do this when splatting a collection, because the default behavior would be to not support s:_* syntax, which breaks the conceptual equivalence between Curried and regular varargs work.

1 Like
32

I just realized that special treatment for named arguments or repeated arguments in this proposal is unnecessary.

I would prefer the different solution.

f(a, s: _*, k = v,d,c)

Translated to:

  f.applyBegin
  .applyNext(a)
  .applyNext(s: _*)
  .applyName("k")
  .applyNext(v)
  .applyName("")
  .applyNext(d)
  .applyNext(c)
.applyEnd
33

I think that having to keep track of the state between applyName and applyNext calls will be a pain point for authors.

Quick sanity check: how often would named var-args actually be useful?

1 Like
34

Actually It would be useful for me quite often. I would prefer in many cases :

(name="test", qty=10)
//instead of 
(name -> "test", qty->10)

IMHO Dynamic will be more painfully for me.I just have suggested less painful case. There are many alternatives.

I think it is not rare case, so dynamic is not best solution

1 Like
35

Given that overloading a method on just the name of the parameter is not supported, this doesn’t seem like a very promising encoding.

36

The last SIP meeting ended up being private-only, but we did discuss this SIP. We will discuss again in public on November 27th, but here is nevertheless the gist of the feedback from the SIP committee.

There seem to be two main aspects for the motivation of the proposal: type safety and performance.

For type safety, we’re basically not very convinced that the use cases cannot be implemented using appropriate usage of implicit evidences and/or union types and/or HLists.

For performance, we believe that it should be possible to implement a forced expansion using macros, perhaps even just with inline in Scala 3.

Therefore, the feedback of the committee is that we think you should try and implement this using existing tools in the language.

37

In spite of the implementation, so does the committee think this feature should be a part of the standard library, or a 3rd party library?

38

A third-party library. Anything that can be outside of the stdlib should prove its value as a third-party library before it can apply for inclusion in the stdlib.

39

This is rather disappointing, as it severely curtails the ability of those of us who are in favor of these proposals to rebut these concerns.

While the next meeting will be public, the situation has become asymmetric. We’ll be working against a formed position (whatever you may claim to the contrary, the SIP is made up of humans, and that’s how the human brain works), rather than participating in the formation of that opinion.

2 Likes
40

A reason why this proposal might not be suitable in a 3rd party library is that it could be a dependency of other core features, including HList, collection initializers, string interpolations.

1 Like
41

The string interpolators of the stdlib are already intrinsified by the compiler, and HLists (aka tuples in Scala 3) already eliminate all the overhead based on inline and IIRC match types. So both are already as efficient as possible.

42

In order for tuples to stand in for varargs, there needs to be some form of auto-tupling (ideally, opt-in at the definition site), else you need to write double parentheses at the call site. I brought this use case up on the auto-tupling removal thread (Let's drop auto-tupling). Unfortunately, unless something has changed, the frontrunner solution for opting in was removed as “not pulling its own weight” (https://github.com/lampepfl/dotty/pull/4311#issuecomment-381112023), which I feel like might have been due to confusion between present usefulness and potential future usefulness.

2 Likes