Proposal To Revise Implicit Parameters

I think the variety of terminolohy could be frustrating
Single “implicit” tag-word now splitted to

  • given keyword
  • that means parameter should be inferred
  • implied definitions
  • context queries

Wouldn’t it be more wise to think about some single new word for all terms like

  • infer keyword
  • that means paarameter should be inferred
  • inferred definitions
  • infer functions
4 Likes

Multiple meanings of implicit were exactly the reason for splitting.

< and > are valid method/ operators names, so there would be some grammar headaches.

Something like ([ and ]) would be more easier to model in grammar, I think. Then the syntax would be: f([global])("abc")([ctx]) which is still relatively easy to grasp.

Or maybe go for: f(given global)("abc")(given ctx)??? That should be least confusing and also fit well into implicit hints in IDE: IntelliJ Scala plugin 2018.2: advanced “Implicit” support, improved patterns autocompletion, semantic highlighting, scalafmt and more | The IntelliJ Scala Plugin Blog

1 Like

Yes. I agree with that. However if we can just draw line between implicit parameters and implicit conversions, it would be enough.
implied, inferred, context queries and given all are about implicit parameters

6 Likes

IIUC implicit conversions are gone already in the new scheme. You need to replace them with typeclasses and/ or extension classes.

There was some effort to write the docs without using the term implicit at all. Just to see whether we could wean ourselves off. I expect that once the new proposals are in, we will come back to talk about implicit parameters, or implicit function types. So this is largely a matter of exposition only.

In terms of syntax, there’s one additional distinction compared to the status quo: Implicit parameters are named given and implicit instances are named implied. It would not work having one term for both because then you would have atrocities like

   given tc given X for Y { ... }

instead of

  implied tc given X for Y { ... }

Also, one of the iron rules of the design is that we want to align parameter syntax and call syntax. I believe that principle forces us to split instance and parameter syntax, otherwise things would get very confusing.

2 Likes

But can we at least call related term accordingly?
So for inferred paremeters we can use infer keyword and rename context queries to inferring functions or inferring expressions or something like that.

3 Likes

I love the capability!

I am less sold on the syntax. The main issue I see is that it sets up a clash between standard visual parsing of precedence and the actual precedence when you write

def f given (a: A)(b: B)

or, less bad but still not great,

def f(a: A) given (b: B): C

The problem is that both visually and by normal rules of precedence, paren-to-paren joins have extremely high precedence while space-separated words have very low precedence. Here it is the opposite: given has higher precedence than adjacent parens or type ascription. Consider, for contrast, if the following pairs were equivalent:

if (p) foo() else bar()()
(if (p) foo() else bar())()
if (p) foo else bar: B
(if (p) foo else bar): B

I don’t have an incredibly awesome idea of what to do about this. You can force the given to go into parens, but then without additional rules you lose the very pleasant

foo(x) given ctx

Nonetheless, I think the irregular precedence problem is sufficiently bad that we should try quite hard for an improvement. (I don’t know what erased means, but I think it has the same problem.)

The most boring possibility is just to restrict it to terminal position (leaving out the implicit syntax for simplicity):

DefParamClauses  ::= {DefParamClause} [[nl] GivenParamClause]
DefParamClause   ::= `(` (GivenParams | [DefParams]) `)`
GivenParamClause ::= 'given' ('(' DefParams ')' | GivenTypes)
GivenParams      ::= 'given' (DefParams | GivenTypes)

with other things as they are now. (Similar changes for ClsParamClauses, and ParArgumentExprs should get an extra entry for `’(’ ‘given’ ExprsInParens ‘)’.)


Again, I love the capabilities. It’s just the syntax which is weird.

4 Likes

Choosing
x.f(given global)("abc")(given ctx)
syntax over
(x.f given global)("abc") given ctx
reduces the distance between parentheses. Also, if x is a multiline expression then wrapping it in extra parentheses can lead to some formatting ugliness.

I believe for most cases given comes last and behaves syntactically as expected. For the few cases where it does not come last and is followed by normal parameters: yes that takes some getting used to. But note that these cases were not expressible at all until now, so they should be pretty rare.

Also note that in expressions given has the same precedence as alphanumeric infix operators, so no surprises there.

If having given inside the earlier parens is not implemented, maybe the existing syntax is fine but the recommended style for non-terminal given blocks should be

def foo[A] given(x: X[A]) (a: A) given(y: Y[A]): Foo[A]

to make it as visually clear as possible that the given is associated with exactly one parameter block following the keyword. The orphan (a: A) still looks kind of weird.

I assume allowing early given is desirable so partial application is a bijection, i.e. for every mix of regular and implicit functions val foo = implicit X => A => implicit Y => Foo there is a unique method representation, and for every method there is a curried form with implicit and regular functions?

Or is it just for path-dependent types (as in the example) where you want to avoid superfluous wrapper objects to get the desired API?

1 Like

The limitation of implicits in Scala 2 that is mentioned here is that you cannot have multiple implicit parameter lists.

But I can’t find how this proposal enables multiple implicit parameter lists? Any examples?

Also, why not just have Scala 2 implicit parameter syntax with multiple parameter lists? What’s wrong with:

def myMethod(x: Int)(implicit y: Int)(implicit z: Int): Int = ???

1 Like

I’m still unclear what problem this proposal is trying to solve. Is it so that you can have implicit parameter lists before regular parameter lists?

How about a place-holder for implicit/inferred parameter lists such as (…) for method calls. In method definitions, we can replace (implicit …) by (. … .)

Like:

def m(. a: A, b: B .)(c: C, d: D)(. e: E, f: F .): R

which would be equivalent to, if this was legal:

def m(implicit a: A, b: B )(c: C, d: D)(implicit e: E, f: F): R

This can be called like this:

m(a, b)(c, d)(e, f)

m(a, b)(c, d)

m(a, b)(c, d)(…)

m(…)(c, d)

m(…)(c, d)(…)

Anonymous parameters would be simply underscores, just like in pattern matching:

def m(. _ : A, _ : B .)(c: C, d: D)(. _ : E, _ : F .): R

Let’s please not. It’s visually very hard to parse. Additionally, it’s unpronounceable.

3 Likes

(Often?) you don’t need ... sections if you place . at the beginning of implicit arguments list.

Going back to my proposal we can have:

def f(given u: Universe)(x: u.T)(given Context) = ???
// callable as
f(given global)("abc")(given ctx) // all arguments explicit
f("abc") // Universe and Context are inferred
f(given global)("abc") // only Context is inferred

Presence of given at the beginning of arguments list determines whether we skip or fill implicit arguments list.

However, what if f returns a class with apply method which signature starts with implicit parameters list?

def f(a: Int) given (b: Int) = new X
class X {
  def apply given (b: Int)(c: Int) = ???
}
(f(5) given 8)(11)

Where the given 8 went? To f or to apply?

3 Likes

I like what you are suggesting.

But this approach still has problems. For instance, when you have

def f(given u: Universe)(given s: Bar)(x: u.T)(given Context) = ???

How would be

f(given a)("abc")

interpreted? Whether do we put the first given argument inferred or the second?


Of course, in practice we always can make the call more explicit, like

f(given a)(given the)("abc")

or

f(given the)(given a)("abc")

but the question is how to interpret an ambiguous case?

If you allow for consecutive skippable implicit argument lists then you’ll have such ambiguities in both my proposal and Martin’s one (I think). To disambiguate we could use solution proposed by @curoli which is:

f(given a)(given ...)("abc")
f(given ...)(given a)("abc")

... means any number of implicit arguments.

Consecutive implicit parameter lists probably won’t be popular enough to warrant such special syntax.

Probably in the simplest way - assume you always start with first implicit arguments list from the consecutive ones.

I think that for someone has no previous background in Scala, something like this would be a lot more intuitive:

def max[T](x: T, y: T, ord: Ord[T] = implicit): T =
  if (ord.compare(x, y) < 1) y else x
max(2, 3, ord = IntOrd)
max(2, 3)
max(List(1, 2, 3), Nil)

I personally do not like the given syntax, and think a default-arg-based syntax would be easier for everyone coming from programming languages with default args (i.e. all of them, these days). But by this point I’m probably sounding like a very small shell script, so I’ll leave it at that. Take from this what you will!

20 Likes

Lately I have been using definitions such as the following:

def foo[A, B, C](a: A, b: B)
                given Ctx1, Ctx2, Ctx3: Lifted[C] = ???

where to update one of the Ctx params I would use the implied for Ctx = ??? syntax. Otherwise, if you want to use given at the call site it’s best to define your inferrable parameters in a curried form, abstracted through types, or else you have to supply the entire list.

I do think it feels a bit surprising to someone unfamiliar with the desugared form that the compiler doesn’t generate the other parameters if you only supply one with given at the call site.

So I am going to state here what I stated on the initial proposal PR, which is that I am not a fan of the proposal in general because I think its solving the wrong problems. In summary my points are

  • The real issue with implicits is not how they are defined or used, but the fact that the compiler gives completely terrible diagnostics especially with chained implicit resolutions. This is not going to solve this. Its also unclear how implied is going to work when combined with other keywords.
  • The only real problematic part of implicits is implicit conversions, which people have (rightly) complained about and are already deprecated and considered an anti-pattern
  • The current implicit is actually much more familiar with the current mechanics of Scala compared to this proposal. implicit works the same way as keywords like final, i.e. they modify a definition to state that “this variable can be implicitly provided”. Likewise using it in a parameter list states that “I want to get a variable from a type that has been implicitly defined”. These concepts are not hard to understand.
  • This proposal goes against the nature of Scala where we do have control over how variables are defined. There is a difference between an implicit val vs implicit lazy val vs implicit def (just like there is a difference between final def vs final val), i.e.
    • implicit lazy can combined with trait can be used for compile time DI
    • implicit def an be used for constructing DSL’s in a safe and principled manner.

I would rather the effort be spent improving the current implicits because honestly we are already 80% there. This includes

  • Giving sane compiler errors about missing implicits (i.e. https://github.com/tek/splain should be part of the scala compiler, not a plugin)
  • Fixing discrepancies when it comes to implicit function application. I.e. If you have function like
    def doSomething(key: String)(implicit ctx: Context): Map[String, String]
    
    This is going to behave differently on .apply rather than
    def doSomething(key: String): Map[String, String]
    
    In the former the .apply is going to be providing the Context where as in the latter the .apply is going to point to the key. In both cases apply should point to key and if you want to explicitly provide the implicit context it should be something like
    doSomething("myKey").explicitly(someContext)
    
    This (along with some other changes) would fix all of the currying issues that we have with implicits.

There are probably some other things which I have missed but the tl;dr is that we should fix the current implicits rather than making an entirely new proposal which isn’t even in the spirit of the language.

IDE’s also need to provide better inspections, Intellij as of late is doing an excellent job here (it will actually show you implicit chains and it now has the ability to tell you where implicits are being used which is incredibly handy). When metals does the same a lot of the quality of life issues with implicits should be solved.

9 Likes

I agree the default argument syntax is attractive because it is familiar. The problem is, it leads to a completely different system of implicits than what we have now.

In particular implicit parameters are tied to normal applications. For instance, we currently have a max extension method on lists, which is roughly defined like this:

    class ListOps[T](xs: List[T]) {
      def max(implicit ord: Ordering[T]) = ...
    }

You call it like this: List(1, 2, 3).max.

With the implicit as default params proposal that would now do something different. It would expand to (ev: Ordering[Int]) => List(1, 2, 3).max(ev), i.e. it would return a function that takes an ordering and produces a number. You have to pass a () argument to max to get the maximum: List(1, 2, 3).max(). That’s just how default arguments work: They fill in missing arguments inside an argument list but if the (...) is missing you get an eta expansion instead.

The example also shows something more fundamental: Current implicits and implicits-as-defaults behave differently under partial application. Given

def max[T](x: T, y: T, ord: Ord[T] = implicit): T = ...

What would max[Int] without further arguments expand to? According to the rules for default parameters it would be:

(x, y, ord) => max[Int](x, y, ord)

But according to current (both old and new-style) implicits it would be:

(x, y) => max[Int](x, y)(IntOrdering)

See the difference? The implicit argument moved from definition to use site! This makes implicits a whole lot more like dynamic scoping. Maybe that would work and maybe it wouldn’t. But it sure is completely different to what we have now! And, I believe it’s fair to say that the behavior of implicits under partial applications works fine as is, at least I can’t recall anybody complaining about it.

One could consider tweaking the rules so that default arguments with implicits behave somehow different from normal default arguments so that we can get back the status quo. But that would be a classical “easy instead of simple” move. We buy the easy familar syntax at the price of complex rules that mix things like implicits, default arguments and partial application that formerly were better separated. So, that’s why, much as I like the syntax. I don’t think implicits as defaults are the right way to go.

6 Likes