@mdedetrich @drdozer This thread is about the implicit parameter proposal presented in https://dotty.epfl.ch/docs/reference/contextual/inferable-params.html whereas some of your comments are related to the separate but related proposal in https://dotty.epfl.ch/docs/reference/contextual/instance-defs.html which will should get its own thread on contributors.scala-lang.org soon enough. Please keep your comments on this thread related to the implicit parameters proposal otherwise things are going to get way too confusing.
1 Like
Agreed with @smarter that this belongs elsewhere, but since we are already in it a general question to @drdozer. As someone who has actually written a lot of code in the new system, would you recommend it over old style implicit parameters?
See the difference? The implicit argument moved from definition to use site!
You can do this right now
I think we are talking at cross purposes here. I was discussing that switching to default parameter syntax would go much deeper than just syntax: you get a different set of implicit rules that drastically change how the whole system hangs together and how your programs are behaving.
I was not arguing that List#max could only be expressed using given. As far as I know it is expressible in all systems we deal with, only in different ways.
This is what I mean when I talk about consistency, because this is completely consistent with how the language Scala works, and this is the beautify of having something that is orthogonal, depending on where you place the
implicityou can control how application works; we don’t need to invent a new language construct, we can already do this! This doesn’t need a new language construct, its already possible.
Exactly the same argument applies to given, minus a number of warts that are removed by it.
I agree with this, but i still think it is worth doing. This entire discussion is “how can we make implicits easy and familiar to people without prior experience”, and all the proposals are essentially different syntactic spellings of the same thing rather than any fundamental changes. We’ve literally spent hundreds of posts arguing over what the keyword should be spelled! Buying “easy and familiar” syntax isn’t some side benefit, but is the entire purpose of this exercise.
As far as I can tell, the given syntax achieves this goal not at all. given, instance, etc will have zero familiarity with people not already familiar with typeclasses, which is the vast majority of programmers. Despite the low value, it comes with the same huge up-front cost of changing such a fundamental syntactic construct, and i believe the net value is definitely negative.
The default argument encoding requires more speccing - it is less of a trivial syntactic desugaring, but it actually can solve our problem once and for all! It can make implicits easy and intuitive for people with zero Scala experience! The fact that it requires a bit more finesse in how it is specced, rather than being a dumb desugaring, seems a tiny cost compared to the massive fixed migration overhead either approach would have to pay anyway
There’s also a third road, that I have brought up before: make all the “semantic” or “boilerplate” improvements, without overhauling the core syntax. Things like multiple implicit argument lists, anonymous implicits, better inference for declarations, making the val/def in implicit val/implicit def optional, etc. can be fitted in reasonably nicely without any changes. Avoiding the implicits-vs-currying ambiguity by introducing a keyword like foo(x).explicitly(y) or foo(x)(implicit y) can similarly be done with relatively little change or disruption.
Thus I would very much be in favor of the solution that makes things easy and familiar to users but takes more work to spec and implement. Second place preference would be to do nothing, syntactically: we can support multiple implicit argument lists, anonymous implicits, and other “semantic” improvements with much less invasive syntactic changes. The given/inferred/etc syntaxes definitely seem a distant third, as an expensive non-solution that has similarly huge costs but gives us very little in return.
14 Likes
I think it’s questionable whether “for someone has no previous background in Scala” the version
def max[T](x: T, y: T, ord: Ord[T] = implicit): T
is more intuitive than
def max[T](x: T, y: T) given (ord: Ord[T]): T
We are all long term Scala users, so for us probably the implicit thing is more familar, but for someone coming new to the language, you have to explain the concept of implicit/implied in any case. I think Martin’s proposal is more clear in terms of how you read the statement, more analogous how you would make a mathematical statement etc.
I’m not against the = implicit per se, but I don’t see it as a big advantage, either. What happens with multiple implicit arguments? This is actually quite a common case in all my scenarios. So
def foo(x: Int)(implicit tx: Tx, cursor: Cursor): Unit
This doesn’t map at all to the = implicit thing, unless you want it to become
def foo(x: Int, tx: Tx = implicit, cursor: Cursor = implicit): Unit
which is not only more verbose, but also more difficult to parse visually, compared to
def foo(x: Int) given (tx: Tx, cursor: Cursor): Unit
5 Likes
Maybe we should try to frame this discussion by mentioning the downsides of the current syntax that we’d like to address, here’s a first attempt:
- The current syntax has a concept of implicit parameter list at definition-site, but does not distinguish implicit and explicit parameter lists at use-site. This means it’s easy to accidentally pass an argument to an implicit parameter (in fact this has happened at least once in Dotty itself).
- The current syntax does not allow having explicit parameter lists after implicit parameter lists
- The current syntax does not allow anonymous implicit parameters (except by using context bounds)
I think the most straight-forward way to address 1. is to keep the definition-site syntax the same but require mirroring it at use-site, e.g.:
def minimum[T](xs: List[T])(implicit ord: Ord[T]) =
maximum(xs)(implicit descending)
minimum(xs)
maximum(xs)(implicit descending)
maximum(xs)(implicit descending(implicit ListOrd))
maximum(xs)(implicit descending(implicit ListOrd(implicit IntOrd)))
(I’m using implicit here since this is the current syntax, I think we should defer discussing keyword renaming for now since this isn’t central to the main discussion).
Once 1. is address then we can just remove the restriction from 2. which isn’t necessary anymore, however this can only be done once all the existing code has been updated to use the new syntax at use-site, so that means a somewhat long transition period.
Finally, 3. could be addressed by allowing something like this:
def minimum[T](xs: List[T])(implicit Ord[T]) =
maximum(xs)(implicit descending)
implicit _: Ord[T] could also work but I’d rather not add more uses of underscores in the language. In any case this seems like something that could be discussed separately as its own proposal.
Compared to the original proposal in this thread this has several advantages I think:
- No existing definition needs to be rewritten
- The use-site syntax does not require spending precious brain-cycles thinking about where to put parentheses to please the parser.
- This is a less radical change, so existing users are less likely to see the new syntax as alien and then declare that “Scala 3 is a new language” which at the very least is bad PR I think.
Opinions?
14 Likes
I really like @smarter’s proposal !
I have to say I think the current one involving given, instance and for seems to be making thing more complex not less; I find the new syntax more opaque/magical.
The way implicit values and parameters work currently, I think, isn’t that complicated; the keyword implicit merely acts as a marker for certain values.
1 Like
I like the clear classification of what is wrong with the current syntax!
But I am afraid I find (implicit x) for an argument an abomination. Scala already suffers from plastering implicit on too many things. We now would do even more of this, and in situations where for a beginner it seems to make no sense at all! Marking an explicit argument as implicit is pretty contorted. It can be understood only by a long-term Scala programmer who has learned that implicit is just a tag, without any particular meaning beyond it. We could just as well call it magic, that would probably improve the situation.
The argument that no existing definitions need to be rewritten looks attractive, until you realize that this means (as you say yourself) a very long, possibly indeterminate migration period. If the definition site syntax does not change you have to wait until all use sites are rewritten before you can change the rules for matching definitions with use sites. This might well be never. The same problem hampers @mdedetrich’s explicitly proposal, which we considered before.
given neatly sidesteps all of these problems. It aligns use site and definition site, has a clear migration strategy, and is immediately familiar. I believe very strongly it’s the single, obvious solution. I say this with conviction, since I have thought so long and hard about alternatives. The only alternative that had a similar “why didn’t I think of this before?” effect on me was @lihaoyi’s proposal of implicits as defaults. But, as I have argued before in this thread, that would lead to a completely different system of implicits. So if one was just starting out designing a new language this would definitely be an interesting candidate. But it can’t be a replacement for what we have.
3 Likes
But that’s not the tradeoff. implicits as defaults lead to a different system of implicits. We could try to invest a lot of spec and implementation work to hide this, but all we would achieve is paper over fundamental differences, most likely ruining both implicits and defaults along the way. Good language design is about discovering simple principles of fundamental applicability and find a clear syntax to match them. Form follows function. If we work hard to hide the principles under a syntax that does not fit we have made things worse, not better.
If we would start out with a new language, implicits as defaults could work. We’d have to make the following design choices:
-
When eta expanding a function, parameters with default arguments are instantiated instead of being lifted out. Given
def f(x: Int, y: Int = 2)then
fshould expand tox => f(x, 2). In Scala it expands to(x, y) => f(x, y)instead. I believe either design could work, but it’s impossible for us to change now. Too much code would break. -
We’d have to accept the fact that every function taking an implicit is applied to some parameters, even if it’s just
(). It would not be possible to pass an implicit to a parameterless function. That’s clearly a severe restriction compared to what we have in Scala. Maybe one could get used to it, or maybe it would be too annoying. But, again, I see no way how Scala could migrate to this.
Compared to this, I believe given is equally obvious to a newcomer. And it has the advantage that it is shorter. E.g. in the dotc compiler we repeat the parameter list
def f(x: T)(implicit ctx: Context)
more than 2500 times. There’s lots of code like this, e.g. parallel or concurrent code taking an ExecutionContext. We can reduce that to
def f(x: T) given Context
which is clearly better. The alternative
def f(x: T, ctx: Context = implicit)
is equally clear but adds more bulk. In summary, given has the advantage that it is both familiar and more streamlined than implicits as defaults.
6 Likes
This is why we have a -scala2 flag in Dotty and why the proposal is being suggested for Scala3 and not current Scala. This is also something that can be in scope of scalafix.
If the only good reason for making this a completely new feature with alien syntax is to avoid this migration (which is how it appears from my end) then we need to seriously question the proposal. I agree with @lihaoyi here, I would rather fix all of the quality of life issues with the current implicit syntax (which can easily be done) rather than adding a completely new feature (whos only real merit seems to be sidestepping migration which we have to do in Dotty anyways).
This is also works, I like using the keyword explicitly better because it is clear that you are passing an argument in explicitly but I am not going to fight over it.
When eta expanding a function, parameters with default arguments are instantiated instead of being lifted out.
Will it work if we do it only for the default params which are implicit? This way, the behavior for the implicits and default params remains consistent with Scala 2, at least when it comes to eta expansion.
Will it work if we do it only for the default params which are implicit? This way, the behavior for the implicits and default params remains consistent with Scala 2, at least when it comes to eta expansion.
Yes, but that would be exactly the messy feature entanglement that I was talking about earlier.
Why is this a bad thing? Its like complaining that final is a problem because we plaster it on too many things, or that val is a problem because we plaster it on too many things.
As long as implicit means the same thing (which in this context it does, you either define something implicitly or you ask for it implicitly), its actually a very good thing because of consistency, and this trait is what actually makes a language easier to learn, not harder!
I don’t want Scala to turn into C++/C# where rather than having a few orthogonal constructs that bring consistency to the language we instead keep on rubber bolting on half baked features to tackle one specific subset of some use case and this proposal is heading down this road.
1 Like
This is why we have a
-scala2flag in Dotty and why the proposal is being suggested for Scala3 and not current Scala. This is also something that can be in scope of scalafix.
… and none of it works for cross compilation. Cross compilation is a hard requirement for us. At any stage we need to be able to compile large codebases in both version N and N+1. If we keep definitions the same, it follows that we can change definition/use alignment only if all uses have been rewritten.
This is not even possible under current Dotty for non trivial code bases. Things like traits no longer taking into account ordering (which is a difference in Dotty) will cause certain codebases under current Scala2 to break, I am not sure this is even a worthy cause unless you can guarantee it in 100% of cases (i.e. the source code can cross compile for both Scala2 and Dotty without any behavior in the program).
I mean look at other major libraries now, some already have different sources for different Scala version as it stands right now.
I don’t want Scala to turn into C++/C# where rather than having a few orthogonal constructs that bring consistency to the language we instead keep on rubber bolting on half baked features to tackle one specific subset of some use case and this proposal is heading down this road.
Our goals are the same, but we are clearly in disagreement what that means. I can just say you should give me a bit more credit. You liked the previous version of implicits that I designed. Trust me, the new one will be even better 
3 Likes
Honestly I am failing to see it because I see it being less orthogonal and consistent then the current version of implicits (assuming we fix the quality of life issues which I detailed earlier), I have already detailed my reasons so there is no point in repeating them.
Note that C++/C# went down this road because of the attitude that I am outlining, which is that rather than fixing current issues they keep on adding new ones. They might have thought that the new feature was better than the existing, but its this intention which is what heads us down this road.
2 Likes
Beginners (and in fact even non-beginners) should rarely need to explicitly pass an implicit parameter so I’m not too worried about that.
Fair point, but using a new syntax and keeping support for the old syntax also means a migration period during which some of the libraries you use might be on the old syntax and some on the new, meaning that you have to be very careful at use-site, and that any library update can potentially break your code silently, e.g. if a library defines:
def foo(implicit x: A): A => B
and you call foo(new A), then the meaning of this call will change if the library switches to:
def foo given (x: A): A => B
And your code might still compile!
Regardless, I also mentioned in my proposal that we could use a different keyword, which would look like this:
def minimum[T](xs: List[T])(given ord: Ord[T]) =
maximum(xs)(given descending)
minimum(xs)
maximum(xs)(given descending)
And keep implicit with its current semantics. But I’m not convinced that this is strictly better, and I’m even less convinced that it’s worth the amount of controversy it’ll generate and is already generating as witnessed by this thread :).
4 Likes
@odersky Apologies for polluting the wrong topic. But yes, in answer to your question, the new system is vastly better. I used to have a lot of problems where f(x: X)(implied y: Y): A => B was tricky or even impossible to use as a lambda because it would tend to expect a Y rather than an A, but with f(x) given (y): A => B it works as expected. Having the option to either explicitly name the given parameter, or just state the type is really nice – particularly when composing implied instances together you often don’t need a name. I think there are perhaps 3 times I’ve needed to use given at the call site to disambiguate which instance to use, and one of those went away when the compiler’s inference was improved. To be honest, because you can do zero overhead opaque types now, it’s less overhead to simply let the types guide things rather than manually wrangling your instances. But where I have had to manually supply instances, the given syntax at the call site has worked out nicely.
opaque type Reversed[T] = T
object Reversed {
def apply(t: T): Reversed[T] = t
implied given Ordering[T] for Ordering[Reversed[T]] = the[Ordering[T]].reverse
}
someList.sortBy(Reversed.apply)
The combination of top-level declarations and functional interface sugar and the implied/given/for syntax is just so much nicer, particularly now that the compiler is substantially improved in what it can infer. There are still some rough edges instantiating function types with given parameters in them - it isn’t always obvious how to do it or even if it can be done.
One gotcha I’ve hit several times when reformatting is that because the for keyword is overloaded, if you put the for on a new line, the parser treats it as a comprehension, even if it is ostensibly part of an implied instance declaration.
But how would you teach implicit parameters? You’d have to teach to a beginner that the compiler will infer the parameter for you. Then you’d have to say the way it is done is that the call is expanded to something like this:
foo(implicit arg)
I could not bring myself to teach that. It would be just not sensical enough to teach with a straight face. If you change implicit to given, it would be better but still a little bit weird.
foo(given arg)
Syntactically, what is this given? It has no correspondence to anything else in the language, just like (implicit ...) for parameters is a highly irregular case. By contrast,
foo given arg
is syntactically an infix operator. So we know how to parse this. It fits with the rest of the language. It’s just not what people are used to now, but that should not be the deciding concern.
4 Likes