Extension Methods
I think some more elaborate example would be helpful to get the feeling, like arithmetic progression sum.
// ??? - what givens, extensions, etc are needed here to make below code compile without changes?
def arithmeticProgressionSum[T: Numeric](firstTerm: T, step: T, termsNum: Int): T =
termsNum * (2 * firstTerm + (termsNum - 1) * step) / 2
val sum = arithmeticProgressionSum(new java.math.BigInteger("3453"), new java.math.BigInteger("245"), 745)
println(sum)
The docs give this example:
given (given outer: Context): Context = outer.withOwner(currentOwner)
This makes anything about existing implicits look like the paragon of clarity in comparison. It looks like the “Buffalo buffalo buffalo” puzzles sometimes given in e.g. philosophy of language classes.
Something should change to make this impossible. For instance, making give provide things and given ask for them would fit naturally into common language usage instead of admitting mindboggling things that read like “given given Foo Foo”.
The syntax of given instances and aliases are both weird special cases. Nothing else in the language works like that. I understand what the point of each is, but the instances look like a weird hybrid of what you’d do for a def and what you’d do for a class, making it anything but clear what the point is. That regular instances use =, e.g. val five = 5, while given instances do not, is especially befuddling. (That object O does not use = is already befuddling enough, but there at least since it isn’t parameterized you can reason that it both declares the singleton class type and automatically gives it a unique name–that is, it’s a sugar for sealed abstract class O.type { ... }; lazy val O: O.type = new O.type {}.)
Since given instances can be parameterized, and aliases can alias to defs, it’s also quite a bit of work to puzzle out what is gained by having both constructs.
Given clauses are pretty nice, though I found the foo(x) given y syntax vastly superior visually in simple cases (but untenable in complex cases). However, the document doesn’t explain whether the (implicit thing: Thing = Companion.defaultThing) pattern works any longer. It looks like no, given the grammar. It also isn’t clear to me whether def f(given foo: Foo, i: Int) can be called as f(5). I don’t think I missed the example?
It’s not clear why they are called “given clauses”. Parameters aren’t called clauses in other places.
It’s not clear how the values are filled in with multiple relevant givens in scope. Presumably something like implicit resolution, which right now forces awkward tricks like creating superfluous traits just to drive resolution? Or…? Should be spelled out.
Looks fine! Seems pretty clear to me. Also doesn’t seem to have changed.
This is a solution to only half the problem (actually, only half of half of the problem). Not knowing that import mylib._ brought in implicits is fixed by givens having to be import mylib.{given, _}. So that’s something. But you still don’t actually know what you’re getting from there. So what you’re required to do doesn’t actually solve even this half of the problem; you have to take the explicit-type option and add all that boilerplate.
But import mylib._ not working because you forgot to bring in implicits is even worse because now they can’t conveniently be inside mylib any more. This suggests that to recover parity with the old way, best practice will be to spam import mylib{given. _} everywhere, whether or not you need it…which then gives you the first problem right back again.
Instead, what we need is a less awkward way to specify precedence, along with deprecation messages. So you could
import mylib._
and if you’d get the implicits (givens) that would make things work, but if you marked them all deprecated, you could have a message like, “This invocation uses a default given from mylib. Please import mylib.fast._ to select the fast versions of these implicits, or import mylib.accurate_ to select the accurate ones.”
Maybe it’s too late for a change like this, but I think the new version is strictly worse than the old version as it stands.
It seems like a tooling problem is being hoisted onto the programmer in an unwelcome way. I’d rather the compiler do more for me e.g. by suggesting places to find givens that I need, than force me to manually do that kind of work myself.
Finally, it’s not clear that the givens can be brought in by type with the full types that could appear in the given alias or instance.
I love the idea, but the syntax is rather ugly. I wouldn’t assume that I’ll just get used to the syntax and eventually love it, because after like a decade of experience with C++ I still find the lambdas impossible, and after a couple of years of Rust I still find |x| bar(x) lambda syntax way less readable than Scala’s x => bar(x).
I’d prefer something like
extend (c: Circle) with {
def circumference: Double = c.radius * math.Pi
}
which could be shortened to
extend (c: Circle) with circumference: Double = c.radius + math.Pi
in the case where you wanted a one-liner.
If you complain that paring this down to be more refined and elegant will just get you back at the current extension syntax, I plead guilty. Yes, it does. Sometimes the extra fluff helps. I think this is one of those times.
This is so awesome! (Note: still need clarification on how givens are resolved when potentially conflicting.)
This seems like kind of a weird and ugly hack. Isn’t there some better way to get around the problem so we don’t have to worry about it? Like, what’s wrong with always having the by-name behavior to avoid circularity?
It’s especially weird because by-name suggest lazy runtime behavior, but here it’s being used to drive compile-time differences. This is a fundamental shift in how by-name parameters work.
Seems reasonable, save for not mentioning how to migrate (implicit thing: Thing = defaultThing), as I brought up earlier.
This was indeed more a “demonstrate the limits of what this can do” example than a piece of recommended code. I propose we remove the example.
What it did is that it allows us to define an inner given using an outer one. As @tarsa asks, it’s not obvious why this should work. In fact it works because trying to resolve the given with itself would produce an implicit divergence. In Scala-3 is a recoverable error, so the outer given would be searched next and that one would succeed. In current Scala the same trick would not work since there divergence errors are fatal. But once can make it work rather perversely by writing this:
implicit val ctx = implicitly[Context].withOwner(currentOwner)
It’s essential that the type of ctx is not given explicitly, since that prevents ctx from being elaborated on its rhs. To my surprise there are actually widely used libraries out there that rely on this trick. But all this is rather too specialized and deep for a casual discussion and it has nothing to do with anonymous instances. So we should just drop the example.
To clarify: I don’t think
given (given Foo): Bar
is a clear and obvious thing either. Having two identical keywords to mean two different features, used in close proximity where they force rapid mental shifts in interpretation, is not good. The given clause has the virtue of being signaled well in advance by def and method name and stuff; here it requires an instant mental flip. This is obvious:
give Bar given Foo
I’m not saying that’s what the syntax should be, necessarily, just that it helps the reader out.
Also, removing the example of given (given outer: Context): Context doesn’t prevent it from being used, or for the problem it illustrates to be not-a-problem. It just seemed particularly egregious to me.
Thanks @Ichoran for the write up. I’d like to second the concerns about the syntax of given instances. I have already explained here why I think we should just have the “given aliases” syntax (like for val and def definitions).
EDIT: this is very similar to the link @julienrf posted just above, I didn’t see it until after I posted this message!
I agree that the syntax for given instances looks like a hybrid thing, by contract given aliases seem more natural to me. So could we drop given instances? This would be one less thing to teach! Instead of:
trait Foo { def foo: Int }
given Foo {
def foo: Int = 1
}
we can use a given alias:
given Foo = new Foo {
def foo: Int = 1
}
The former translates to an object, the latter translate to a lazy val and an anonymous class, those should be semantically equivalent.
And instead of:
trait Bar[T](x: T) { def foo: T }
given (given x: Int): Bar[Int](x) {
def bar: Int = 1
}
we can also use a given alias:
given (given x: Int): Bar[Int] = new Bar[Int](x) {
def bar: Int = 1
}
The former translates to a class and a def, the latter also translates to a class and a def and the semantics are again equivalent as far as I can tell.
Of course, the main issue here is the repetition, so let’s invent even more syntax (sorry!):
given Foo = new {
def foo: Int = 1
}
given (given x: Int): Bar[Int] = new(x) {
def bar: Int = 1
}
The unparameterized version actually already works in Dotty, the parameterized one has been suggested before by @propensive in a more general context, and @odersky gave some reasons it might not be worth it in Expunging `new` from scala 3 - #124 by odersky, but I think it’s worth revisiting in the context of givens: it it allows us to drop given instances that would be a huge simplification and more than make up the extra complexity it introduces.
The (non-alias) given instance syntax actually looks better once we allow a with:
given Monoid[String] with
def (x: String) combine (y: String): String = x.concat(y)
def unit: String = ""
given Monoid[Int] with
def (x: Int) combine (y: Int): Int = x + y
def unit: Int = 0
BYOB (“bring your own braces if you like them”). I don’t like the alias syntax with a new. I’d like to get away from using new – it’s connotations are far too operational! Writing
given Monoid[String] = new {
def (x: String) combine (y: String): String = x.concat(y)
def unit: String = ""
}
emphasises the point that we create a new object to be a monoid. But nobody cares about that and it is in fact misleading.
This would also provide a possible path to support extension methods that need to be anchored on a generic type, if the extension target can have it’s own type arguments:
extend[A] (a: A) with {
def pure[C[_]](given Applicative[C]): C[A] = summon[Applicative[C]].pure(a)
}
Which is effectively impossible with the current syntax, as the only way to enable 1.pure[Option] involves implicit conversions which are (justifiably) made painful to use.
extend would be nice to use, but we can’t. It’s too common as a word to take away as a keyword. And in this position the option of making it a soft keyword is out of the question since it would create too many ambiguities.
Having multiple type parameter lists in extension methods would also be great, but there’s another caveat. The way things stand, it’s a huge effort to spec and implement. I do not think we’ll get there for 3.0, but hopefully for a release soon after. In the meantime I did change the Tasty format so that it can accommodate such a change if it happens. https://github.com/lampepfl/dotty/pull/7504
I’m not familiar with the specific token parser limitation but does a word combination reduce the ambiguities if one of them is a keyword? Something like
extend class (c: Circle) with {
def circumference: Double = c.radius * math.Pi
}
Or maybe even go “silence of the lambs” and use the extends keyword 
It’s unclear if extension methods are related to given instances or not. The documentation provides three ways to extend a class with a method:
def
def (foo : Foo) op (bar : Foo) = ???
given ... extension
given fooOps : extension (foo : Foo) {
def op (bar : Bar) = ???
}
given...AnyRef and the new style for def
given fooOps : AnyRef {
def (foo : Foo) op (bar : Bar) = ???
}
Problems:
The fact that we have of three different syntactic sugars to do the same thing (in case of a single method) is a concern. How frequent is it to extend with just a single method to merit the first example?
It is unclear how extension methods are imported into scope. Do they follow the same given import rules? They are suppose to be separate mechanism from given instances, so this is highly confusing. The first example does not use the given keyword at all, so is it included in regular imports?
I would like to know. What was so wrong with implicit classes other than using implicit?
In general, to my eyes, all the options for anonymous given instances, clauses, etc., are just more confusing than actually cleaning up the code, because I have to look for and parse two kind of texts (with and without names).
I would much rather have _ : Something for anonymity. Yes, this is an “new” underscore usage, but still in the context applied to declare something “we do not care about”.
Edit: I also wonder how code completion tooling would react to this. A tool will not know if we require suggestions for term or type names.
Does the new implicits have anything better for prioritizing them yet? Now that we’re not bound to the syntax of val/def anymore I can’t see why something like given(n) can’t be used to give a custom priority.Saves you from creating a whole new trait for each priority level.
If we want to completely replace implicit, we also need different names or even different mechanisms for:
DummyImplicit. Used to differentiate between different methods with the same signature after erasure. To me this always felt like a hack. Why can’t the compiler do it by itself?@implicitNotFound. Can there a better way to provide custom error messages?@implicitAmbiguousI would argue such annotation is redundant. Rust compiler can give suggestions for missing imports (implicit values candidates) itself, by scanning the classpath. Scala compiler could do that too. I’ve raised that issue in the topic about implicits (the topic about “why implicits in Scala aren’t a runaway success, but people love Rust with typeclasses”).
Untrue. I use it frequently to provide custom error messages. It’s not just a better suggestion for missing imports.FWIW, I currently use macros to customize the error annotation.
There are good example about postconditions in “implicit function types”.
I like this pattern.
But I sometimes need to use implicit conversion and extension in my practice for such case.
So the code:
import PostConditions.{ensuring, result}
val s = List(1, 2, 3).sum.ensuring(result == 6)
will be converted into something like:
{ //we will need braces if we do not want that isDistinctFrom is visible outside of expression.
import PostConditions.{given,_}
val s = List(1, 2, 3).sum.ensuring( 6.isDistinctFrom)
}
It is not very convenient and it need time to accustom. So it seems that more preferable construction will be something like:
val s = new Postcondision(List(1, 2, 3).sum){
ensuring( 6.isDistinctFrom)
}
I am not sure that is is good way for scope reusage. But it seems it is still the less error prone and succinct way in such case.
It seems that there are still not enough context reusage in such patterns in general.
Implicit Conversions
Unfortunately they don’t have parity with existing implicit defs yet. There are at least three differences:
Conversion, you’ll have to define two traits and override a runtime method with an inline method. example: [2]implicit punning technique – replacing a typeclass’s companion with a slew of implicit defs to ensure that whenever a typeclass is imported, its syntax is ALWAYS available with it. examples: [usage] [implementation with overloading]. Dotty’s extension methods and the fact that they’re available whenever an implicit is in lexical scope, which happens in strictly more cases than when a type name is imported make this trick unnecessary in most cases, except when extension methods are insufficient like in @morgen-peschke’s example.Honestly I don’t find the syntax great. given doesn’t seem to give us anything that implicit doesn’t: if we find-and-replace all the givens with implicits, we’d get all the benefits and much less disruption. The usage of :, and how strongly it binds left and right, also goes against what someone programming in Scala would expect.
Some more specific issues:
given intOrd: Ord[Int] {
def compare(x: Int, y: Int) =
if (x < y) -1 else if (x > y) +1 else 0
}
If Scala didn’t have refinement types maybe this could be ok, but as is I find this extremely hard to parse. And the precedence is all wrong: normally { has higher precedence than :, but here it’s reversed.
given instances look much worse than what we currently have with implicits:given Ord[Int] { ... }
given [T](given Ord[T]): Ord[List[T]] { ... }
Neither of these look particularly good. The top one has a type expression in a place that looks like no other language construct in existence, while the bottom looks like keyword soup with the two givens that do not match to any normal english usage of the word.
givens honestly would look fine as implicit
given global: ExecutionContext = new ForkJoinPool()
implicit global: ExecutionContext = new ForkJoinPool()
In fact, that’s almost the syntax now! We just need to make the def/val/lazy val optional, and we’re done.
given, :, extension and { seems all wrong in the given examplesgiven stringOps: extension (xs: Seq[String]) {
def longestStrings: Seq[String] = {
val maxLength = xs.map(_.length).max
xs.filter(_.length == maxLength)
}
}
Overall the usage of : in the given syntax doesn’t work out. It looks too much like something we’re already familiar with, and comes with baggage of the precedence we’re already used to, but is parsed in a totally different way. Using a new keyword would be far better than overloading : in this way
Can you outline (again if I missed it) why extend is too common but given is rare enough? What’s the threshold (in say of scala files with a particular identifier) ? Was this measured on some corpus?
I find extend or extension for making extension methods far more straight forward and important enough to merit a keyword IMO. Reusing keywords was part of what made implicit confusing (conversions vs the static-dependency injections).