Overloaded extension methods from parent class get "forgetten" in child class IFF child class adds or modifies an overload

Apologies if this is something that’s already been brought up, but I’ve not been able to find mention of this interesting issue elsewhere. Basically, what I’ve found is that overloaded extension methods sometimes work, and sometimes not.

Overloaded extension methods seem to work beautifully when either:

  • There is no inheritance going on
  • All overloads are declared on the parent and the child doesn’t touch that overload family
  • The child overrides every overload from the parent

Otherwise, compilation fails when trying to use overloaded extension methods. The main failure mode seems to be that a child class modifying the family of overloads in any way causes all the overloads from the parent class to be invisible to the compiler.

Here’s some minimal code that reproduces the issue in Scala 3.4.2:

trait Foo:
  extension (a: Int)
    def foo(other: Int): Int = a + other
    def foo(other: String): (Int, String) = (a, other)

object Bar extends Foo: // Having the TWO override lines immediately below both commented or both uncommented results in expected behaviour
//  extension (a: Int) override def foo(other: Int): Int = a + other  // Uncommenting just this line results in a failure to compile on the second print statement
//  extension (a: Int) override def foo(other: String): (Int, String) = (a, other)  // Uncommenting just this line results in a failure to compile on the first print statement
//  extension (a: Int) def foo(other: Double): Double = other  // Uncommenting just this unused additional overload causes a failure to compile on both of the first two print statements, but works fine if the other two overloads are also uncommented

  extension (a: Int) def bar(other: Int): Int = other // Adding or not adding this non-overload extension has no effect on the result
end Bar

import Bar.foo
import Bar.bar

println(1.foo(1))  // Expected: 2
println(1.foo("1")) // Expected: (1, 1)

println(1.bar(1)) // Expected: 1; always works

Interestingly, it seems like part of an overload family can still work after modification in a child class, but only if that part has an incompatible extension target.
For instance, consider

println("a".foo("b"))

If we define the following extension on Foo:

extension(a: String) def foo(other: String): (String, String) = (a, other)

then our println continues to work regardless of whether any of the lines in Bar are uncommented because (I think) a String is never an Int.
On the other hand, the very generic

extension[L](a: L) def foo[R](other: R): (L, R) = (a, other)

which should also handle our println case gets messed up by any of the extension(a: Int) overloads, because (I think) the compiler prefers Int to generic L. If none of those overloads are performed, this extension works though.

Small update: Just tried the same test code on 3.5.0-RC2 and found the same issue.

This is “just” (pardon the expression) ordinary overload resolution.

For an application, m(42), if m(Int) is overloaded with one alternative in a subclass and the other in a superclass, it will pick the alternative in the subclass.

For your extension methods, it doesn’t check the next parameter list to see if it is a valid application. That is, it doesn’t compare m(Int)(Double) and m(Int)(Int), where the first is in the subclass and the second in the superclass. It picks the first one, and the expression 42.m(27), which is taken as m(42)(27), fails to typecheck because the double is expected.

On the other hand, given two alternatives defined at the same “level”, it can’t distinguish them just from the first parameter list, and so it will consider the second parameter list.

I guess then the question is, should this be how overloads are resolved? From a user perspective, it feels extremely arbitrary. It defies (my) normal inheritance expectations to have it matter to the consumer of a class whether a particular method is defined in that class or in a parent of that class.

The fact that the overloads sometimes work, but sometimes don’t, and they would work if I defined them all on just one level of the type hierarchy, seems to just block code that relies on overloads from benefiting from inheritance.

I’m pretty sure it also breaks type expectations, in the sense that:

  1. Foo contains a method foo that will resolve 1.foo("1")
  2. Bar <: Foo
  3. therefore Bar contains a method foo that will resolve 1.foo("1")

Does not hold in this case.

1 Like

Yes, I think you must adjust your expectations in this case.

The intuition is that the member of the subclass is in the narrower type and is preferred. This also applies for resolving implicits, so it’s a simple and handy rule.

“Overloading is evil.” So adding more overloads would fall under, “Two wrongs don’t make a right.”

Inheritance can be brittle.

It’s determined at the definition, not the use site, so it’s the person coding the subclass who has to know what they’re doing. That’s true for all API design, of course.

“sometimes work, but sometimes don’t” describes almost all software.

That’s changing for implicits soon, right? Change rules for given prioritization by odersky · Pull Request #19300 · scala/scala3 · GitHub

2 Likes

I mean, that of course makes sense, child members should always be preferred – but surely not to the blind exclusion of all applicable parent members? When an interface in the child does not match the types required, surely the parent can also be checked for matching interfaces? After all, this is what happens when the child doesn’t declare any new members, so it seems arbitrary that the process would be blocked by the child adding or modifying certain members.

The fact that I can potentially work around this issue by naively copy-paste reimplementing every single non-overridden parent method into the child (which is, intuitively, what a parent-child inheritance means – child gets all non-overridden parent members) shows that currently, this is a boilerplate issue.

Adding more overloads is just overloading, so only one wrong :wink: But more seriously, if overloading is evil, why does Scala support it at all? If it’s a JVM compatibility issue, it would still be possible to prevent Scala source code specifically from implementing overloads, or at the very least including a compiler warning. Having a language feature (overloading) interact poorly with another language feature (inheritance) feels like a bug.

And just a personal note here – I don’t think overloading is a bad thing at all. I think it’s a terrible solution for giving default arguments to methods, and I think that like givens, inheritance, and being able to name stuff using almost any alphanumeric string, it’s possible for an inexperienced programmer to create an unholy mess using the feature. But the ability to say to the compiler, “hey, please pick the correct interface for me from this set of interfaces” is a fundamentally fantastic ability to have.

I agree that it’s very brittle in this case, but I don’t see why it should be, especially since, at minimum, the issue could be fixed with (ideally autogenerated) boilerplate.

Kinda. In my use case, Foo is API code, but Bar is user code. I’m creating a library of extensible and mixable traits to allow for creating very complex typeclasses really easily. Working things out in a way that allows extension of the traits in my API under the restrictions imposed by this issue, without imposing undue limitations on the user, is difficult. Also, consider this snippet:

trait Foo:
  extension (a: Int)
    def foo(other: Int): Int = a + other
    def foo(other: String): (Int, String) = (a, other)

object Bar extends Foo:
  extension (a: Int) override def foo(other: Int): Int = a + other
end Bar

val boop: Foo = Bar
import boop.foo  // This import works
//import Bar.foo  // Using this import instead doesn't, even though it's the same object

println(1.foo(1))  // This prints 0 !!! i.e. it uses the definition from Bar
println(1.foo("1"))  // And this prints (1, 1), using the definition from Foo

Doesn’t the fact that import boop.foo works (and even makes use of the override in Bar) but import Bar.foo doesn’t seem arbitrary? Especially since the override in Bar is what then gets used.

Agreed, but almost all of software development that isn’t adding new features is identifying and fixing those cases of unreliability or failure.

I’ll just add in case I’m coming off as hostile here (I really don’t mean to be, but you know how communicating over the internet is! :sweat_smile:) that I’ve found this discussion very productive and it’s already helped me think of a few potential workarounds. I just think this is an area where Scala could shine - the library that I’m busy putting together just wouldn’t work in any other language that I know of, and what allows it is the unique mix of features that Scala, incredibly, manages to have while keeping its types strong and safe. I just think this is an area where Scala can genuinely be directly improved and made more safe and intuitive.

Thanks for the link @spamegg1 ! I read through that thread, but I’m not gonna lie, my brain is too smooth to tell if applying the same logic to overload resolution would or would not fix my issue here :sweat_smile:

1 Like