What's up with type arguments in patterns?

Language Design
1

I’ve gotten into the habit (form any of years) of simply never using type arguments in patterns (case Foo[T](x, y)) because I thought they didn’t work. Recently, I discovered that they in fact do work, just not in the way you might think.

class Foo[T] {
  def unapply(x: Int): Option[Int] = Some(4)
}

object Foo {
  def unapply[T](x: T): Option[Int] = Some(5)
}

1 match {
  case Foo(x) => println(x + " then")
}

1 match {
  case Foo[Int](x) => println(x)
}
// prints 
// 5 then
// 4

I can’t seem to find the history of this IMHO very surprising behavior. I found this bug, but it is still open. I also found some discussion by @LPTK here. Nieither seems to acknowledge that you can (kind of) have type parameters in patterns, just not in the way that you would expect.

Also note some other surprises – this compiles:

class Foo[T]() {
  def unapply(x: Int): Option[Int] = Some(4)
}

1 match {
  case Foo[Int](x) => println(x)
}

but this doesn’t

class Foo[T](x: Int = 1) {
  def unapply(x: Int): Option[Int] = Some(4)
}

1 match {
  case Foo[Int](x) => println(x) // error: missing arguments for constructor Foo in class Foo
}

but you can of course to

class Foo[T](x: Int = 1) {
  def unapply(x: Int): Option[Int] = Some(4)
}

val M = Foo[Int]()
1 match {
  case M(x) => println(x + " then")
}

It seems to be that the unsurprising thing would be for type arguments in a pattern to be interpreted as type arguments to the call to unapply. I think there must be some reason why it doesn’t work this way, and I’m hoping someone here might know why. Thanks!

1 Like
2

Regarding your last example note that in Scala 3 parameterless constructors are on purpose equivalent to constructors with a single empty parameters list to make Universal Apply Methods pattern work nicely. E.g.

class Bar
val bar1 = new Bar
val bar2 = new Bar()
val bar3 = Bar()

class Baz()
val baz1 = new Baz
val baz2 = new Baz()
val baz3 = Bar()

In case of patterns in pattern matching I would say the best way of thinking about them is that always only the last list of parameters in parentheses are things that you extract - all other parentheses and brackets are used to construct an instance of an extractor. The only problem here is that constructing the extractor on the fly is not fully supported. Currently you can write

case Foo[Int](x) => println(x)

in scala 3 but not in scala 2. Not really sure if this was changed on purpose or accidentally. Potentially you could also have patterns like

case Quux(10)(x) => println(x)

which might be really convenient in some situations but it looks like this is just not supported yet although this feature would nicely fit into the existing syntax.

3

Is this true – particularly the part about “all other parentheses”? I checked and this also works:

class Foo[T](x: Int = 1) {
  def unapply(x: Int): Option[Int] = Some(4)
}

def M[T] = Foo[T]()

1 match {
  case M[Int](x) => println(x)
}

so it is true that empirically, the stuff before parens is evaluated to get an extractor object.

But I would also ask: why? It’s very weird the type parameters of an unapply can be inferred by the compiler but not specified by the programmer. There’s very little loss in saying that type arguments to a pattern match are passed to the unapply method. I think that is what most new programmers would expect? Iff you want to use an extractor object, you can always do val MyExtractor = <whatever expr>, which is more powerful because you can specify Quux(10). I could even see a case for making y match { case Foo[Int](x) => ... } be equivalent to Foo.unapply[Int](y) but y match { case (Foo[Int])(x) => ... behave as it does now.

Another option that I don’t like as much: since patterns are “backwards” in that the return type comes in what is normally argument position, you could also have the type parameters come backwards too:

case Foo(x)[Int] => ...

could mean Foo.unapply[Int] and case Foo[Int](x) could mean what it does now.

1 Like
4

From what I remember, this was to make it nicer to have patterns for things like

case TypeTag[Int]() => ...

without having to store the extractor in a variable. We could add a warning when a class and its companion both define unapply?

3 Likes
5

I don’t know, this all feels pretty broken. This is a weird case where adding explicit type parameters changes what function is called. This doesn’t compile:

class Foo[T] {
  def unapply(x: Int): Option[Int] = Some(4)
}

1 match {
  case Foo(x) => println(x) // error: Foo cannot be used as an extractor in a pattern because it lacks an unapply or unapplySeq method
}

and neither does this

class FooImpl {
  def unapply(x: Int): Option[Int] = Some(4)
}

def Foo = FooImpl()

1 match {
  case Foo(x) => println(x + " then") // not found: Foo
}

or this

class FooImpl[T] {
  def unapply(x: T): Option[Int] = Some(4)
}

def Foo[T]() = FooImpl[T]()

1 match {
  case Foo[Int](x) => println(x + " then") // error: Foo must be called with () argument
}

So the rule is: for a pattern X[Y](z), you call a paramless constructor for class X[Y], no-arg constructor for class X[Y](), or paramless method def X[Y] (but not no-arg function def X[Y]()) and then call unapply on the result. For a pattern X(z) you look for a stable identifier X (either an object or val) and call unapply on that identifier.

I think it would be possible to keep the current behavior but also add the behavior where you pass the type arguments to the stable identifier (as the second option)? That way, both case Foo(x) and case Foo[Int](x) could call the same function provided that the type parameters can be inferred from context? And it would no longer be the case that type arguments on an unapply method would be declarable but not accessible in a match.

6

I just submitted Better handling of type parameters in constructor pattern by odersky · Pull Request #15356 · lampepfl/dotty · GitHub, which does pretty much what you suggest. In that PR, we first try Foo.unapply[T](s) and only if that fails we try Foo[T].unapply(s). I think the first interpretation is the more natural one. But there’s a (extremely small) chance that this invalidates existing code.

3 Likes
7

Fantastic! Even better. I was worried about backwards compatibility but if you’re not then great!

8

And not to press my luck, but while you’re looking at this code, do you thoughts on https://github.com/lampepfl/dotty/pull/13807 (underlying feature request here More powerful implicit resolution in case matches · Issue #242 · lampepfl/dotty-feature-requests · GitHub). Related because once it becomes possible to have type parameters on unapply, it might. be nice to improve type inference of those parameters.

2 Likes
9

About backwards compatibility. It seems that in Scala 2 we could not write type parameters in constructor patterns, period. So now that we can do that in Scala 3, it seems best to prefer the
most obvious interpretation.

About #242: That’s a completely different ballgame and much more difficult to achieve (probably unfeasible). Information in Scala flows exclusively from the selector to the pattern, never the other way round. Changing that would mean we have to completely rethink everything about pattern matching.

10

Information in Scala flows exclusively from the selector to the pattern, never the other way round.

In principle, you I think it’s you should be able to get the same performance (and use the same code) for normal typedApply if you create an inverted function out of the selected unapply method no? That is, if you select def unapply(x: Int): Option[(String, Boolean)], you should perform type inference as if you have (String, Boolean) => Int? I’m sure there are some details to work out, but I’m surprised you think it’s unfeasible?

11

It’s a question of where types flow. If you have a function f(x: Int): Option[(String, Boolean)] then you do compare the result Option[(String, Boolean)] with the expected result type and derive constraints from that. But sometimes there is no expected result type, for instance when you apply f like this:

val y = f(22)

In the case of unapply there is never an expected result type since typing flows from the selector to the pattern.

12

I put up a proof-of-concept of what I meant here. The main part is here, pasted here for clarity:

// If we have e.g. unapply(t: T)(using u: U): Option[(Int, String)], produce the MethodType representing
// (Int, String) => (using U) => T
val invertedUnapplyAppFunType = {
  var currType = mt.resultType
  var implicitParamLists: List[List[Type]] = Nil
  while (currType.isImplicitMethod) {
    val mtType  = currType.asInstanceOf[MethodType]
    implicitParamLists =  mtType.paramInfos :: implicitParamLists
    currType = mtType.resultType
  }
  MethodType(argTypes, implicitParamLists.foldLeft(unapplyArgType)((resType, paramList) => ImplicitMethodType(paramList, resType)))
}
val unapplyApp = adapt(ApplyTo(tree, unapplyFn, invertedUnapplyAppFunType, null, FunProto(args, invertedUnapplyAppFunType)(this, ApplyKind.Regular), invertedUnapplyAppFunType), unapplyArgType)

14 tests fail when running testCompilation, and I think they are all edge cases around varargs/unapplySeq, empty unapply argument lists, and get/isEmpty matches.

I don’t understand your point about information flow. I agree that you are accurately describing the current implementation, I just don’t see why it is infeasible to do any better. My implementation shows that you can (I think, obviously I might have missed something, and maybe some of the test failures aren’t just edge cases that I haven’t handled yet), and I don’t think there’s any efficiency problems – type inference should be no more expensive than the same function call outside a pattern.The implementation isn’t pretty right now, but it can be cleaned up, and probably there’s some stuff I could do better if I were more familiar with the code.

Is the worry just that if you add some extra flow, it might be worse because people will be confused about what will infer and what won’t? I’d be surprised if that were a problem since the compiler already surprises me in many cases – mostly when it can’t infer things. But it’s a fair concern that one shouldn’t make it worse.

EDIT: there is at least one thing that my approach doesn’t handle well, which is implicit default arguments

object Matcher{
    def unapply(s: String)(implicit secondParam: Option[String] = None) = Some("Got: " + s + " and " + secondParam)
  }

I’m also aware that there may be some problems with co/contravariance when inverted the function signature, but I haven’t found any yet.

13

Scala 3 no longer has default implicit arguments.

14

This test seems to indicate they are still hanging on somehow.

15

I don’t think that will work in the general case, unfortunately.

FunProto(args, invertedUnapplyAppFunType)

means that args will be evaluated as expressions on demand during type inference. But args is really a list of patterns, it would not always work to treat them as expressions.

16

That appears not to be the case, since so many tests pass. I believe that the recursive typing of args is still in “pattern” mode and so are handled in the expected way.

Taking a step back, I think the core question is: should

object Unapply {
  def unapply(x: T): Option[R1] = ...
}

(x: T) match {
  case Unapply(y: R2) => f(y)
}

compile to (conceptually)

Unapply.unapply(x).map((y: R2) => f(y))

in the case where R1 <: R2, i.e. should it skip the cast from R1 to R2 and allow type inference to take advantage of that? Of course if R2 <: R1 then you have to (conceptually) desugar to something like

Unapply.unapply(x).flatMap((y: R1) => if (y.isInstanceOf[R2]) Some(f(y.asInstanceOf[R2])) else None)
17

But if the typing is in pattern mode, then we lack an expected type for the arguments which is otherwise assumed to exist for patterns. No matter how we turn it, the system is just not designed for this