Twice-generic extension methods do not appear representable using the extension syntax

Splitting from this comment

Occasionally an extension method will need to be written that is generic on two axes: the anchoring type and a type argument to the method itself. A simple example of this is the pure method on cats.Monad.

This is representable in Scala 2 in a fairly straightforward manner:

implicit final class Lifts[A](val a: A) extends AnyVal {
  def pure[F[_]](implicit P: Pure[F]): F[A] = P.pure(a)
}

An expanded live example can be found here.

This does not appear to be representable with the new extension method syntax. I tried a couple different approaches, and have annotated the following code snippet with the errors caused by uncommenting the various attempts:

trait Pure[F[_]] {
  def pure[A](a: A): F[A]
  
  extension [A] (a: A) def pureOne: F[A] = pure(a)
}
object Pure {
  given Pure[List] {
    def pure[A](a: A): List[A] = a :: Nil
  }
  
  extension [A,F[_]: Pure] (a: A)  def pureTwo: F[A] = summon[Pure[F]].pure(a)
  
  final class PartiallyAppliedPureThree[A](val a: A) extends AnyVal {
    def apply[F[_]: Pure] = summon[Pure[F]].pure(a)
  }
    
  extension [A] (a: A) def pureThree: PartiallyAppliedPureThree[A] = new PartiallyAppliedPureThree[A](a)      
}

def trialOne() = {
  // Without the explicit import of givens, fails with:
  // "value pureE is not a member of Int"
  import Pure.{given _}
  
  // Fails with:
  // value pureE is not a member of Int.
  // An extension method was tried, but could not be fully constructed:
  //
  //     Pure.given_Pure_List.extension_pureE[List](1)
  //println(1.pureOne[List])
  
  // Fails with:
  // Found:    (1 : Int)
  // Required: List
  //println(Pure.given_Pure_List.extension_pureOne[List](1))
}

def trialTwo() = {
  // Without explict import of method, fails with:
  // value pureTwo is not a member of Int
  import Pure.pureTwo
  
  // Fails with:
  // value pureTwo is not a member of Int.
  // An extension method was tried, but could not be fully constructed:
  //
  //     Pure.extension_pureTwo[List](1)
  //println(1.pureTwo[List])
  
  // Works if called explicitly with explicit type parameters
  println(Pure.extension_pureTwo[Int,List](1))
}

def trialThree() = {
  // Without explicit import of method, fails with:
  // value pureThree is not a member of Int
  import Pure._
  
  // Fails with:
  // value pureThree is not a member of Int.
  // An extension method was tried, but could not be fully constructed:
  //
  //     Pure.extension_pureThree[List](1)
  //println(1.pureThree[List])
  
  // Fails with:
  // Found:    (1 : Int)
  // Required: List
  // println(Pure.extension_pureThree[List](1))
  
  // Works, if called explicity with the type parameter second
  println(Pure.extension_pureThree(1)[List])
  // More explicit version of the preceding call
  println(Pure.extension_pureThree(1).apply[List])
  
  println(1.pureThree.apply[List])
}

@main
def run(): Unit = {
  trialOne()
  trialTwo()
  trialThree()
}

A live version can be found here

Additional examples:

The official example for Functor also displays this behavior, and creates a really unpleasant experience for folks who are used to the Scala 2.0 Apis.

In this lightly modified example, the first of these work, and the third fails:

List(1,2,3).fmap(_.toString)
List(1,2,3).fmap(Some(_))
List(1,2,3).fmap[Option[Int]](Some(_))

If you want to make sure you get a List[Option[Int]] instead of a List[Some[Int]], you need to either specify both the input and output types, or explicitly type the results:

List(1,2,3).fmap[Int, Option[Int]](Some(_))
List(1,2,3).fmap(Some(_)): List[Option[Int]]

This should probably be at least documented.

5 Likes

The type parameter of pureOne is the type of its input so that can’t possibly work, 1.pureOne[Int] does work.

pureTwo has two type parameters and you’re only passing one, so that can’t work either. If you write 1.pureTwo[Int, List] it works.

It would indeed be nicer to be able to write 1.pure[List] somehow, but we’ve determined that the best way to do that would be to support multiple type parameter lists on the same method, and while this is something we’d like to implement, it will have to wait until after 3.0 is out.

It’s also worth noting that you can write 1.pureOne: List[Int]

1 Like

I’d pretty much worked out why those don’t work, however I felt it was important to “show my work” as it were, and those are the closest analogs to the old syntax (and thus what people are most likely to try).

A minor issue with this is part of the benefit of this syntax is not having to specify the type of the anchor value (Int in this case).

The bigger issue with this is that if you’re going to use this in a chain, you might as well go for the non-extension method anyway, as you’ll have to add a bunch of nested (_: A) wrappers in a way that gets ugly really quickly:

type EitherErrorOr[A] = Either[NonEmptyList[ErrorADT], A]

((1.pureOne: List[Int])
  .pureOne: EitherErrorOr[List[Int]])
  .pureOne: Future[EitherErrorOr[List[Int]]

I checked, and it doesn’t look like you can omit any of the intermediate type ascriptions:

// Fails with:
// value pureOne is not a member of Int
println(1.pureOne.pureOne: List[Option[Int]])
1 Like

For context, this would effectively block migration of a large portion of our internal libraries, and make migrating to 3.0 pretty much a non-starter :disappointed:

I’m also extremely worried about the implications of the inability to collapse this:

1.pureThree.apply[List]

Into this:

1.pureThree[List]
1 Like

Scala 3 will still support implicits (including implicit classes) for a while, so this shouldn’t block anything. For the latter point I’m not sure off-hand if this can and should be supported so I can’t say anything.

It’s less a technical blocker and more an issue of selling the upgrade. The more times I have to put an asterisk on a new feature, the harder it is to convey that Scala 3.0 is production ready and worth the effort of upgrading. The ambiguous status of significant whitespace is already going to make things difficult enough as it is.

“We can upgrade, but this chunk of core functionality isn’t representable in idiomatic Scala 3.0, we’ll have to use the old syntax”, is a hard sell because there isn’t really an answer to the inevitable response of, “This seems half baked, lets wait a few versions.”

4 Likes

A solution to this problem is to have two separate groups of type args (receiver and application type args) for extension methods:


extension [A] (a: A)  def pureTwo[F[_]: Pure]: F[A] = summon[Pure[F]].pure(a)

(1/*A = Int*/).pureTwo[List /* F = List */]

But this is out of the scope of Scala 3.0 I think.

scala> extension [A](a: A) def pure: Option[A] = Some(a)
def extension_pure[A](a: A): Option[A]

scala> 4.pure[Int]
val res2: Option[Int] = Some(4)

Very strongly related to this issue, how does it make sense that the pure method suddenly has a type parameter here? I get that it is somehow a consequence of how extension methods are desugared. But from a usage point of view it seems like just an annoying leaky implementation detail. With the extra unfortunate consequence that this doesn’t work:

extension [A](a: A) def pure[F[_]: Pure]: F[A] = summon[Pure[F]].pure(a)

Which worked just fine with how extension methods were encoded in Scala 2.

Also consider this:

implicit class PureSyntax[A](a: A) extends AnyVal {
  def pure[F[_]: Pure]: F[A] = implicitly[Pure[F]].pure(a)
}

42.pure[Option]

This last line is eventually desugared into

PureSyntax.pure$extension[Option, Int](42)(optionPureInstance)

Then why can’t the Scala 3 equivalent desugar into this?

extension_pure[Option, Int](42)(given_Pure_Option)
4 Likes

Hi, I could get your desired syntax with a dependently-typed magnet as follows:

trait Pure[F[_]] {
  def pure[A](a: A): F[A]
  
  extension[A](a: A) def pureOne: F[A] = pure(a)
}
object Pure {
  given Pure[List] {
    def pure[A](a: A): List[A] = a :: Nil
  }
  given as Pure[Set] { def pure[A](a: A) = Set(a) }
  
  type TypeHolder <: { type A }
  object TypeHolder {
    import language.implicitConversions
    
    extension(t: TypeHolder) inline def a: t.A = t.asInstanceOf
    
    implicit final def magnetize[A1](a1: A1): TypeHolder { type A = A1 } = a1.asInstanceOf
    
    // still prints a warning on use-site for implicitConversions
    // given magnetize[A1] as Conversion[A1, TypeHolder { type A = A1 }] = _.asInstanceOf
  }
  extension[F[_]: Pure](t: TypeHolder) def pureFour: F[t.A] = t.a.pureOne
}

def trial() = {
  // Without explicit import of method, fails with:
  // value pureFour is not a member of Int
  import Pure.pureFour
  // Even though the conversion used are defined in companion object (aka good conversions)
  // dotty still requries implicitConversions on use-site
  import language.implicitConversions
  
  val i: Set[Int] = 1.pureFour[Set]
  val i2: Set[1] = 1.pureFour[Set]
  println(1.pureFour[List])
  println(1.pureFour[Set])
}

@main
def run(): Unit = {
  trial()
}

Interesting takeaways:

  • now abstract types have their own companion objects - the magnetize conversion is found when placed in the companion object of the type TypeHolder alias, not in its outer object
  • language.implicitConversions is required even for conversions in companions, even for new-style given-Conversions, usages of anything but implicit classes trigger warnings which means l:implicitConversions has to be enabled for all currently existing Scala 2 code and usages of nearly all existing Scala 2 libraries. As it is, it’s not serving a purpose in helping migration because it has to be blanket-enabled everywhere, much more so than language.higherKinds before.

Upd: implicitConversions are avoidable with an implicit value class as the magnet, but I’m not sure whether boxing is avoided for the value class (for that matter, I’m fairly sure that primitive types will be boxed in the abstract type solution above as well): https://scastie.scala-lang.org/hQXyYcyjRPyIdeHgSyHwMg

trait Pure[F[_]] {
  def pure[A](a: A): F[A]
  
  extension[A](a: A) def pureOne: F[A] = pure(a)
}
object Pure {
  given Pure[List] {
    def pure[A](a: A): List[A] = a :: Nil
  }
  given as Pure[Set] { def pure[A](a: A) = Set(a) }
  
  implicit final class TypeHolder[A1](val xa: A1) extends AnyVal { 
    type A = A1
  }
  
  extension[F[_]: Pure](t: TypeHolder[?]) def pureFour: F[t.A] = t.xa.pureOne
}

def trial() = {
  // Without explicit import of method, fails with:
  // value pureFour is not a member of Int
  import Pure.pureFour
  
  val i: Set[Int] = 1.pureFour[Set]
  val i2: Set[1] = 1.pureFour[Set]
  println(1.pureFour[List])
  println(1.pureFour[Set])
}

@main
def run(): Unit = {
  trial()
}
2 Likes