Pre-SIP: Add support for exposing types exiting a `transparent inline def`

Scala Improvement Process
1

Motivation

transparent inline defs are quite powerful, and quite useful. The issue with them is that it is very hard, or sometimes impossible for the callsite to reason about or narrow to the type returned by the transparent inline def.

Note: I saw another proposal talking about transparent params, but was not certain that it was getting at the same concept, so I created a separate thread.

Those familiar with implicits in scala are probably familiar with the Aux pattern. It lets you require an ev: F[A], and then use the dependent type like ev.B.
(Please forgive the somewhat silly example, as you could obviously just do (A, A), but I think it demonstrates the point).

  trait TupleMe[A] {
    type Tup
    def tup(a: A): Tup
  }
  object TupleMe {
    type Aux[A, _Tup] = TupleMe[A] { type Tup = _Tup }
    trait Typed[A, _Tup] extends TupleMe[A] {
       override final type Tup = _Tup
    }

    inline def apply[A](using ev: TupleMe[A]): ev.type = ev

    def tup[A](a: A)(using ev: TupleMe[A]): ev.Tup = ev.tup(a)
  }

And then, when specifying your instances, you specify the type as Aux, so the compiler knows what the output type should be:

    given inst: [A] => TupleMe.Aux[A, (A, A)] = ???

This works great, but based on some testing, it seems almost impossible to extend this concept to macros and transparent inline def. Your transparent inline def can specify something like

transparent inline def derived[A]: TupleMe[A] = ${ derivedImpl[A] }
transparent inline def derived[A]: TupleMe.Typed[A, ?] = ${ derivedImpl[A] }
transparent inline def derived[A]: Any = ${ derivedImpl[A] }

and then the caller can do something like:

val inst = TupleMe.derived[Class1]
val inst: TupleMe[Class1] = TupleMe.derived[Class1]
val inst: TupleMe.Typed[Class1, ?] = TupleMe.derived[A]

and it will all type-check into the inst, but it seems impossible to narrow the type on inst itself.
This is even worse when trying to use givens, because val inst = is not possible with givens, it needs a type, so you are forced to do something silly like:

val innerInst = TupleMe.derived[Class1]
given inst: innerInst.type = innerInst

We will see that by our 3rd attempt, we are able to get a simple example to type, but it feels like a fight, and quite unreliable. Unreliable in the sense that in the current state of transparent inline def, if we want to expose any of the types coming out of the definition, we are forced to leave the thing completely untyped. Given that by attempt 3, we are able to get our example to type check, this leads me to believe that such a feature should be possible, because the type system is able to figure out the types, we are just missing a syntax to define this concept.

Attempt 1

  final case class Class1(a: Int = 5, b: Option[String] = None, c: Boolean) derives Macros.TupleMe
  val c1: Class1 = Class1(0, None, false)

  oxygen.quoted.Util.showExpr(Macros.TupleMe[Class1])
  oxygen.quoted.Util.showExpr(Macros.TupleMe.tup(c1))

[info] 18 |  oxygen.quoted.Util.showExpr(Macros.TupleMe[Class1])
[info]    |  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[info]    |=====| TypeRepr.of[A] |=====
[info]    |                   type: oxygen.meta.Macros.TupleMe[oxygen.meta.MacroSpec.Class1]
[info]    |             type.widen: oxygen.meta.Macros.TupleMe[oxygen.meta.MacroSpec.Class1]
[info]    |           type.dealias: oxygen.meta.Macros.TupleMe[oxygen.meta.MacroSpec.Class1]
[info]    |type.dealiasKeepOpaques: oxygen.meta.Macros.TupleMe[oxygen.meta.MacroSpec.Class1]
[info]    |
[info]    |=====| expr.toTerm.tpe |=====
[info]    |                   type: oxygen.meta.MacroSpec.Class1.derived$TupleMe
[info]    |             type.widen: oxygen.meta.Macros.TupleMe[oxygen.meta.MacroSpec.Class1]
[info]    |           type.dealias: oxygen.meta.MacroSpec.Class1.derived$TupleMe
[info]    |type.dealiasKeepOpaques: oxygen.meta.MacroSpec.Class1.derived$TupleMe

[info] 19 |  oxygen.quoted.Util.showExpr(Macros.TupleMe.tup(c1))
[info]    |  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[info]    |=====| TypeRepr.of[A] |=====
[info]    |                   type: oxygen.meta.MacroSpec.Class1.derived$TupleMe.Tup
[info]    |             type.widen: oxygen.meta.MacroSpec.Class1.derived$TupleMe.Tup
[info]    |           type.dealias: oxygen.meta.MacroSpec.Class1.derived$TupleMe.Tup
[info]    |type.dealiasKeepOpaques: oxygen.meta.MacroSpec.Class1.derived$TupleMe.Tup
[info]    |
[info]    |=====| expr.toTerm.tpe |=====
[info]    |                   type: oxygen.meta.MacroSpec.Class1.derived$TupleMe.Tup
[info]    |             type.widen: oxygen.meta.MacroSpec.Class1.derived$TupleMe.Tup
[info]    |           type.dealias: oxygen.meta.MacroSpec.Class1.derived$TupleMe.Tup
[info]    |type.dealiasKeepOpaques: oxygen.meta.MacroSpec.Class1.derived$TupleMe.Tup

// doesnt compile
  val res: (Class1, Class1) = Macros.TupleMe.tup(c1)

Attempt 2

  final case class Class1(a: Int = 5, b: Option[String] = None, c: Boolean)
  val c1: Class1 = Class1(0, None, false)

  given inst: Macros.TupleMe.Typed[Class1, ?] = Macros.TupleMe.derived

  oxygen.quoted.Util.showExpr(Macros.TupleMe[Class1])
  oxygen.quoted.Util.showExpr(Macros.TupleMe.tup(c1))

[info] 18 |  oxygen.quoted.Util.showExpr(Macros.TupleMe[Class1])
[info]    |  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[info]    |=====| TypeRepr.of[A] |=====
[info]    |                   type: oxygen.meta.Macros.TupleMe[oxygen.meta.MacroSpec.Class1]
[info]    |             type.widen: oxygen.meta.Macros.TupleMe[oxygen.meta.MacroSpec.Class1]
[info]    |           type.dealias: oxygen.meta.Macros.TupleMe[oxygen.meta.MacroSpec.Class1]
[info]    |type.dealiasKeepOpaques: oxygen.meta.Macros.TupleMe[oxygen.meta.MacroSpec.Class1]
[info]    |
[info]    |=====| expr.toTerm.tpe |=====
[info]    |                   type: oxygen.meta.MacroSpec.inst
[info]    |             type.widen: oxygen.meta.Macros.TupleMe.Typed[oxygen.meta.MacroSpec.Class1, _ >: scala.Nothing <: scala.Any]
[info]    |           type.dealias: oxygen.meta.MacroSpec.inst
[info]    |type.dealiasKeepOpaques: oxygen.meta.MacroSpec.inst

[info] 19 |  oxygen.quoted.Util.showExpr(Macros.TupleMe.tup(c1))
[info]    |  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[info]    |=====| TypeRepr.of[A] |=====
[info]    |                   type: oxygen.meta.MacroSpec.inst._Tup
[info]    |             type.widen: oxygen.meta.MacroSpec.inst._Tup
[info]    |           type.dealias: oxygen.meta.MacroSpec.inst._Tup
[info]    |type.dealiasKeepOpaques: oxygen.meta.MacroSpec.inst._Tup
[info]    |
[info]    |=====| expr.toTerm.tpe |=====
[info]    |                   type: oxygen.meta.MacroSpec.inst.Tup
[info]    |             type.widen: oxygen.meta.MacroSpec.inst.Tup
[info]    |           type.dealias: oxygen.meta.MacroSpec.inst._Tup
[info]    |type.dealiasKeepOpaques: oxygen.meta.MacroSpec.inst._Tup

// doesnt compile
  val res: (Class1, Class1) = Macros.TupleMe.tup(c1)

Attempt 3

  final case class Class1(a: Int = 5, b: Option[String] = None, c: Boolean)
  val c1: Class1 = Class1(0, None, false)

  val innerInst = Macros.TupleMe.derived[Class1]
  given inst: innerInst.type = innerInst

  oxygen.quoted.Util.showExpr(Macros.TupleMe[Class1])
  oxygen.quoted.Util.showExpr(Macros.TupleMe.tup(c1))

[info] 19 |  oxygen.quoted.Util.showExpr(Macros.TupleMe[Class1])
[info]    |  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[info]    |=====| TypeRepr.of[A] |=====
[info]    |                   type: oxygen.meta.Macros.TupleMe[oxygen.meta.MacroSpec.Class1]
[info]    |             type.widen: oxygen.meta.Macros.TupleMe[oxygen.meta.MacroSpec.Class1]
[info]    |           type.dealias: oxygen.meta.Macros.TupleMe[oxygen.meta.MacroSpec.Class1]
[info]    |type.dealiasKeepOpaques: oxygen.meta.Macros.TupleMe[oxygen.meta.MacroSpec.Class1]
[info]    |
[info]    |=====| expr.toTerm.tpe |=====
[info]    |                   type: oxygen.meta.MacroSpec.inst
[info]    |             type.widen: oxygen.meta.Macros.TupleMe.Typed[oxygen.meta.MacroSpec.Class1, scala.Tuple2[oxygen.meta.MacroSpec.Class1, oxygen.meta.MacroSpec.Class1]]
[info]    |           type.dealias: oxygen.meta.MacroSpec.inst
[info]    |type.dealiasKeepOpaques: oxygen.meta.MacroSpec.inst

[info] 20 |  oxygen.quoted.Util.showExpr(Macros.TupleMe.tup(c1))
[info]    |  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[info]    |=====| TypeRepr.of[A] |=====
[info]    |                   type: scala.Tuple2[oxygen.meta.MacroSpec.Class1, oxygen.meta.MacroSpec.Class1]
[info]    |             type.widen: scala.Tuple2[oxygen.meta.MacroSpec.Class1, oxygen.meta.MacroSpec.Class1]
[info]    |           type.dealias: scala.Tuple2[oxygen.meta.MacroSpec.Class1, oxygen.meta.MacroSpec.Class1]
[info]    |type.dealiasKeepOpaques: scala.Tuple2[oxygen.meta.MacroSpec.Class1, oxygen.meta.MacroSpec.Class1]
[info]    |
[info]    |=====| expr.toTerm.tpe |=====
[info]    |                   type: oxygen.meta.MacroSpec.inst.Tup
[info]    |             type.widen: oxygen.meta.MacroSpec.inst.Tup
[info]    |           type.dealias: scala.Tuple2[oxygen.meta.MacroSpec.Class1, oxygen.meta.MacroSpec.Class1]
[info]    |type.dealiasKeepOpaques: scala.Tuple2[oxygen.meta.MacroSpec.Class1, oxygen.meta.MacroSpec.Class1]

  // this finally compiles
  val res: (Class1, Class1) = Macros.TupleMe.tup(c1)

So, in the end, we are able to get things to type, but its done in a very roundabout way that feels very inconsistent and unreliable, and is definitely more verbose than one might hope for.

Proposal

Add support for transparent inline def and caller of transparent inline def to either

  1. specify a “type hole” in the transparent inline def and callsite, that the compiler will fill in based on the type of the expr returned from the transparent inline def
  2. add a paradigm that allows us to reference types coming out of a transparent inline def, whether this comes from the transparent inline def “returning” its type to the caller, or the caller defining a reference for this type, and transparent inline def being the one to define what that type is.

Proposal 1 - type holes

Here, ????? is an arbitrary choice used to denote a type hole, but could really be anything. transparent, transparent hole, transparent type, type hole, typehole, ?????, etc. ????? is just the name for this concept that I will use here.

How this looks:

macro:

    private def derivedImpl[A: Type](using Quotes): Expr[TupleMe.Typed[A, ?????]] = {
      val aRepr = TypeRepr.of[A]
      type _Tup
      given Type[_Tup] = TypeRepr.tuplePreferTupleN(aRepr, aRepr).asTypeOf // (A, A)

      '{
        new TupleMe.Typed[A, _Tup] {
          override def tup(a: A): _Tup = ${ mkTup('a).asExprOf[_Tup] }
        }
      }
    }

    transparent inline def derived[A]: TupleMe.Typed[A, ?????] = ${ derivedImpl[A] }

callsite:

  given inst: Macros.TupleMe.Typed[Class1, ?????] = Macros.TupleMe.derived[Class1]

Here, the compiler would look at the type in the ????? position returned by the transparent inline def, and then populate that type in place of the ????? in the definition of given inst.

Proposal 2 - transparent inline def defines params and passes back to caller.

This one is a bit less intuitive, and not sure if it would be at all possible to pull off in the compiler, but it feels more well defined. Its entirely possible there is a better syntax for this, but I will arbitrarily define one here in an attempt to get the point across.

macro:

    private def derivedImpl[A: Type, _Tup: TransparentType](using Quotes): Expr[TupleMe.Typed[A, _Tup]] = {
      val aRepr = TypeRepr.of[A]
      given Type[_Tup] = TypeRepr.tuplePreferTupleN(aRepr, aRepr).asTypeOf // (A, A)

      '{
        new TupleMe.Typed[A, _Tup] {
          override def tup(a: A): _Tup = ${ mkTup('a).asExprOf[_Tup] }
        }
      }
    }

    transparent inline def derived[A][transparent Tup]: TupleMe.Typed[A, Tup] = ${ derivedImpl[A] }

callsite:

  given inst: Macros.TupleMe.Typed[Class1, B] = Macros.TupleMe.derived[Class1][transparent B]

In this case, derivedImpl accepts a _Tup: TransparentType, and must define a given Type[_Tup], which the transparent inline def would then pas back out to the caller.

Again, this feels admittedly a bit weird, so lets look at a 3rd option that might potentially be more intuitive here…

Proposal 3 - callsite defines a transparent type, and passes it to the transparent inline def

macro: (same as in proposal 2)

    private def derivedImpl[A: Type, _Tup: InferredType](using Quotes): Expr[TupleMe.Typed[A, _Tup]] = {
      val aRepr = TypeRepr.of[A]
      given Type[_Tup] = TypeRepr.tuplePreferTupleN(aRepr, aRepr).asTypeOf // (A, A)

      '{
        new TupleMe.Typed[A, _Tup] {
          override def tup(a: A): _Tup = ${ mkTup('a).asExprOf[_Tup] }
        }
      }
    }

    transparent inline def derived[A][transparent Tup]: TupleMe.Typed[A, Tup] = ${ derivedImpl[A] }

callsite:

  given inst: Macros.TupleMe.Typed[Class1, transparent B] = Macros.TupleMe.derived[Class1][B]

Here, the transparent B is defined in the type of inst, and then passes it into the macro as a transparent param. Then, B would be populated with the type of the given Type[_Tup] in the macro impl. A caller who did not care about such explicit typing could use such a function in the same way as in the original examples:

  given inst: Macros.TupleMe.Typed[Class1, ?] = Macros.TupleMe.derived[Class1]
  given inst: Macros.TupleMe[Class1] = Macros.TupleMe.derived[Class1]

Final Notes

Assuming a proposal like this was implemented, one semantic that I would consider necessary for the feature to behave well and be usable is as follows:

  final case class Class1(a: Int = 5, b: Option[String] = None, c: Boolean) derives Macros.TupleMe

derives clauses which call a transparent inline def with a ????? or transparent param, would need to expand to something like:

  final case class Class1(a: Int = 5, b: Option[String] = None, c: Boolean)
  object Class1 {
    given Macros.TupleMe[Class1, ?????] = Macros.TupleMe.derived[Class1]
    given Macros.TupleMe[Class1, B] = Macros.TupleMe.derived[Class1][transparent B]
    given Macros.TupleMe[Class1, transparent B] = Macros.TupleMe.derived[Class1][B]
}

In the absence of such a feature being implemented, it seems to me that a derives clause should behave like this anyway, seeing what the type of the derived instance is and narrowing the type of the given defined in the companion object, by either doing:

  val instInner = Macros.TupleMe.derived[Class1]
  given instInner.type = instInner

or, since its creating the definition anyway:

  given Macros.TupleMe.Typed[Class1, (Class1, Class1)] = Macros.TupleMe.derived[Class1]
1 Like