Named Tuples - field updating

Language Design
1

Now Named tuples are slated for release, I thought I would bring up a dilemma from “real world” usage: how to update only a few fields? Use case: Query DSL based on named tuples

e.g.

val city = (name = "Lausanne", population = 140000, country = "CHE")

city.copy(population = ???) // how to do this?

the best there exists now is to basically invent a new DSL on top using Selectable,

e.g.

Fields(city).update(_.population := 150000, _.country = ???) // DSL

so I am opening this thread to fish for ideas, bike shed etc about a possible standard library/syntactical solution here

2 Likes
2

Not a solution for the standard library, but I expect Monocle to eventually support NamedTuples: NamedTuple support · Issue #1491 · optics-dev/Monocle · GitHub

1 Like
3

Normal tuples support .copy methods:

@ (1, 2, 3).copy(_1 = 4) 
res0: (Int, Int, Int) = (4, 2, 3)

Would make sense for named tuples to support .copy methods as well, perhaps with compiler magic to bridge the gap between the _1 concrete accessors and the named accessors.

8 Likes
4

I tried to write that but ran into trouble with inefficiency of repeated concat and/or *:. The basic idea is val zs = xs updateWith ys would have for zs(i) either xs(i) if the i’th name was not in ys, or ys(j) if the j’th name of ys is the same name as the i’th name of xs.

Anyway, conceptually I think the named-update idea is what we want. Actually writing it so it works well and efficiently probably requires macros, and maybe I’ll get around to it, but I don’t yet have many uses for named tuples, so I can’t be sure I will bother. (The variance doesn’t suit most of my use cases.)

Edit: forgot to say–I don’t think we need any specific compiler support for this, or if we do, it should be to make sure we can do these kinds of things more conveniently at library-level, not solve this one issue for named tuples but leave other similar applications hanging. If we have (name = "Lausanne", population = 140000, country = "CHE") and (population = 140500) both as named tuples, it’s pretty obvious what ought to happen if the second is an update for the first! The question is just how to make it sufficiently convenient to do that.

1 Like
5

yes i think there was already a prototype of patch which probably does the inverse check of ++, however checking that the fields are compatible is going to be harder than exclusivity

6

Hang on, why would the fields need to be compatible? Usually updated and everything like that allows you to change the field to whatever you want. And certainly with copy you can get anything.

scala> List(1, 2, 3).updated(1, "eel")
val res5: List[Int | String] = List(1, eel, 3)

scala> (4, 5, 6).copy(_2 = "salmon")
val res6: (Int, String, Int) = (4,salmon,6)

Maybe one would also like a type-preserving patch, which, I agree, seems more difficult. But the basic updated / copy should just splice in by name where it’s logically

for name <- names yield
  if that.names contains name then that.name
  else this.name
7 
scala> (4, 5, 6).copy(_2 = "salmon")
val res6: (Int, String, Int) = (4,salmon,6)

ok that’s great, should be very simple then as you say, just check names type is a subset (for copy equivalent)

8

got the types working:

implementation of method still needed - probably the implicit evidence should be the a mapping of index to index, rather than just a subset check

4 Likes
9

Yes, that looks good! My version found indices instead, e.g.

  type LabelIndex[Ns <: Tuple, L <: LabelVal, N <: Int] <: Int = Ns match
    case EmptyTuple => -1
    case L *: _     => N
    case _ *: tail  => LabelIndex[tail, L, N+1]

(called with 0 in the last position when starting). But I ran into trouble when trying to use the indices because rebuilding the tuple with *: is expensive, at which point it doesn’t matter if you can

inline constValue[LabelIndex[Ns, L, 0]] match =>
  case -1 => t *: copyRestImpl(...)
  case n => u(n) *: copyRestImpl(...)

(LabelVal is an alias to String & Singleton, which I use for manipulating tuple names in my code so that the compiler will reject, early, some of the more nonsensical things.)

Anyway, given type flexibility the match type turned out to be the easy part and the performant implementation the hard part, in my hands, and that is where I gave up.

10

In this case i would create a tuple type of indices, then use compiletime.constValueTuple to materialize to a value, which always calls the constructor directly.

I think it actually would be pretty good if there was an equivalent compiletime.constValueArray/compiletime.summonAllArray which could materialize a specialized array of the Tuple.Union type

11

I don’t quite understand what the const value is that’s being materialized? I can materialize [("a", 2, true)] but not [("a", 2, x.type)] even if x is a stable identifier. So I don’t see how I could materialize [(tup(0), upd(3), tup(2), upd(0), upd(2), tup(5), upd(1))], which is what the natural type computation would yield for something like (a=9, b=8, c=7, d=6, e=5, f=4, g=3).copyFrom((d="hi", g="there", e="bye", b="now"))

12

perhaps you could use ValueOf to materialize the singleton type of each tuple, and use an opaque type to wrap the selection.
e.g. opaque type Sel[T <: Tuple, U <: Tuple, I <: Int] = Any
given makeSel: [T <: Tuple & Singleton, U <: Tuple & Singleton, I <: Int] => (t: ValueOf[T], u: ValueOf[U], i: ValueOf[I]) => Sel[T, U, I] = Sel(if i.value < 0 then t.value.apply(i.value) else u.value.apply(i.value) (or indeed possibly branching implicits if you can statically prove I is negative) then summonAll and cast to the return type.

13

here is a complete solution:

import NamedTuple.{Names, AnyNamedTuple, NamedTuple}
import scala.compiletime.ops.int.{`*`, +, S}

type ContainsAll[X <: Tuple, Y <: Tuple] <: Boolean = X match
  case x *: xs => Tuple.Contains[Y, x] match
    case true  => ContainsAll[xs, Y]
    case false => false
  case EmptyTuple => true

type FilterName0[N, Ns1 <: Tuple, Vs1 <: Tuple] <: Option[Any] =
  (Ns1, Vs1) match
    case (N *: ns, v *: vs) => Some[v]
    case (_ *: ns, _ *: vs) => FilterName0[N, ns, vs]
    case (EmptyTuple, EmptyTuple) => None.type

type OptIndexOf[N, N2 <: Tuple, Acc <: Int] <: Option[Int] = N2 match
  case N *: _ => Some[Acc]
  case _ *: ns => OptIndexOf[N, ns, S[Acc]]
  case EmptyTuple => None.type

type Copy0[N <: Tuple, N1 <: Tuple, V1 <: Tuple, N2 <: Tuple, V2 <: Tuple, Acc <: Tuple] <: AnyNamedTuple = (N1, V1) match
  case (n1 *: ns1, v1 *: vs1) => FilterName0[n1, N2, V2] match
    case Some[v2] => Copy0[N, ns1, vs1, N2, V2, v2 *: Acc]
    case _ => Copy0[N, ns1, vs1, N2, V2, v1 *: Acc]
  case (EmptyTuple, EmptyTuple) => NamedTuple[N, Tuple.Reverse[Acc]]

type Indices0[Idx <: Int, N1 <: Tuple, N2 <: Tuple, Acc <: Tuple] <: Tuple = N1 match
  case n1 *: ns1 => OptIndexOf[n1, N2, 0] match
    case Some[i] => Indices0[S[Idx], ns1, N2, i *: Acc]
    case _ => Indices0[S[Idx], ns1, N2, (-1 * (Idx + 1)) *: Acc]
  case EmptyTuple => Tuple.Reverse[Acc]

type Copy[T <: AnyNamedTuple, U <: AnyNamedTuple] <: AnyNamedTuple = (T, U) match
  case (NamedTuple[ns1, vs1], NamedTuple[ns2, vs2]) => Copy0[ns1, ns1, vs1, ns2, vs2, EmptyTuple]

type Indices[T <: AnyNamedTuple, U <: AnyNamedTuple] <: Tuple = (T, U) match
  case (NamedTuple[ns1, _], NamedTuple[ns2, _]) => Indices0[0, ns1, ns2, EmptyTuple]

@scala.annotation.implicitNotFound("Can not copy fields from named tuple of type ${U}, it has fields not present in type ${T}.")
final class IndicesOf[T <: AnyNamedTuple, U <: AnyNamedTuple](is: Indices[T, U]):
  val values: Iterator[Int] = is.productIterator.asInstanceOf[Iterator[Int]]

object IndicesOf:
  inline given [N <: Tuple, V <: Tuple, N1 <: Tuple, V1 <: Tuple]
    => (ContainsAll[N1, N] =:= true)
    => IndicesOf[NamedTuple[N,V], NamedTuple[N1,V1]] =
      IndicesOf[NamedTuple[N,V], NamedTuple[N1,V1]](compiletime.constValueTuple[Indices[NamedTuple[N,V], NamedTuple[N1,V1]]])

extension [T <: AnyNamedTuple](t: T)
  def copy[U <: AnyNamedTuple](u: U)(using is: IndicesOf[T, U]): Copy[T, U] = {
    val t0 = t.asInstanceOf[Tuple]
    val u0 = u.asInstanceOf[Tuple]
    val arr = IArray.from(
      is.values.map(i => if i < 0 then t0.productElement(Math.abs(i) - 1) else u0.productElement(i))
    )
    Tuple.fromIArray(arr).asInstanceOf[Copy[T, U]]
  }

if this was added to the std lib, there could also be an optimisation pass for small tuple sizes to avoid the intermediate value.

13 Likes
14

Nice! Would you have time to do a pull request for this?

3 Likes
15

Working on it - if we do the library-based solution here that allows subfields/out of order fields, then there can be no target typing

3 Likes
16

see here for a PR Add experimental NamedTuple copyFrom method by bishabosha · Pull Request #23135 · scala/scala3 · GitHub

4 Likes