I think you might have misunderstood my reply. I was specifically answering the parent comment, and only answering the parent comment. In particular I am not arguing that tuples are a replacement for this SIP. I was arguing that this SIP wouldn’t be used to implement tuples.
Ah, I can totally see how the comment I actually replied to was only claiming that. I simply clicked “reply” on the latest post from you, when I really meant to reply to your earlier statement:
1 Like
It would still be nice to have efficient collection initializers.
One thing that annoys me is that there still doesn’t seem to be a nice way of efficiently constructing a singleton Iterator or Set in Scala. I have to write Iterator(x) or Set(x) and pay for allocating an array and its wrapper every time… or use Set.empty + x for the latter, which is more verbose.
2 Likes
Unfortunately, there’s a difference between “can be implemented” and “can be implemented by someone with extensive background in Shapeless or macros”. One of the really nice things about this proposal is it lowers considerably the difficulty of writing something like show"".
Can I interest you in Iterator.single(x)?
Right, I had missed this one, thanks. But there are plenty of collections that do not have a single constructor. The biggest offenders are Sets and Maps.
2 Likes
Wouldn’t a shapeless solution to the same problem also be much slower to compile?
3 Likes
Yes, and it would be considerably harder to debug.
Just a challenge to put out there: implement Cats’ show interpolator using just implicit evidences, union types, or HLists.
Because I’ve tried doing it, and the restrictions of the string interpolator API mean that you’re pretty much stuck with implicit conversions. In Dotty this means enabling implicit conversions on a project wide basis (or adding it to the import tax), and is very close to a dealbreaker. If there’s a reasonable way to do this, it implies a rather large gap in the current Dotty documentation, because I couldn’t find anything hinting this was possible, short of resorting to implicit conversions.
However, as show"This is $a, and this is $b!" (currently desugars to approximately this: new StringContext("This is", ", and this is", "!").show(WrappedArray(a, b))) would desugar to this:
new StringContext("This is", ", and this is", "!").show.applyBegin.applyNext(a).applyNext(b).applyEnd
The implementation would now be possible without enabling implicit conversions or resorting to a compiler plugin:
class ShowInterpolaterState(parts: List[String], builder: StringBuilder)
def applyBegin: ShowInterpolaterState = this
def applyNext[A: Show](a: A): ShowInterpolatorState =
parts match
case p :: rest => new StringInterpolatorState(rest, builder.append(p).append(a.show))
case Nil => builder.append(a.show); this
def applyEnd: String =
parts.foreach(builder.append)
builder.toString
def (sc: StringContext) json:
def show: ShowInterpolaterState =
new ShowInterpolatorState(sc.parts.toList, new StringBuilder)
I just watched part of 2019 November SIP Meeting at https://www.youtube.com/watch?v=jjEcYY2R9mU. @odersky mentioned that there will be a combination explosion problem when overloading applyNext methods. It’s true, just like any complex API that relies on method overloading. On the other hand, applyNext with type classes (like @morgen-peschke’s example) can easily solve the combination explosion problem but is hard to inline.
Fortunately, this proposal gives the library author the freedom to choose either method overloading or type class. Therefore, a sophisticated library author can take the advantage from both approaches and avoid the pitfall of each approach. For example, in html.scala, I use applyNext with type classes to avoid combination explosion problem, and also provide only a few additional overloading inlined applyNext methods to optimize the most frequent use cases.
The criticism was that the complexity of type-checking applications is already very high - we do not want to make the problem even more complicated and (most likely) slower to resolve by adding more choices that the compiler has to check.
Anyway, why not try the meta-programming approach for this?
Thinking about the complexity of type-checking is a good perspective. In fact, if we can implement as a built-in feature or an analytic plugin, it will be almost no more overhead on the type checker’s end than manually written builders, because it expand the vararg call before each parameter is type-checked. On the other hand, any other meta-programming based approaches will be slower, especially in nested vararg calls, because they all have to type-check the parameters twice.
Unfortunately analytic plugin is not supported in Scala 3. That’s why I thought making this feature built-in compiler might be a wise option.
2 Likes
I think analytic plugin could be an approach to reduce the complexity of the type checker, because with the help of analytic plugin, we can move many features out of the core type checker, instead they can be implemented as compiler build-in plugins.
Analytic plugin might be helpful for better modularizing the type checker, even when they are all running in one phrase.
It’s a conscious decision that there will be no analytic plugin, since that would lead to language fragmentation.
In the history of Scala 2, analytic plugin does not lead to language fragmentation, instead it leads experimental features be easier. Many of them from @milessabin have became part of the mainstream of the Scala language. On the other hand, some features like Scala Continuations were unsuccessful even it was an “official” feature developed by people in EPFL. But the analytic plugin mechanism still helped as it prevented the core compiler being affected when the feature is introduced and abandoned.
The current process of SIP requires an implementation before an SIP is reviewed. Since @odersky is both the maintainer of Dotty and a SIP Committee member, if there is no other way to create an implementation for an SIP, I am worried that the SIP meeting itself might become an implementation review for the Dotty pull request. After all, every proposal increases the complexity of Dotty unless it can be implemented as a plugin or a macro.
3 Likes
Do you mean, “why not try the meta-programming approach to implement curried varargs?” or, “why not try the meta-programming approach instead of curried varargs?”
I meant, to implement curried varargs. It seems like this should be possible. Essentially, you need to define an inline apply method that takes arguments to type Any* and maps to calls of applyBegin, applyNext and applyEnd. Since the formal arguments are all of type Any, that macro needs to verify that each actual argument is indeed of the right type demanded by the corresponding applyNext and cast it to this type. If the argument is not of the right type a compile error should be issued.
Is there really no way to implement straightforward desugaring as a macro? I couldn’t see one in the docs, but they start fairly arcane and dive into what look to the uninitiated like compiler internals distressingly quickly.
Straightforward desugaring is done by inline methods, and is really easy. The catch is that the call to the inline method has to be well typed; there is no analogue of Scala-2’s untyped macros.
Thanks for taking the time to explain this.
If I’ve understood correctly, we can desugar to the applyNext calls, but only after all type information has been erased to Any?
That doesn’t seem like it would be feasible to implement this with Dotty’s macro capabilities, without duplicating much of the work of the typer. So it might be feasible for someone with your background, but not mine.
My uninformed guess would be that this duplications would be painfully to program and slow to compile unless the macro framework exposes considerably more powerful type inspection capabilities than are described in the documentation (where at least in the examples given, you seem to need to know the types you’re expecting ahead of time).
As the contextual abstractions appear to be insufficient to implement something like the show"" interpolator, we’re in a situation where Dotty is strictly less usable than Scala 2.
To provide context about why this is such a big concern for me, we’re currently using show to help ensure that personally identifiable information doesn’t leak into logs & endpoint responses. Losing this capability presents a difficult barrier for championing adoption of Scala 3, if the situation remains as-is.