Improve opaque types

[Jasper-M]

@LPTK

Again, this is not needed , provided we can use the @specialized annotation (it already works in Scala 2, not sure about Scala 3, but it should be ported in any case).

AFAIU there’s no intention to port @specialized to Scala 3. Correct me if I’m wrong.

[LPTK]

Damn. Is there any public documentation/explanation about this decision?

[sideeffffect]

Btw, the companion objects of opaque types are part of given (aka implicit) scope, so something like this works:

trait SemiGroup[T]:
   extension (x: T) def combine (y: T): T

object A:
  opaque type EvenInt = Int
  object EvenInt:
    // EvenInt forms a SemiGroup with addition
    given SemiGroup[EvenInt] with
      extension (x: EvenInt) def combine (y: EvenInt): EvenInt = x + y

object B:
  def instance = summon[SemiGroup[A.EvenInt]] // works

Scastie - An interactive playground for Scala

That makes me really happy

[dcsobral]

@gvolpe

But more importantly, there hasn’t been enough exploration on the design of a library like this. Haskell’s newtype isn’t a good match because Haskell doesn’t have inheritance, it doesn’t have to support the Java type hierarchy, and it isn’t object oriented.

Newtypes are not about inheritance

You completely misunderstood my point, which is that they are clearly defined in Haskell because it does not have to contend with a type system with inheritance and an object oriented design.

For example, if you “newtype” String, what is the type of its toString() method? Can you answer that without any reference to the type hierarchy?

You are basically saying it can be easily done without actually going to the trouble of explaining how it should be done. Give the exact rules and I’ll give examples that break them.

This is basically opaque types with extra steps, which should be better handled by the language itself. It doesn’t have anything to do with inheritance, or Java type hierarchy.

Why is it better handled by the language? It’s funny that you mention below that “the above response should also answer this question” but you actually didn’t answer anything. You say it is better handled by the language without any arguments to support it. I gave arguments to support the position that it is not handled better by the language.

simply inline/forwarding methods won’t work either for various reasons I remarked on before. Some existing methods would break the opacity and need to be excluded, some need modified type signatures, and some take primitive/string parameters that should not be changed to the opaque types, or have return values be likewise.

Exactly. This is why I believe this feature would be much better if implemented in the language itself.

You believe it needs to be implemented in the language itself because the classes from external libraries are full of special cases? That makes no sense whatsoever.

[Jasper-M]

@LPTK

Damn. Is there any public documentation/explanation about this decision?

Don’t know of any (semi-)official reference, but I think I’ve seen it mentioned in passing in other discussions or issues. As a data point, there’s now a compiler phase specifically dedicated to the specialization of functions only (Revive function specialization by liufengyun · Pull Request #10452 · lampepfl/dotty · GitHub).

There’s no mention of this anywhere in the “dropped features” or “changed features” sections of the documentation. Would be nice to have someone from the dotty team chime in on this.

[gvolpe]

simply inline/forwarding methods won’t work either for various reasons

You mentioned this before, from which I understood there are a lot of edge cases to handle to guarantee unboxing. This kind of stuff is usually really hard to deal with at the library level and would be better done in the language itself.

Anyway, this discussion is going south and that was not the intention of the post. If there are no intentions to support newtypes in Scala 3 that’s okay. The idea was to communicate that there are actual unsolved problems and whether there’s something we could do.

[odersky]

Scala 3.0 only has function specialization. Full specialization was outside the scope of what we could do for this release. That’s not a statement that we want to get rid of specialization. It’s an acknowledgment that we lacked so far the resources to do it properly. Specialization is really hard, so it won’t be easy to port. If you want another example, just look at how long Valhalla is taking. We need the right people to do it and we need to have them funded so that they have the necessary time to devote on it.

Also, we might want to rethink the approach to specialization. The current one is not the only possibility. All of this will take time.

Specialization is an (important) implementation detail. So it might well land later in the 3 cycle.

[gvolpe]

I started something out from what I learned from this and the other post: https://github.com/gvolpe/newtypes

Any contributions more than welcome. If this proves to be useful, I can publish it to Maven Central under ProfunKtor.

[curoli]

Does this mean that in Scala 3.0, things like Option[Int] or Seq[Int] will always contain boxed Ints?

[sjrd]

@curoli

Does this mean that in Scala 3.0, things like Option[Int] or Seq[Int] will always contain boxed Int s?

Option[Int] and Seq[Int]s have always contained boxed Ints. The collections are not specialized in Scala 2 any more than in Scala 3. The only things that are specialized in the std lib are function types up to 2 parameters, and tuples up to 2 (or 3?) elements.

[nafg]

More generally, what are the ramifications of this regression?

[sjrd]

In code that only uses @specialized from (transitively) the std lib, probably none.

For libraries whose designs heavily rely on @specialized to provide decent performance … it’s probably bad.

For Scala.js code … significant code size improvement, and probably an improvement in performance as well.

[amsayk]

Whatever happened to dotty-linker ?

[sjrd]

Its author moved on before it was ready to be integrated and maintained by other people, and no one picked it up.

[LPTK]

In Scala 2, the following code will not box any of the integers, AFAIK (someone correct me if I’m wrong):

List((0, 1), (2, 3), (4, 5)).foreach(x => useAnInt(x._1 + x._2))

In the current implementation of Scala 3, the tuples will box the integers they hold, and the function passed to foreach will have to unbox them before adding them up.

[SethTisue]

@LPTK

In Scala 2, the following code will not box any of the integers, AFAIK (someone correct me if I’m wrong):

You’re not wrong. If you javap -c it you’ll see that Scala 2 uses Tuple2$mcII$sp (specialized) throughout, rather than an ordinary Tuple2, to avoid boxing the Ints. Whereas Scala 3 boxes them.

[BardurArantsson]

Is the art of writing one’s application as a set of libraries totally lost?

^ was a jokey comment to address a serious issue: Adding a burden for library writers is actually harming the ecosystem enormously… it requires a lot more time and resources for library writers/publishers to keep up. Cross compilation/special rules, etc. only adds to that.

[Swoorup]

Now that inline def works on opaque types. Do we need those specialised classes for primitives types anymore, @gvolpe? This should be enough I think?

trait NewType[Wrapped]:
  opaque type Type = Wrapped
  inline def apply(w: Wrapped): Type = w
  extension (t: Type) inline def value: Wrapped = t
  given (using CanEqual[Wrapped, Wrapped]): CanEqual[Type, Type] = CanEqual.derived

[benhutchison]

Two and a half years after this thread went quiet, and from my POV opaque type remains significantly less easy to use than Haskell’s newtype.

Haskell gives you a nicely-made box to put your data in, while Scala gives you a flatpack and a screwdriver and insists that’s just as good

Re-reading this thread I think @gvolpe provided an accurate description of where Scala’s limitations are, but his proposals seemed to fly into a wall of opposition.

One thing I’d add to the ask is the ability to auto-derive arbitrary type-class instances by delegation to the underlying type, as in GeneralizedNewtypeDeriving:

newtype Dollars = Dollars { getDollars :: Int } deriving (Eq,Show,Num)

How many lines of code does it take to express the above in Scala atm?

[bishabosha]

I think it should be possible to implement deriving on an opaque type, the issue is that because opaque types are so flexible, what would the deriving clause desugar to? One idea I had was that it still desugars to e.g. Show.derived in the companion, but then we provide the user factory methods to select which kind of mirror they should provide for an opaque type.