Improve opaque types

Note you can avoid the erasure issue by defining the extensions in the companion objects of the opaque types:

object data:
  opaque type Bar = String
  object Bar:
    def apply(v: String): Bar = v
    extension (x: Bar) def value: String = x
  end Bar

  opaque type Foo = String
  object Foo:
    def apply(v: String): Foo = v
    extension (x: Foo) def value: String = x
  end Foo
end data

An added benefit is that the extensions are then always in scope – no need to import data members.

6 Likes

Sweet, thanks!

2 Likes

You can abstract this stuff away pretty well:

scala> trait NewType[A]:
     |   opaque type Type = A
     |   def apply(a: A): Type = a
     |   extension (a: Type) def value: A = a
     | end NewType
// defined trait NewType

scala> object Bar extends NewType[String]
// defined object Bar

scala> val bar = Bar("foo")                                                
val bar: Bar.Type = foo

scala> bar.value
val res6: String = foo

Of course it’s not 100% the same API and has some drawbacks as discussed in the other thread (mainly with boxing of primitives).

2 Likes

In fact, extensions are in scope no matter whether they are defined alongside the opaque type or in a companion object.

4 Likes

Of course it’s not 100% the same API and has some drawbacks as discussed in the other thread (mainly with boxing of primitives).

Thanks, I saw this in the other thread.

The idea of newtypes / opaque types is that they should never box primitives (aka zero-cost wrappers). If we lose this guarantee, I think we would be better off using case classes instead.

Moreover, this still feels like a hack. I would expect a better solution in a completely new language such as Scala 3. Hopefully, we’re still on time to agree on what a “better API” could be :pray:

The boxing of primitives only happens if you use the generics. It can easily be avoided with, for example,

trait NewIntType:
  opaque type Type = Int
  def apply(a: Int): Type = a
  extension (a: Type) def value: Int = a
end NewIntType
1 Like

I just want to chime in and say opaque types are probably the feature I’m most looking forward to using (I really wish they could have been in 2.13). Also, I really like that the default does not make an API that builds the type from the internal representation.

If we look back at the SIP, that was quite intentional and I like that choice.

I hope opaque types stay as they are in scala 3.

9 Likes

The other thread also mentions that using @specialized should resolve the erasure problem. The thing is, I don’t know what’s the status of support for @specialized for class type parameters in Scala 3. Does anyone know if it’s supported, or will be in the future?

1 Like

I gathered by the different responses that the community is a bit split on this topic so I’ll try to reformulate what I meant with starting this post because I noticed the title might sound demanding, which was not the intention (tried to edited it but it seems I can not?).

Some people are happy with opaque types as they are. Some others like myself want something similar to newtypes. It would be great if you could make both sides happy.

I would love if Scala 3 would provide a native way to define proper newtypes, either in a future version or in the planned 3.0.0 release (obviously the sooner the better).

Why is it so important to me? Because I make use of the scala-newtype library absolutely everywhere, to the point that it has become a blocker when it comes to migrating over to Scala 3.

I’ve promoted the idea of newtypes + refinement types extensively. There’s also a section about them in my book Practical FP in Scala. From what I gathered, a lot of folks have started using both the scala-newtype and refined libraries as well, so I believe these are the users that would benefit from having newtypes support in Scala 3.

Can we have both?

This may sound silly but, is there any chance we could have both opaque types and newtypes (probably built on top of opaque types) in Scala 3? I would love to see this feature being part of the language but if that’s a no-op, I would be fine having it as a library, if that’s possible to build.

Has this been discussed before?

At the moment, cross-building my projects with Scala 3 is really painful due to the use of newtypes and some other plugins like context-applied.

Appreciate your consideration :pray:

Sincerely,
Gabriel.

4 Likes

The replies Improve opaque types and Improve opaque types address your concerns about boilerplate for declaring newtypes and avoiding boxing of primitives. Can you elaborate what is your remaining issue with the current design of opaque types?

The replies Improve opaque types and Improve opaque types address your concerns about boilerplate for declaring newtypes and avoiding boxing of primitives. Can you elaborate what is your remaining issue with the current design of opaque types?

These approaches partially address my concerns. I see them as workarounds that involve quite some boilerplate. I would love to see this feature being baked in into the language, if that’s possible.

To avoid boxing, we need to create multiple traits for every primitive type.

trait NewIntType:
  opaque type Type = Int
  def apply(a: Int): Type = a
  extension (a: Type) def value: Int = a
end NewIntType

trait NewStringType:
  opaque type Type = Int
  def apply(a: Int): Type = a
  extension (a: Type) def value: Int = a
end NewIntType

trait NewBooleanType:
  opaque type Type = Int
  def apply(a: Int): Type = a
  extension (a: Type) def value: Int = a
end NewIntType

// and so on...

object data:
  object ZeroCost extends NewIntType
  type ZeroCost = ZeroCost.Type

  object Bar extends NewBooleanType
  type Bar = Bar.Type

  object Foo extends NewStringType
  type Foo = Foo.Type
end data

Surely this could be made into a library and we can all move on. However, I think it’s a pity we have a complete new language without this feature so the idea of sparking this conversation once again was to see whether it was possible to ship proper newtypes as part of the language.

1 Like

While true, a lot of this could be hidden behind a simple interface in a library.

scala> class NewType[A]:
     |   opaque type Type = A
     |   def apply(a: A): Type = a
     |   extension (a: Type) def value: A = a
     | end NewType
// defined class NewType

scala> class NewIntType extends NewType[Int]:
     |   override opaque type Type = Int
     |   override def apply(a: Int): Type = a
     |   extension (a: Type) override def value: Int = a
     | end NewIntType
// defined class NewIntType

scala> import scala.compiletime.erasedValue

scala> transparent inline def newType[A] = inline erasedValue[A] match {
     |   case _: Int => new NewIntType
     |   case _ => new NewType[A]
     | }
def newType[A] => NewType[? >: Int & A <: Int | A]
scala> val Foo = newType[Int]; type Foo = Foo.Type
val Foo: NewIntType = NewIntType@346d8002
// defined alias type Foo = Foo.Type

scala> val Bar = newType[String]; type Bar = Bar.Type
val Bar: NewType[String] = NewType@593d5f39
// defined alias type Bar = Bar.Type

Notice you have one entrypoint to make a newtype “module”, newType[A], and when A is an AnyVal you statically get the specialized instance, otherwise the generic variant for types that don’t suffer from boxing.

Taking it even further would probably require something like macro annotations…

6 Likes

That’s not bad! Could definitely use something like this if it’s not built into the language :+1:

Why burden the language with something that can be implemented as a library? That goes against Scala’s design principles.

Yes, so what? The user doesn’t have to deal with it, and even the code for the traits can be done mechanically as generated source – except for method redirection which needs to be curated. More on this later.

Strings are not primitive. They are just used like one. :slight_smile:

I’ve already mentioned the design principle concern, so let’s talk about why it should not be provided as part of the standard library. First, I heard the standard library is intentionally being left alone for 3.0, so there’s that.

But more importantly, there hasn’t been enough exploration on the design of a library like this. Haskell’s newtype isn’t a good match because Haskell doesn’t have inheritance, it doesn’t have to support the Java type hierarchy, and it isn’t object oriented.

The example implementation I gave is rather useless, because it has no methods other than extracting the value again, but simply inline/forwarding methods won’t work either for various reasons I remarked on before. Some existing methods would break the opacity and need to be excluded, some need modified type signatures, and some take primitive/string parameters that should not be changed to the opaque types, or have return values be likewise.

This is something that needs to be explored at length, tried out, and revised based on real world usage. For that matter, there might be a call for different designs to satisfy different needs, in which case it might be best not to have it in the standard library at all. But even if that’s the right place for it, it would probably be better to wait on valhalla and, at any rate, now is the wrong time.

It would be great to start a discussion or a project implementing such a library for Scala 3. That would be more productive, in my opinion.

1 Like

Again, this is not needed, provided we can use the @specialized annotation (it already works in Scala 2, not sure about Scala 3, but it should be ported in any case).

1 Like

But more importantly, there hasn’t been enough exploration on the design of a library like this. Haskell’s newtype isn’t a good match because Haskell doesn’t have inheritance, it doesn’t have to support the Java type hierarchy, and it isn’t object oriented.

Newtypes are not about inheritance, they are plain zero-cost wrappers for extra type-safety at compile time. The runtime representation is the underlying datatype, which can be treated differently if it’s a primitive (i.e. no boxing). This is basically opaque types with extra steps, which should be better handled by the language itself. It doesn’t have anything to do with inheritance, or Java type hierarchy.

Why burden the language with something that can be implemented as a library? That goes against Scala’s design principles.

The above response should also answer this question.

Yes, so what? The user doesn’t have to deal with it, and even the code for the traits can be done mechanically as generated source – except for method redirection which needs to be curated. More on this later.

I’ll say it again, I’m not opposed to have this feature as a library. After all, I use newtypes library in Scala 2 but I was hoping Scala 3 could replace this library with the opaque types feature. I only realized when I started trying to migrate some projects that this is not the case.

simply inline/forwarding methods won’t work either for various reasons I remarked on before. Some existing methods would break the opacity and need to be excluded, some need modified type signatures, and some take primitive/string parameters that should not be changed to the opaque types, or have return values be likewise.

Exactly. This is why I believe this feature would be much better if implemented in the language itself.

It would be great to start a discussion or a project implementing such a library for Scala 3. That would be more productive, in my opinion.

That’s a different discussion. One I was hoping it would not be necessary but I guess I was wrong…

Again, this is not needed , provided we can use the @specialized annotation (it already works in Scala 2, not sure about Scala 3, but it should be ported in any case).

I would love to see this working in Scala 3.

1 Like

The boilerplate is limited to the library and does not leak to the user (when combined with new powerful inlining features). It’s clear that people have different preferences for the newtype encoding so the current design makes a lot of sense IMO since it’s flexible and caters to different needs.

2 Likes

AFAIU there’s no intention to port @specialized to Scala 3. Correct me if I’m wrong.

1 Like

Damn. Is there any public documentation/explanation about this decision?

6 Likes