Proposal to introduce this#type and constructor constrains to improve definition of java.lang.Object.clone-like methods


#1

Not sure how many people may find it useful, but idea is to make not only builder-like methods signatures convenient and type-safe (as they are after introduction of this.type), but also prototype-like factory methods signatures (such as java.lang.Object.clone method)

So proposal is to define this#type and constructor constrains like T <: {this(payload: String)} and similar Self-type constraints using constraint-defining structural type. (Indirect invocation of bound in constraint constructor may look like val foo: T = ... ; new (foo#type)(payload = "bar"))

This can be demonstrated by the following code snippets:

  // type safe java.lang.Object.clone method signature
  trait AnyRefEx {
    abstract override def clone(): this#type = super.clone().asInstanceOf[this#type]
  }

  // describe type that impose some constraint on non abstract class to have appropriate 1-arg constructor
  type HasStringConstructor = {
    def this(payload: String)
  }

Usage of this definitions may look like this

  class Foo(val payload: String) {
    // this: {def this(payload: String)} =>
    this: HasStringConstructor =>

    // factory methods with type-safe call to appropriate constructor of `this.getClass`
    // which is required to be there by `this: HasStringConstructor =>` constraint
    def `new`(val payload: String): this#type = new (this#type) (payload)
  }

  class Bar(payload: String) extends Foo(payload) with AnyRefEx with Cloneable {
    // this: {def this(payload: String)} =>
    this: HasStringConstructor =>

    override def toString = s"Bar(payload=$payload,hash=${Objects.hash(this)})"
  }

  def testFooBar(): Unit = {
    val foo = new Bar("text")
    val bar = foo.clone() // bar.type <: Bar
    val baz = new (foo#type)("baz-text") // baz.type <: Bar
    val baz2 = foo.`new`("baz2-text") // baz2.type <: Bar
    // should print: foo = Bar(payload=text,hash=1136497449) , bar = Bar(payload=text,hash=863125071) , baz = Bar(payload=baz-text,hash=1693847691)
    println(s"foo = $foo , bar = $bar , baz = $baz")
    // should print: bar.payload = text , baz.payload = baz-text , baz2.payload = baz-text
    println(s"bar.payload = ${bar.payload} , baz.payload = ${baz.payload} , baz2.payload = ${baz2.payload}")
  }

In other words, it would be nice to have this#type (or more widely foo#type) very similar to this.type (foo.type) but which rather “refers” to runtime type of this (or any val foo) instance.
It may look like this.type is also fit well at least for typing clone-like factory methods, bat as for my understanding correct meaning of foo.type can be expressed like bar.isInstanceOf[foo.type] <==> (bar == foo),
proposed foo#type in general should be more close to foo.getClass so that classOf[foo#type] == foo.getClass and bar.isInstanceOf[foo#type] <==> foo.getClass.isInstance(bar)

So I would like to use correct signature of java.lang.Object.clone, which should/could be def clone(): this#type. And I would like to be able to write some prototype-like factory methods with signatures like def makeNew(...): this#type.

Currently it is actually possible to use this.type in signatures of that factory methods, and then just brutally cast to this.type (in method body) which will look OK in place of usages of that factory methods, but it will be actually “not much” type-safe in place of their implementations, and as I underhand it definitely violates correct meaning of this.type.

So here is that kind of working example which mimics previous code snippets using this.type + reflection + unsafe casts

  trait AnyRefEx {
    abstract override def clone(): this.type = super.clone().asInstanceOf[this.type]
  }

  class Foo(val payload: String) {
    // construct new instance of the same type as this
    def `new`(payload: String) : this.type = {
      this.getClass.getConstructor(classOf[String]).newInstance(payload).asInstanceOf[this.type]
    }
  }

  class Bar(payload: String) extends Foo(payload) with AnyRefEx with Cloneable {

    override def toString = s"Bar(payload=$payload,hash=${Objects.hash(this)})"
  }

  def testFooBar(): Unit = {
    val foo: Bar = new Bar("text")
    val bar: Bar = foo.clone()
    val baz: Bar = foo.`new`("baz-text")
    // should print: foo = Bar(payload=text,hash=1136497449) , bar = Bar(payload=text,hash=863125071) , baz = Bar(payload=baz-text,hash=1693847691)
    println(s"foo = $foo , bar = $bar , baz = $baz")
    // should print: bar.payload = text , baz.payload = baz-text
    println(s"bar.payload = ${bar.payload} , baz.payload = ${baz.payload}")
  }

Also I should mention that in contrast to C# new constraint I would like to invoke that (defined in constraint) constructor of unknown at compile-time type T not from T itself, but form some instance of T, so that if in C# it should be invoked as new T(), here proposed invocation should look like val foo: T = ...; new (foo#type)
On bytecode level I would assume that instead of going to reflection and finding some appropriate constructor there at runtime, it rather should use some hidden (non static) method with name like “$new” and same argument list as in appropriate constructor from constraint (in all non abstract subclasses this method just need to be overridden with correct implementation for that sub-class).
Then new (foo#type)(arg1, ... , argN) should be translated to foo.$new(arg1, ... , argN)

From the other perspective, similar correct typing (but without “built-in” factory methods) can be achieved by constructs like

    trait HasThis {
      type This <: HasThis
    }

    trait HasThis1[This <: HasThis2[This]]

And then used like

    class Foo extends HasThis {
      type This <: Foo
    }

    class Bar extends Foo {
      type This <: Bar
    }

    class BarImpl extends Bar {
      type This = BarImpl
    }

Actually from this#type I would expect similar behavior to that behavior of This abstract type (from snippets above). But with that exception, that this#type should be “refined automatically” (to eliminate that need to refine that abstract type This in each subclass manually)

So that it would be possible to rewrite about snippets like

    trait HasThis {
      type This = this#type
    }

And then

    class Foo extends HasThis {
      // it should be true that `This <: Foo`

      // following method should be type-safe, however without imposing constructor constraint, it still can fail at runtime
      def `new`(): this#type = this.getClass.newInstance()
    }

    class Bar extends Foo {
      // it should be true that `This <: Bar`
    }

    class BarImpl extends Bar {
      // it should be true that `This <: BarImpl`
    }

In above snippet it was also assumed that actual return type of this.getClass should become Class[this#type], so that inferred type of expression this.getClass.newInstance() should be this#type


#2

# is a method if it is present after a term. Why not use this.class instead of this#type?


#3

I would rather consider #type as compound operator (such as += or ::), And I would choose # instead of . because it looks similar for me to that case of nested classes, so I would consider that difference between this.type and this#type should be somewhere near to that difference between outer.Inner and Outer#Inner.
However for me it looks not so important what would be the name of that operator (I would easily accept any other reasonable name for it).


#4

I don’t understand what the use case is meant to be (not that I’ve ever really understood it for this.type either). I find all the example uses hopelessly confusing, because it’s never clear what type is meant. I expect the intended meaning is similar to what you get with F-bounded quantification? IMO the verbosity of T extends Foo[T]-style constraints is justified because they make it very clear exactly what kind of constraints the subclass has to follow. E.g. if I have a Rectangle type with a copy() method that returns Rectangle, and I subclass it as Square, then if Rectangle implements CopyableWithTwoIntParameters[Rectangle] then it’s very clear that Square still implements CopyableWithTwoIntParameters[Rectangle] and I can call Square#copy(Int, Int) and get a Rectangle back. That all makes sense to me.

Could you go into more detail about what you’re trying to achieve, what a Copyable[T]-style solution for the same use case would look like, and why/how this offers an improvement? To me this looks like a complex feature for the sake of an obscure corner case that is going to come up pretty rarely and confuse readers when it does, but I’m hoping there’s something I’ve missed that makes this more valuable.


#5

Ah, good old MyType. A cursory search didn’t reveal any new findings on this topic but a lot of people here are more plugged in to PLT research than me. :slight_smile:


#6

@lmm Supporting your geometrical approach, representative example may look like

  trait Mutable2dFigure {
    def shift(x: Double, y: Double): this.type
  }

  trait Immutable2dFigure {
    // using suggested `this#type` notation
    def shifted(x: Double, y: Double): this#type
  }

So justification for those should be quite simple: when you are moving some Bar you will probably expect that it will not become Circle or Triangle. Since this constraint is geometrically quite intuitive and obvious, you may probably have some natural wish to express this constraint in code. (Also supposing that simplicity of written form for of this construct should match to obviousness of this geometric concept). While you can express this constraint easily for mutable structures by the means of this.type, currently you can not express that constraint with the same simplicity level for the same case but with immutable structures.

If I understood your question (or irony :slight_smile:) correctly, answer may look like this:

  def moveScale(figure: Mutable2dFigure, x: Double, y: Double, ratio: Double): Mutable2dFigure = figure.scale(ratio).moveBy(x, y)

  trait Movable {
    def moveBy(x: Double, y: Double): this.type
  }

  trait Scalable {
    def scale(ratio: Double): this.type
  }

  trait Mutable2dFigure extends Movable with Scalable

Without this.type you will end up with boilerplating (cause it will be always need to duplicate all “builder-line” methods definitions and refine return values in all sub-trites / sub-classes), or otherwise figure.scale(ratio).moveBy(x, y) will be invalid …

Actually more widely “target audience” for such construct is methods similar to that SeqLike.updated / MapLike.updated / <Case Class>.copy-like / java.lang.Object.clone-like / etc

Hoverer I must admit that such approach can not fit/cover 100% existing cases (which may look relevant for it applications).
For example consider following definition

  trait GoodMap[K, V]{
    // ...
    // using suggested `this#type` notation
    def updated(key : K, value : V) : this#type
    // ...
  }

  trait GoodMap2[K, +V]{
    // ...
    // using suggested `this#type` notation
    def updated[V1 >: V](key : K, value : V1) : this#type with GoodMap2[K, V1]
    // ...
  }

First one (GoodMap) has problem with variance, while second one (GoodMap2) some hove “fixes” that variance-problem, but become more verbose and looks not so clear as it should be. Other problem lays in
Map.Map1 …Map.Map4
For those types constraint that map.updated(...).getClass == map.getClass is false at all, same as
map.getClass.isAssignableFrom( map.updated(...) .getClass). Since for example Map.Map3 after updating may become either
Map.Map2 or Map.Map3 or Map.Map4 and requirement map.getClass.isAssignableFrom( updated_map.getClass ) can not be preserved for all that cases (at least for Map.Map2 which can not be subclass of Map.Map3)

I found mentioned there discussion rather interning. Actually proposed there class.type construct perfectly matches to Roost’s self-types (also mentioned in this post)
So according to that post, at least java.lang.Object.clone-like methods in Rust could be declared correctly (with return type Self)

trait Cloneable {
    fn clone(&self) -> Self;
}

Hoverer, in flavor of nested/inner classes (types), I would prefer to have possibility to point to “SelfType” not only for this, but for any other constant value as well. If it is possible to write (someInstance).Nested , (someInstance).type, (someInstance).type#Nested, why should’t we allow some (someInstance)#type (or more “Rust’ly” saying - something like (someInstance).Self-type). In this case if we have val a = ... ; val b : a#type = ...; then it should be obvious that b#type <: a#type same as, this dose’nt requires that a.getClass == b.getClass, it only requires that a.getClass.isAssignableFrom(b.getClass) (assuming that a != null && b != null).

Also in that previous discussion I can see some final “solution” for HasThisType-like traits / classes. It is something like:

  trait HasThisType[ThisType <: HasThisType[ThisType]] {
    this: ThisType =>

  }

Hoverer it looks too verbose and less convenient comparing for an instance with that Roost’s self-types

Regarding semi-abstract methods (refereed to as “must be redefined” methods) suggested in previous discussions I would say that in fact some correct generalized approach is that “it should be some primary source of that this#type instances”. But that “primary source” could be defined / implemented differently.

So in fact it is not a problem to use that this#type in any method definition just in the same fashion, as one can use this.type (in place of both input parameters type and/or return value type and/or local variables etc), the only problem, is that how we can get some “fresh instances” of type this#type ? (the only well defined available instance of that type is this itself).
For this reason it was proposed to “enforce” some constructor signatures, to require that all sub classes of such class (which need to have such this#type factory) should also mandatory have that/same constructor signature in their definitions. Then based on that constraint some automatically generated (hidden) virtual method should be generated, and actually that method should serve as that “primary source” of that “fresh instances” of this#type. This approach was also referred to as “constructors constrains” in initial post.

An other possible technique, more close to Rust’s approach - is to allow this#type abstract factory method be implemented using some particular class (like new Foo(...)), but ONLY in final classes (When Impl-class is final, then this#type become referencing on some strict, very well known class/type).

Well, Maybe … Or maybe not :slight_smile:. (At lest probably Rust language designer may argue to your opinion :slight_smile:)

From my personal point of view, I would introduce something like this#type (aka “MyType”) at lest for fixing signatures of java.lang.Object.clone and java.lang.Object.getClass methods. Correct version should defiantly look like

  trait AnyRefEx2 {
    // using suggested `this#type` notation
    def clone(): this#type
    // using suggested `this#type` notation
    def getClass: Class[this#type]
  }

Same as something.getClass should be typed as Class[something#type] etc


#7

So this is a feature that exists primarily for mutation? I can understand why Rust would see that as worthwhile, but I don’t think it’s worth it for Scala. The HasThisType solution is indeed verbose, but it covers the use case and is made of existing Scala features that a reader can already understand.


#8

The Problem with HasThisType, or any f-bounded polymorphism, is that you cannot write the general type down outside of it’s definition, at least as far as I’m aware. Meaning, you cannot be more specific than:

val htt: HasThisType[_] = ???

Which makes you lose the value that HasThisType was to provide in the first place.

Also, HasThisType works only one level deep. If a class or trait extends HasThisType and fully defines the type, it’s over. HasThisType has lost its effect once you’re at that second level.


#9

I think that problems could be demonstrated by following examples.

(1) If you have some simple types hierarchy

    class Foo {
      // some code to be refactored with ThisType approach
    }

    class Bar extends Foo {
      // some code to be refactored with ThisType approach
    }

And you would like to refactor it to start using something like ThisType, your type hierarchy will probably end up with something like this

    trait HasThisType[ThisType <: HasThisType[ThisType]] {
      this: ThisType =>

    }

    trait FooLike[ThisType <: FooLike[ThisType]] extends HasThisType[ThisType] {
      this: ThisType =>

      // some code after refactoring Foo with ThisType approach
    }

    class Foo extends FooLike[Foo]

    trait BarLike[ThisType <: BarLike[ThisType]] extends FooLike[ThisType] {
      this: ThisType =>

      // some code after refactoring Foo with ThisType approach
    }

    class Bar extends BarLike[Bar]

So as it was mentioned, Foo itself will be not much useful for subclassing, and for inheritance you probably will be using FooLike[ThisType] instead

(2) As it was mentioned HasThisType[_] is quite useless in first place, hoverer, to be honest, it is possible to “workaround” that by extracting your code snippet into generic method, or by using forSome (existential) type
Conceptually It may look like

    trait HasThisType[ThisType <: HasThisType[ThisType]] {
      this: ThisType =>
    }

    // more "useful" replacement of `HasThisType[_]` may look like following
    type HasThisType_ = ThisType forSome {type ThisType <: HasThisType[ThisType]}

    // or you can extract your code snippet in some generic method
    def doSomething[ThisType <: HasThisType[ThisType]](that: ThisType) = ???

Hoverer first option should be not available in Scala3 (actually I don’t know whether there will be some adequate replacement for that kind of construct after forSome-decommission)

More complete example of that concept may look like this

    trait HasThisType[ThisType <: HasThisType[ThisType]] {
      this: ThisType =>
    }

    trait FooLike[ThisType <: FooLike[ThisType]] extends HasThisType[ThisType] {
      this: ThisType =>

      val times: Int
      def `new`(times: Int): ThisType

      // some `updated`-like operation
      def foo(): ThisType = `new`(times + 1)
    }

    trait BarLike[ThisType <: BarLike[ThisType]] extends FooLike[ThisType] {
      this: ThisType =>

      // some `updated`-like operation
      def bar(): ThisType = `new`(times - 1)
    }

    class Foo(val times: Int) extends FooLike[Foo] {
      override def `new`(times: Int): Foo = new Foo(times)
    }

    class Bar(val times: Int) extends BarLike[Bar] {
      override def `new`(times: Int): Bar = new Bar(times)
    }

    // ----  ----  ----  ----

    def threeTimesFoo[FooLikeType <: FooLike[FooLikeType]](that: FooLikeType): FooLikeType = that.foo().foo().foo()

    type FooLikeType = FooLikeType forSome {type FooLikeType <: FooLike[FooLikeType]}

    def foo : FooLikeType = new Foo(0)
    // you can chain/invoke FooLikeType.foo() methods as many times as you wish, while FooLike[_].foo() can not be chained
    def foo3Times : FooLikeType = foo.foo().foo().foo()
    def foo3Times_ : FooLikeType = threeTimesFoo(foo)
    assert(foo3Times.times == foo.times + 3)
    assert(foo3Times.times == foo3Times_.times)

By the way, more “kind” & equivalent notation (using nested types instead of generic parameters) unfortunately doesn’t work …

  trait HasThisType2 {
    this: HasThisType2#ThisType =>
    type ThisType <: HasThisType2
  }
  // compilation fails with error:
  // Error:(80, 24) illegal cyclic reference involving value this
  //       type ThisType <: HasThisType2

Maybe things can go better, if we would be able to use inner types as some “named type parameters”, then
HasThisType2 could be rewritten as

  trait HasThisType2 {
    this: ThisType =>
    // supposing that some marked inner types could be accesses/passed as `Container[InnerType = ...]`
    // something like `ThisType <: HasThisType[ThisType]`
    @isGenericTypeParameter type ThisType <: HasThisType2[ThisType = ThisType]
  }

And probably in this form it should become more correct.

On the other hand, using suggested notation of this#type , this: {this(...)} => , new (this#type) (...) previous example could be rewritten as

    trait HasThisType {
      // uses suggested notation `this#type`
      type ThisType = this#type
    }

    trait FooLike extends HasThisType {
      // uses suggested notation to impose constrictor constraint - none abstract implementor should have
      // `this(times: Int)` constructor
      this: {def this(times: Int)} =>

      val times: Int
      // usage of that (bound in constraint) constructor - using suggested notation of `new (this#type) (...)`
      def `new`(times: Int): ThisType = new (this#type) (times)

      // some `updated`-like operation
      def foo(): ThisType = `new`(times + 1)
    }

    trait BarLike extends FooLike {
      // uses suggested notation to impose constrictor constraint - none abstract implementor should have
      // `this(times: Int)` constructor
      this: {def this(times: Int)} =>

      // some `updated`-like operation
      def bar(): ThisType = `new`(times - 1)
    }

    // Foo conforms `this: {def this(times: Int)} =>` constraint since it has appropriate constructor
    class Foo(val times: Int) extends FooLike

    // Bar conforms `this: {def this(times: Int)} =>` constraint since it has appropriate constructor
    class Bar(times: Int) extends Foo(times) with BarLike

    // ----  ----  ----  ----

    def threeTimesFoo(that: FooLike): FooLike = that.foo().foo().foo()

    def foo : FooLike = new Foo(0)
    // you can chain/invoke FooLike.foo() methods as may times as you wish
    def foo3Times : FooLike = foo.foo().foo().foo()
    def foo3Times_ : FooLike = threeTimesFoo(foo)
    assert(foo3Times.times == foo.times + 3)
    assert(foo3Times.times == foo3Times_.times)

Summarizing examples mentioned above I can say that, yes, it is good that something is already can be done/implemented even by using only existing features of Scala language, but the way how it looks like now is not that good enough (not that perfect) to be not improved by introductions of something like this#types (aka MyType or Roost’s self-types)