Expunging `new` from scala 3


#104

I withdraw my previous Pre-SIP proposal. It has a fatal flaw that was overlooked by my reasoning with analogy (which is always dangerous!). The flaw is in this part:

The generation of an apply method is suppressed if that method would clash with an apply method already given in that companion object. The meaning of “clash” is the criterion that currently determines whether an apply method for a case class should be generated.

This works fine for case classes, but breaks down for normal classes. Why? Consider an idealized Array class:

class Array[T](length: Int)
object Array {
  def apply[T](xs: T*): Array[T] = ...
}

Under my previous proposal, Array(10) would again produce an uninitialized array of length 10 instead of an array of length one with element 10. So this would change the semantics of existing
programs.

The same rule works OK for case classes because we have always generated an apply method for them.

I’ll post in the next comment a new pre-SIP proposal that implements function call syntax for instance creations in a different way. I leave aside for the moment the part dealing with how to replace new expressions


#105

Pre SIP: Creator Applications

Creator applications allow to use simple function call syntax to create instances
of a class, even if there is no apply method implemented. Example:

class StringBuilder(s: String) {
   def this() = this(s)
}

StringBuilder("abc")  // same as new StringBuilder("abc")
StringBuilder()       // same as new StringBuilder()

Creator applications generalize a functionality provided so far only for case classes, but the mechanism how this is achieved is slightly different. Instead of an auto-generated apply method, we add a new possible interpretation to a function call f(args). The previous rules are:

Given a function call f(args),

  • if f is a method applicable to args, typecheck f(args) unchanged,
  • otherwise, if f has an apply method applicable to args as a member, continue with f.apply(args),
  • otherwise, if f is of the form p.m and there is an implicit conversion c applicable to p so that c(p).m is applicable to args, continue with c(p).m(args)

The proposal is to add a fourth rule following these rules:

  • otherwise, if f can be interpreted as a type designator that designates a class C with a constructor that is applicable to args, continue with new f(args).

Motivation

Leaving out new hides an implementation detail and makes code more pleasant to read. Even though it requires a new rule, it will likely increase the perceived regularity of the language, since case classes already provide function call creation syntax (and are often defined for this reason alone).


#106

What does “converted” mean here ?


#107

otherwise, if f can be converted to a type name

I just fixed it.


#108

So if I understand correctly, we’d have the following semantics?

val A = (x: Int) => x + 1
class A(x: Int)

Some(A(123))     // has type Some[A]
Some(123).map(A) // has type Some[Int]

It’s not a showstopper, but can be surprising.

I’m all for this proposal as long as it does not include deprecating new, which is very useful for DSLs (even more so in Dotty, where one can have the type being new'd be inferred from the expected type!).

Also, couldn’t the scheme be unified with the way Java-static members work? i.e.,

  • all classes are assumed to have a fictive Java-static apply constructor method; and

  • if A.foo (where foo may be apply) does not refer to a member in the companion object, then we look for Java-static members of A named foo to select.


#109

I agree.


#110

This matches the current implementation but neither “matching” of methods nor overload resolution in the spec mention repeated parameters at all. This works for matching (meaning they don’t match) but as far as I can tell there is no rule which explains why apply(Int) takes precedence over apply(Int*).

But in the end both interpretations cause migration problems. If Array(10) calls the constructor, you’re silently breaking code, which is very bad. If Array(10) still calls the old apply method, any automatic migration tool must not change new Array(10) to Array(10) (and you need to keep new around as a fallback or provide a different rewrite that is not too ugly for common cases like new Array).


#111

I’ve always found this part of Array confusing, and prone to errors in any case. I would prefer all uninitialised arrays with specified dimensions to go through a well-named method on the companion. So for example, require the use of Array.ofDim for this.

Is this a source-rewrite tooling fixable issue? To mechanically rewrite new Array[Int](len) to Array.ofDim[Int](len)? Or does this happen for enough non-array cases that it’s a realworld problem?


#112

If Array(10) still calls the old apply method, any automatic migration tool must not change new Array(10) to Array(10) (and you need to keep new around as a fallback or provide a different rewrite that is not too ugly for common cases like new Array ).

In fact a rewriting tool could be smarter than that. It could try to replace new Array(10) by Array(10) and observe whether that resolves to the same symbol. If it does, it can go ahead.

About keeping new around: yes, probably. That part of the proposal has been dropped for now.


#113

From the educational standpoint, I want to throw in a little support for this. Having the ability to create new instances without using new more uniformly will definitely help simplify the language for the novice programmer. So part 1 is a big plus in my classroom.

On the other hand, the proposed syntax for anonymous classes scares me a bit, only because the inline form requires two statements in a block. I would need to think more about how often we use this. I think the main place it happens is when making calls on Java libraries or thin Scala wrappers. The primary one that occurs to me is ScalaFX when we do GUIs and graphics. The standard way I have students write that code now is using nested anonymous classes. It looks something like the following.

object SimpleGUIApp extends JFXApp {
  stage = new JFXApp.PrimaryStage {
    scene = new Scene(width, height) {
      val guiStuff = ...
      contents = guiStuff
    }
  }
}

I’m trying to imagine what this looks like with the modified style, and I don’t think it looks good. Perhaps something like this.

object SimpleGUIApp extends JFXApp {
  stage = { object st extends JFXApp.PrimaryStage {
    scene = { object sc extends Scene(width, height) {
      val guiStuff = ...
      contents = guiStuff
    }; sc }
  }; st }
}

There is also a non-nested style, and I might just have to switch to that. However, for things like event listeners I agree with @ramnivas that this is going to make for much uglier code.


#114

JFXApp uses delayedInit which is going to be dropped.
So I think more precise example can be:

object SimpleGUIApp extends JFXApp {
  object stage extends JFXApp.PrimaryStage { stage => 
     override def body():Unit = {
        ....
     }
  }
}

So object name extends is not big overhead when we compare it with other code.


#115

That wouldn’t work. It would need to be:

private[this] object _stage extends JFXApp.PrimaryStage { stage => ... }
stage = _stage

Which is quite horrendous, IMHO. Please, just leave new alone!

EDIT: Also note that with Dotty, one can currently write the amazingly simpler:

object SimpleGUIApp extends JFXApp {
  stage = new {
    ...
  }
}

This is because rhs in scene = rhs has expected type JFXApp.PrimaryStage.

It looks like the following is not supported yet, but in principle we could go as far as to also allowing:

    scene = new(width, height) {
      val guiStuff = ...
      contents = guiStuff
    }

#116

@MarkCLewis Looking at your

  stage = { object st extends JFXApp.PrimaryStage ...; sc }

I thought, well, since a singleton in this context doesn’t need a name, you just want the reference, we could make extends return a reference the same way new does when we drop object st. This rids us of a keyword but gains us nothing in semantics. This thread would just become “Expunging extends from scala 3.” It makes some sense, since extends and new both can run a constructor. (Though extends might not.) Maybe extends would be less controversial?

Here’s a more serious idea. What if we allow extends in place of =?

def x extends Array(10)
val x extends Array(10)
lazy val x extends Array(10)

This would let us be rid of new, and would let us have a clear separation between when we’re calling apply vs a constructor. So then

object SimpleGUIApp extends JFXApp {
  stage = new JFXApp.PrimaryStage {
    scene = new Scene(width, height) {
      val guiStuff = ...
      contents = guiStuff
    }
  }
}

becomes

object SimpleGUIApp extends JFXApp {
  stage extends JFXApp.PrimaryStage {
    scene extends Scene(width, height) {
      val guiStuff = ...
      contents = guiStuff
    }
  }
}

#117

It has something butifull.

I do not think it should be an aim.
But I like the idea to consider

  • val
  • object

as something similar.

note:
I am not sure that it will be work without Delayedinit or its alternative


#118

Never mind. My idea doesn’t quite get rid of new, because you would still need it for anonymous constructor calls. I can’t convince myself that extends is better than = new. Maybe it would be good in combination with something else.


#119

Hey @MarkCLewis - I really do feel your pain with the DSL thing. I have the exact same issue with callbacks.

object SimpleGUIApp extends JFXApp {
 stage = { object st extends JFXApp.PrimaryStage {
   scene = { object sc extends Scene(width, height) {
     val guiStuff = ...
     contents = guiStuff
   }; sc }
 }; st }
}

This is unacceptable to me. What I’d want is something vastly closer to:

object SimpleGUIApp extends JFXApp {
  stage = JFXApp.PrimaryStage {
    scene = Scene(width, height) {
      val guiStuff = ...
      contents = guiStuff
    }
  }
}

#120

I think this sounds very reasonable


#121

That counter-proposal “Pre SIP: Creator Applications” looks reasonable to me.


Dropping new for anonymous classes, in favor of anything that involves the extends keyword, and in particular { object O extends A {...}; O } would be a killer for Scala.js. Indeed, in Scala.js, if C is a JavaScript class/trait (that inherits from js.Any),

new C(...args) { ...defs }

is not equivalent to

{ object O extends C(...args) { ...defs }; O }

The latter defines a subclass of C and instantiates it. The former instantiates a C, then directly adds ...defs to the resulting object (because you can do that in JS), so basically it is equivalent to:

{
  val o = new C(...args)
  o.def1 = implem1
  ...
  o.defN = implemN
  o
}

That is semantically different from a subclass, and JavaScript libraries care about the difference. Dropping the ability to define enriched objects like that would have significant consequences for the usability and friendliness of Scala.js. We know because, when we added that ability in Scala.js, it was a huge relief for the definition of many JavaScript facade types and their usages.

See also


#122

Unfortunately, I think that the rush to expunge new misses an opportunity which I raised in my original proposal, here:

We frequently find ourselves writing definitions like,

implicit val x: SomeLongClassType[SomeLongParameterType] = new SomeLongClassType(a, b, c)

where we have to repeat SomeLongClassType twice (but we get the type parameter inferred, thankfully). This sort of repetition looks almost as bad as Haskell’s method definitions which repeat the method name at least twice…

My proposal was essentially to replace this definition with,

implicit val x: SomeLongClassType[SomeLongParameterType] = new(a, b, c)

and to use type inference to infer the type to be instantiated. (See the original proposal for a more complete explanation.)

I didn’t read every post in this long thread, but it seems that while it was pointed out that new is “special” because it introduces new heap objects, which was then rejected, nobody seemed to point out that new is invoking a constructor which is a different thing, defined in a different way, and in a different place, from method definitions. Scala chooses to give constructor definitions a more privileged status in its syntax than Java does, so if anything, it makes more sense to me for Scala than Java to have a special keyword to indicate when we’re invoking a constructor instead of a method.

There are no guarantees, of course, that calling a method without new won’t create new instances, but calling new does indicate unambiguously that a constructor is being invoked directly. The approach of creating synthetic apply methods with even slightly unclear rules creates an extra step of indirection: if I call a class I don’t know, say Foo(1, 2, 3), should I go and look directly at Foo's constructor, because 90% of the time it will be the thing that’s being called, or should I first check its companion object to see if it there’s an explicit apply method which is being called instead? What if it’s only defined 1% of the time, and I don’t bother to check (it’s so rare, I didn’t think of it!) and waste a lot of time trying to work out why calling Foo(1, 2, 3) doesn’t do what the constructor says it should? Could it introduce another “gotcha” where an apply method on a companion object of a commonly-used class does something subtly different from its constructor, and everyone wrongly assumes that the constructor is being called directly?

By the way, in case you still didn’t read it, my original proposal suggests changing the syntax of new to make it more predictable with respect to precedence of . vs whitespace.


#123