Expunging `new` from scala 3

Language Design
1

Would it be possible to find a way to expunge the keyword new from scala 3?

There are two reasons why I’d like to do this. Firstly, there are a bunch of situations where you use or don’t use new in fairly ad-hoc ways when introducing implementations that are allocated there and then. For example, but in no way limited to:

object foo extends Foo { ... }
lazy val bar = new Bar { ... }

or during callbacks:

foo.callWithCallback(new { override def onSuccess ... ; override def onFailure ... }

Once the magic of inlining and stack allocation and all the rest of it has taken place, the new is often a lie. No object was actually allocated on the heap. Indeed, potentially there was never even an implementing class to instantiate.

The other place is when creating a new instance of a non-case class.

val c = new java.awt.Color(128,0,255)

This leads to various syntactic irregularities if you go on to call methods on this instance directly. It wouldn’t be unimaginable to have dotty generate the glue to allow us to say:

Color(128, 0, 255)

and proxy that through to the equivalent of new in Java.

I think this requires a re-working of how blocks are dealt with. The rule would be something along the line of:

{ ... } : A

Given this block, look at the last expression. If it conforms to A then treat it as an expression block to be evaluated. If it does not, look for member declarations needed for A. If it looks like an attempt at implementing A, treat it as such.

6 Likes
3

I would love to see new go away if the compiler can automatically infer when constructors are called. One consideration is the following situation:

class Color(x: Int)
object Color {
  def apply(x: Int): Color = {
    println("hello")
    new Color(x)
  }
}

What would Color(42) call in this case? Currently, it references apply and we would need some syntax to reference the constructor if new didn’t exist.

4 Likes
4

This case, where the apply has the same arguments as the constructor, is an anti-pattern that I’d be happy to see go away. It reliably leads to accidental infinite loops. So the ‘fix’ for this particular case IMHO is to make it illegal for a companion to provide an apply that has the exact same argument list as the class constructor.

5

Related discussion: https://github.com/lampepfl/dotty/issues/5509

2 Likes
6

I’d also be happy if we could get rid of new. It’s an annoying special case, in more ways than one.

So far I see two problems with it: How to do anonymous classes and how to disambiguate between constructor invocation and apply, i.e. what @olafurpg brought up.

Anonymous classes could be handled by something like Kotlin’s object expressions. That would also solve the problem how to get rid of with meaning multiple implementations, replacing it with commas. I.e instead of

new A with B {}

we’d now write

object extends A, B {}

No need for with anymore.

I don’t see a good solution yet for the disambiguation problem between apply and constructor invocation. A typical use case is:

class C(x: T)
object C {
  def apply(x: T) given Ctx = {
    // validate x using data in Ctx
    new C(x)
  }
}

How can we replace the new above by a plain method call?

5 Likes
7

One answer could be:

class C(x: T)
object C {
  def apply(x: T) given Ctx = {
    // validate x using data in Ctx
    object extends C(x) {}
  }
}

It’s not ideal, since it creates an extra class. But if the use case is sufficiently rare maybe that’s OK.

8

A simple heuristic would be that Foo(...) always refers to Foo.apply() if any apply method exists. So if there’s a constructor taking an Int, and an apply taking a String, then Foo(42) would be an error.

Only if no companion apply method exists with any signature, Foo(...) is then considered to refer to the constructor.

1 Like
9

Another solution could be to let the constructor take precedence over apply and require explicit .apply to disambiguate. This would be a breaking change.

10

A simple heuristic would be that Foo(...) always refers to Foo.apply() if any apply method exists. So if there’s a constructor taking an Int , and an apply taking a String , then Foo(42) would be an error.

In that case, how do you invoke the constructor if there is an apply method? Including from the apply method itself?

1 Like
11

Another solution could be to let the constructor take precedence over apply and require explicit .apply to disambiguate. This would be a breaking change.

That could work. So, given C(...) where C is the companion object of some class:

  • try to call the constructor of C first.
  • if that does not typecheck, expand to C.apply(...)

Looks reasonable, even though it would be a breaking change.

2 Likes
12

One detail concerns abstract classes. For those, we cannot call the constructor, so we go straight to trying an apply method. This means that my validation example could be expressed as follows:

abstract class C(x: T)
object C {
  def apply(x: T) given Ctx = {
    // validate x using data in Ctx
    object extends C(x) {}
  }
}

This is strikingly similar to what we do for case classes in the Dotty compiler codebase.

If we implement this scheme, then we should do the same for case classes. That is, case classes don’t generate an apply method anymore and their companion object does not automatically implement a function type.

1 Like
13

My thinking was that the constructor would never be visible externally, the compiler would auto-generate an apply with matching signature (exactly as case classes currently do) unless one was explicitly supplied.

new Foo(...) would then be reserved only for use in the companion. With opaque types, we’ve already opened ourselves to the idea of logic and constructs that are unique to companions.

1 Like
14

But if you still need new Foo(...) even though restricted in the companion, you have not gained that much in terms of simplification.

15

I’m starting to feel like in a few years almost all my code will need to be rewritten.

Is Scala 3 the last chance to make language changes forever? Is there a reason we’re redesigning every aspect of the language now?

2 Likes
16

True, though I can only think of 3 possibilities here:

  1. restrict access to new to just within the companion - this would involve the least change to the language

  2. expose the constructor as a “phantom” method on the companion: Foo.new(...) - one possible advantage here is that you could then just call it as new(...) from other methods in the companion. new would still be a reserved word, and one you could never use directly in code as a method name, nor would it end up in bytecode

  3. expose it as apply on the singleton with matching signature and make it an error to explicitly implement said signature - but this then blocks the given Ctx use-case

I personally find the object extends approach to be unwieldy.

17

What about this:

  class X(a: Int)

  object X {
    def apply(a: Int): X = {
      type UniqueName = X
      UniqueName(5) // here X.apply won't be tried
    }
  }

If we add new before UniqueName(5) then we get valid Scala 2 code.

18

That is way too convoluted and does not work in any case. In UniqueName(5), UniqueName is an undefined term name. The type definition has no effect whatsoever.

19

The idea is that right now (in Scala 2) using UniqueName I can only invoke constructors, but I can’t invoke methods from companion object. For example I can write new UniqueName(5), but I can’t write UniqueName.apply(5). This removes ambiguity - referring to UniqueName(5) in the new scheme with implied new must mean invoking constructor directly.

In which cases that doesn’t work?

20

I was considering for a while before this discussion (2): Foo.new(...). That’s a possibility. But I’m not sure the use case is common enough to demand it.

3 Likes
21

I’m (unsurprisingly) in favour of Foo.new(...)

There’s prior art in other languages to support the concept, and it’s more syntactically regular - writing (new Foo).method with the parentheses is a common practice because the right-associativity of new as an operator doesn’t scan naturally.

It would also combine neatly with being made a low-priority fallback for if no matching apply could be found for a call to Foo(...)