Expunging `new` from scala 3

Language Design
22

Here’s another shot (this also works under Scala 2.12):

type Id[A] = A

class X(val a: Int)

object X {
  def apply(a: Int): X = {
    new Id[X](5) // this `new` can be inferred without ambiguity
  }
}

If Id is global then this scheme doesn’t require new unique name per class. I admit it looks somewhat weird, but it does solve the problem. If it will be used in some rare corner cases only then why not? To make things clearer Id could be renamed to e.g. ClassType or New:

type New[A] = A

class X(val a: Int)

object X {
  def apply(a: Int): X = {
    New[X](5) // here I omitted `new`, which is inferred without ambiguity
  }
}
23

I would support getting rid of new if we could find a way to substitute its less common usage patterns (e.g. defining anonymous inner classes). Perhaps we may need to leave it in in case of ambiguity (calling constructor vs calling companion-apply) but in the vast majority of cases it is unambiguous and the compiler should know what to do when it sees Foo(...)

Kotlin doesn’t have new, and Python doesn’t have new, and both seem to get by just fine.

3 Likes
24

Not at all, you can start rewriting your code now in Scala 2.13 thanks to the deprecation of package object inheritance. That’s a warmup for things to come in Scala 3.

Seriously, was hoping to put off a major rewrite until Scala 3, but it seems like the above merged PR and changes to Scala collections in 2.13 will entail some pain as well.

In some ways Scala is become less like Scala and more like something else. Not too thrilled about removing core language features, but apparently they don’t carry their weight in terms of justifying the maintenance/complexity overhead.

25

Kotlin uses object extends instead, which is even more verbose and irregular, and Python does not have something like anonymous classes and mixin composition AFAIK. So the comparison is moot.

What’s the problem with new? It’s always been clear to me that it’s for instantiating classes directly, with all the extra power that comes with it, like the possibility of defining additional parent classes, members, or overrides.

This is the way Java does anonymous classes, too. IMHO the way Kotlin does it is irrelevant. I don’t understand why one would consider breaking this in Scala for no clear reason (the ones given in this thread are really unconvincing).

Maybe it would be enough if there would be a way to have apply methods synthesized for non-case classes? These methods could even be synthesized by default. But by all means, don’t mess with new just for the sake of it (unless perhaps it’s for a real improvement, as the one proposed in Further enhance `new` syntax to reduce boilerplate · Issue #2 · lampepfl/dotty-feature-requests · GitHub).

6 Likes
26

Another solution could be to let the constructor take precedence over apply and require explicit .apply to disambiguate. This would be a breaking change.

This looked like an elegant solution but unfortunately the breakage is too severe. Consider Array(10). With the proposed change that would allocate a new array with 10 elements instead of a one-element array. No change in the types, so this will slip through. We can’t allow that to happen.

So, we’d need to interpret C(...) as C.apply(...) first and could fall back to new only if that fails. That means we’d still need syntax for new.

The most conservative change would be to keep new C(...) and to simply allow C(...) as alternate syntax (in case there is no matching apply, which would take precedence).

A more sweeping change would be to invoke new as if it was a method. i.e, C.new(...) instead of new C(...). This leads to less awkward expansions. For instance, if we have

class B(x: Int) { ... }
class A(x: Int) { ... }

then

A(1).B(2)

would expand to

A.new(1).B.new(2)

In “classical” new syntax, this cannot be expressed, since the following is not legal syntax:

new (new A(1).B(2))

Another question concerns anonymous classes, i.e. new A { ... }. If we wanted to drop “classical” new syntax, these would have to be replaced by something like object expressions. Maybe that’s for the good. In my many years of teaching I have always found anonymous classes a major stumbling block for students. Problems include:

  • Anonymous classes re-bind this in ways no student suspects at first.
  • Type inference works differently for anonymous classes compared to normal new. E.g.
scala> class A[T]
defined class A

scala> val a: A[Int] = new A
a: A[Int] = A@1213ffbc

scala> val b: A[Int] = new A {}
<console>:12: error: type mismatch;
 found   : A[Nothing]
 required: A[Int]

In Scala-3 there will likely be the added problem how to express anonymous classes with multiple parents, since with will probably go away.

27

Yes, but I think these are both secondary concerns. To quote Alan Kay: “Simple things should be simple and complex things should be possible”. Anonymous classes and mixin composition count as complex things in my book.

28

local objects don’t have the same semantics that local anonymous classes have. Consider:

def test() = {
  object A {
    println("1")
  }
  val x = new {
    println("2")
  }
}
test()

This will print 2 but not 1 because objects are only initialized when first called, so we can’t replace new by object extends (and I think keeping new just for disambiguation and making anonymous classes is fine, even if one is confused by it, googling Scala new should clear things up).

1 Like
29

The proposal would be to let

object extends A { ... }

be a shorthand for

{ object $anon extends A { ... }; $anon }

which is exactly the same as the current expansion of new A {...}.

This is clunkier and more verbose, but solves some of the problems with anonymous class syntax. First, it makes it clearer that this is in fact rebound. Second, being visually different from new A() there would be no expectation that type parameter inference should behave the same. Third, it generalizes readily to multiple parents:

object extends A, B { ... }

But it is clunkier, so I am not sure which one is better yet.

1 Like
30

Weren’t anonymous classes with multiple parents going to be deprecated?

31

Weren’t anonymous classes with multiple parents going to be deprecated?

No such decision has been made. If we want to get rid of with, but keep the existing anonymous class syntax, then we might need to deprecate it. But that’s two conditions that are not cast in stone yet.

32

@nafg It’s a good question. Before I answer “of course not”, I want to explain what the motivations are and what that means for possible language design decisions and their timing.

We have accepted that Scala 3 will introduce breakage and that we will resort to automatic rewrite tools to mitigate that as much as possible. My goal is that with rewrite tools in place necessary manual fixes should be typically on the order of 1 for every 1000 loc, and furthermore, that the overwhelming majority of breakage should result in compile-time errors. Silent changes in runtime behavior should be kept to the absolute minimum. So far, the most significant change in runtime behavior is that lazy vals do safe publishing only if they are declared @volatile, and we will probably revert that change, since it does not conform to that goal.

But code is just one dimension of breakage. The other is teaching and documentation. WIth Scala 3 we are doing something quite radical. Rather than putting new layers of additional complexity on what is already there, we accept that Scala 3 is when all the books have to be rewritten. The goal is to arrive at a language that is faithful to Scala’s core, but that is clearer and simpler than what we currently have. You cannot rewrite books continually, in fact it is a huge bet to demand that at all!

These considerations guide the timing. Essentially, if there is a proposed change that simplifies the language, replacing a more complicated construct, we have to roll it into Scala 3, if we want to do it at all. It can coexist with the “old style of doing things” for a while. The aim is that it should be possible to produce tutorials that only use the new, simpler style and that need not mention the previous, more complicated constructs.

That reasoning explains some of the choices for Scala 3.

  • If we want to make implicits easier to use it has to be for 3.0
  • If we want to use ? instead of _ as a wildcard type, we have to do it for 3.0.
  • If we want to drop new, we have to do it for 3.0.

On the other hand, some choices could also come later, e.g.

  • Kind polymorphism
  • Polymorphic function types
  • Null safety

These are good additions to have, for sure, and they might make it in time for the Scala 3 feature freeze. But if they do not make it in time, they can also be added in 3.1 or 3.2, because the presence or absence of these features does not fundamentally influence how one writes a language tutorial.
Kind polymorphism and polymorphic function types are advanced features and null safety is mostly about interop. It’s perfectly possible to write a language tutorial without mentioning these aspects.

So that explains why it is a good time to discuss expunging new now. The outcome might very well be that we will keep new as it is, after all. But it’s worthwhile to give that question some thought.

5 Likes
33

To be more explicit: my problem with object extends is that in all other contexts, if you see object you need to remember that the constructor won’t be called at this point but that doesn’t hold here. This risks making the (already somewhat surprising) semantics of object creation even murkier.

1 Like
34

So here is some real code. I prefer looking at real code to toy examples, where possible.

implied ValueAndThenPosition[Buffer, A] for ParseOneThenOther[Value[Buffer, A], Position[Buffer], Value[Buffer, A]] =
    (lhs, rhs) => (buff, pos) => lhs(buff, pos)(new {
      override def matched(lEnd: NonNegativeInt, value: A) = rhs(buff, lEnd)(new {
        override def matched(rEnd: NonNegativeInt) = ValueResult.wasMatch(rEnd, value)
        override def mismatched = ValueResult.wasMismatch
      })
      override def mismatched = ValueResult.wasMismatch
    })

It’s part of my toy parser combinator library. There are nested call-back instances that handle the parser success and failure conditions. It is, I freely admit, a horrible heap of spaghetti to read. But the point I want to make is that the new keyword is shot through, and it really doesn’t help any, and ideally once the compiler has got at this, there should be no or minimal allocations.

implied ValueAndThenPosition[Buffer, A] for ParseOneThenOther[Value[Buffer, A], Position[Buffer], Value[Buffer, A]] =
    (lhs, rhs) => (buff, pos) => lhs(buff, pos){
      def matched(lEnd: NonNegativeInt, value: A) = rhs(buff, lEnd){
        def matched(rEnd: NonNegativeInt) = ValueResult.wasMatch(rEnd, value)
        def mismatched = ValueResult.wasMismatch
      }
      def mismatched = ValueResult.wasMismatch
    }

This is the same thing with new removed, and with the “block treated as implementation” heuristic. I’ve also stripped out override.

If we were also allowed to in-place (un-)curry method implementations, I could have written:

implied ValueAndThenPosition[Buffer, A] for ParseOneThenOther[Value[Buffer, A], Position[Buffer], Value[Buffer, A]] =
    (lhs, rhs) => (buff, pos) => lhs(buff, pos){
      def matched = (lEnd, value) => rhs(buff, lEnd){
        def matched = (rEnd) => ValueResult.wasMatch(rEnd, value)
        def mismatched = ValueResult.wasMismatch
      }
      def mismatched = ValueResult.wasMismatch
    }

While this is still not ideal, it is, I’d contend, far more readable than the original. We’ve got rid of nearly all the explicit type annotations and (which is a big win for dyslexics like me) reduced the nested brackets and braces substantially.

35

According to the criteria I have laid out above, it looks like our course of action is greatly restrained, which is a good thing. So, here’s a modest proposal:

  • Objects of any class C can be created by plain applications C(...). This is arguably simpler and more uniform, since it generalizes what we do for case classes already. Case classes have a number of additional properties, which all but one rely on the fact that case classes keep their constructor parameters as fields of a Product. The only extra ability that has nothing to do with this is the construction syntax without having to go through new. It makes sense to decouple that functionality and make it available for all classes, not just for case classes.
  • Facing an application C(...) where C is the companion object of a class we first try an apply method, and only if that fails convert to an instance creation. We have seen this order is essential for backwards compatibility.
  • No other changes are considered at this time. We still keep new both in its simple and in its anonymous class forms. We need some way to express these two functionalities anyway. Furthermore, with the added C(...) syntax being the recommended way of doing things, explicit new will be used only in relatively rare cases (when an anonymous class is needed, or when we have to create an object in an apply method that takes the same parameters). So if we want to change this part of the language, it can well be after Scala 3.0.
7 Likes
36

It seems good.
If there are ImplicitFunctionClass the scala will be more closer to language of my dreams.

37

Perhaps I’ve not entirely groked what you are after, but the @functionalInterface annotation on SAMs may be the droid you are looking for.

@FunctionalInterface
trait And[B] {
  def and(lhs: B, rhs: B): B
}

implied BooleanAnd for And[Boolean] = _ && _
38

Sorry, It seems, I have maken too short description.
I want to say that new is not big boilerplate for me.
It is not very hard problem to make object.apply, for me.
The real boilerplate is the lack of kotlin receiver function

I think it can be made by simple syntactic sugar in most cases. For example in the topic I have maken the example:

abstract class ExtendedFunction{
  def execute():Unit
}
def doSomeThing(f:ExtendedFunction):Unit = {}
def main(args:Array[String]):Unit = {
      doSomeThing{
          println("ok")
      }
     /*it is similar to 
     doSomeThing{ new ExtendedFunction{
        override def execute():Unit = {
          println("ok")
        }
      }    
     */ 
}
39

How would this be done?

  • Every class will now have a companion object? It’s a breaking change with no obvious automatic fix, and means many more generated classes in the bytecode.

  • Or is this going to be some magic rule that treats a type name as if it was a value under some circumstances, but not other? e.g., A(0) may work but not println(A).

In fact, even the second option is a bad breaking change. Consider:

def A(n: Int) = n + 1
class A(n: Int)
A(0) // currently returns `1`

The thing is either class A or def A could come from another package entirely, via an import. How is the refactoring supposed to work, then?

EDIT: some instances of new would have to be left behind anyway to avoid breaking code; my main qualm with this second alternative really is about the loss of value/type namespace distinction.

2 Likes
40

This is by far my favorite option. The irregularity of language constructs is familiar but, I think, fundamentally complicates matters. Given that companion objects exist (or that static methods exist, for languages that have those instead), having magical different syntax for default object creation is a peculiar choice. All sorts of factory methods exist for many classes (e.g. Java is replete with .from and .of, and Scala has a bit of that plus lots of .apply); why have an alternate syntactic form for default object creation as opposed to all the other types of object creation?

If it were possible to actually make it act like a method (e.g. you could get a default creation function by doing Foo new _), I think the language would be considerably more regular.

(Same thing with match; rather than being a peculiar language construct, it should be called and act like a slightly-enhanced method invocation.)

The object extends Foo syntax is, admittedly, rather a clunky way to create anonymous functions. For Scala, where this is usually rather rare, I think the benefits are worth it.

A slightly cleaner syntax is available by abandoning extends rather than with. That is,

Foo.new(15) with {
  override def toString = "example!"
}

This syntax could also be extended to relatively transparent forwarders, so you could (given a constructor-like apply)

Foo(12) with {
  override def hashCode = 12
}
2 Likes
41

Yeah this is exactly what I suggested. Basically let people instantiate classes without new when there’s no ambiguity, leave new in place, start teaching people to use the new-less version unless there’s some real reason they need to disambiguate and then they can use new.

As Martin said, it’s OK for irregularities to remain as long as we’re making the common case slightly smoother, and I’m OK for new to remain indefinitely as something slightly esoteric that most people will not need to worry about day-to-day.

Maybe in future if we decide to get rid of new and replace it with other things we can, but for now this would be 100% backwards compatible (both w.r.t. code and people) and would allow a gradual migration to using the new new-less convention with zero breakage.

3 Likes