Expunging `new` from scala 3

Language Design
63

If you don’t generate code for them you have objects which you could call methods (apply) on but cannot pass around like other values. Like those awkward package objects.

64

Good point. But in fact, I realized that it’s already the case with Java classes due to static members:

Starting dotty REPL...
scala> String.valueOf(0)
val res0: String = "0"
scala> println(String)
1 |println(String)
  |        ^^^^^^
  |        Java defined class String is not a value
scala> List.empty[String.type]
1 |List.empty[String.type]
  |           ^^^^^^
  |           Java defined class String is not a value

More concerning is the breakage of existing code that used, in the same scope, unrelated values and classes with the same name. I don’t really know how extensive such breakage would be, though.

1 Like
65

I’m still missing the point of this whole discussion. Can someone explain to me what’s so bad about new?

3 Likes
66

It introduces (in my opinion) a special case about obtaining and forming objects, that essentially points to a low level primitive of the JVM. It introduces non-uniformity where “regular” classes and case classes are instantiated, and it’s plain ugly in its infix position when you keep on selecting elements on the resulting instance. A starting point of the discussion on Gitter was implied for Foo {} which essentially is implied for Foo = new {}, so doing away with the new operator here.

It’s a thread about the possibility of getting rid of this special operator, as several other languages do without problems (SuperCollider/Ruby, Kotlin, Python being some examples).

Personally, I have designed almost all my librarise around total avoidance of new in the public API. The only library where it’s still extremely visible, is Scala-Swing.

So it’s an opportunity to discuss whether Scala 3 really needs a special new keyword or not.

I still very much like SuperCollider, inspired by SmallTalk, has literally no special operators. if (foo) bla translates to foo.if(bla), i.e. if is a method on Boolean, new is a method on the meta class, etc. I like the Scala Virtualized project, I hope some of the ideas can go into mainstream Scala.

67

It isn’t a special case at all. It’s a basic functionality.

Yes, you can hide it in a method and call that method instead. Sometimes the method might be generated for you. But it’s still the way you instantiate things. What is it a special case of? It’s a fundamental primitive operation.

I can’t speak to the new syntax for Implicits being experimented with in Dotty, other than to say I hope we regain our senses before it’s too late. Not to get off on a tangent, but if you force every single Scala programmer to learn a new language, while claiming it’s still Scala, I don’t know what percentage of them will do so.

1 Like
69

The truth is that nobody sell such arguments.

It is a question of priorities.
It is a pitty that there is not any roadmap which says we is planning to do it one day after such thing or no we will never do it because of that. We have discussed it and it is over. It is open source, so just do it yourself :)) (it is ironic smile)

70

@AMatveev well the post of nafg comes across like that. Frankly I don’t grasp the tone. This is a forum to think freely about ways of improving the Scala language, and if you scroll through the thread, you can see where the discussion originated from, and my impression is and was that Martin Odersky was interested in exploring this question. So you know asking me to fork and implement something is just ridiculous, with and without ironic smile.

I’ll unsubscribe from this thread now. Thanks.

71

If you have top-level definitions (which you do in dotty), you could say that a constructor of a concrete class conceptually is a special top-level (or of course nested iff the class is nested) method. Which it actually really is if you think about it.

And actually implicit classes have already implemented this idea. implicit class Foo(a: Int) defines a type Foo along with an implicit “constructor” method def Foo(a: Int): Foo.

That still doesn’t solve the anonymous class usage of course.

1 Like
72

It is ironic, but actually I agree with you. I think the language will be better without new. I have written that it is a good idea.

if such idea is forgotten it will be a pity.

But there are many other good ideas, I just do not know how it can be solved without misunderstandings.

73

A bazillion operations in scala do or can potentially allocate memory on the heap. There’s only a very, very small number of these that are actually achieved through an explicit new expression.

val x = 42
vall y = "Hi Mum"
val xy = (x, y) // implicit allocation of an object on the heap

New, as a keyword, pushes us towards documenting in the code how something is implemented, where as the whole push of declarative approaches to programming is to keep you focused upon the intent rather than the implementation details. We should be moulding scala 3, imho, to at every stage let coders express the intended relationships within their application domain, and minimise the amount of time they have to deal with, frankly, irrelevant details as to how that’s implemented the layer below the language.

74

new has nothing to do with allocation IMO, at least not as related to a heap.

new is the operator that creates an new object identity, which is going to be ne to every other object that was or will be created.

It’s not a low-level operation.

(as such, I will agree with anyone who says that new shouldn’t be possible for extends AnyVal types, but that’s quite tangential to the present discussion)

3 Likes
75

It’s an operator that creates a new object identity. All other such operators do, ultimately, defer to it, but there is so much deferral that it hardly matters any more.

After all we don’t

Map(1 -> "salmon", 2 -> "herring").foreach{ new x =>
  println(x)
}

even though in most Map implementations (including Map.Map2, which is used there), that is a new object that isn’t equal to the ones we put in.

So, while true, I think the point is mostly irrelevant to whether we ought to keep new in its special language-construct position or demote it to regular-method-with-a-bit-of-special-support (possibly under a different name).

1 Like
76

Sorry if I got lost in the long discussion, but could we simply autogenerate an apply method like we already do for case classes, but now for all classes?

77

Huh? It’s a function. The implementation of foreach will provide the value. It’s the one from the Map. There’s no new identity.

The only new identity is that -> calls Tuple2.apply which calls new.

So yes, the only way you get a new instance of a class is with somebody calling new.

1 Like
78

42 and “Hi Mum” are literals. They are special cases. Builtins. Besides, where they are “allocated” isn’t especially clear. (42 isn’t an allocation, and strings might be interned… not so familiar.)

(x, y) is syntactic sugar for Tuple2(x, y) which is for Tuple2.apply(x, y) which is a method that calls new.

79

But to be pedantic, Tuple2.apply doesn’t call new. It compiles down to the appropriate invokespecial <init>.

80

I’m afraid you will find that it does:

  public <T1, T2> scala.Tuple2<T1, T2> apply(T1, T2);
    Code:
       0: new           #28                 // class scala/Tuple2
       3: dup
       4: aload_1
       5: aload_2
       6: invokespecial #31                 // Method scala/Tuple2."<init>":(Ljava/lang/Object;Ljava/lang/Object;)V
       9: areturn
81

Sorry - my bad. To many coffees in one day - what I was trying to get at is that it doesn’t use the scala new keyword anywhere. It just goes direct to the bytecode needed to make the allocation and initialisation. Apologies.

82

Here’s a concrete Pre-SIP proposal. It comes in two parts.

  1. When and where to generate a creation method
  2. What the body of a creation method is, and how that can replace new.

When and Where to Generate a Creation Method

For every concrete named class C with definition

class C[Ts](ps1: Us1) ... (psN: UsN)

where the type parameter and value parameter lists can be missing:

  • If C has a companion object, generate an apply method in it. The definition has the form

    inline def apply[Ts](ps1: Us1)...(psN: UsN): C[Ts] = 
  "new C[Ts](ps1)...(psN)"

    The generation of an apply method is suppressed if that method would clash with an apply method already given in that companion object. The meaning of “clash” is the criterion that currently determines whether an apply method for a case class should be generated.

  • If C does not have a companion object, synthesize a special empty companion object

    object C {}

    and proceed with step (1). Unlike explicitly given companion objects or companion objects of enums or case classes, these synthesized objects can only be used as a prefix of a selection. They cannot be referred to as a value on their own. In that respect they are treated like companion objects of Java classes.

Details

  • Classes without value parameters get a creation method with an empty
    parameter list. I.e.

    class StringBuilder { ... }

    would get the creation method

    object StringBuilder {
 inline def apply() = ...
}

  • The visibility modifiers private and private[enclosing] are inherited
    by the generated apply method from the class constructor.

How to Replace new

The new-expressions of the bodies of the creation methods above were given in
quotes, meaning that we want the effect of the presented new expressions, but
not necessarily the syntax. In fact Scala already offers with object definitions
a way to simulate new expressions that represent anonymous classes. I.e.

new C[Ts](es) { ... }

can be replaced with

{ object x extends C[Ts](es) { ... }; x }

where the name x is arbitrary. Applying the same technique to regular instance creation expressions, we get:

new C[Ts](es) { ... }

is replaced with

{ object x extends C[Ts](es); x }

This is currently not a semantics-preserving expansion, since the new expression creates an instance of class C itself, whereas the object definition creates an instance of an anonymous subclass of C.

The proposal is to close the semantic gap by the following rule:

  • An object definition of the form object x extends C[Ts](args1)...(argsN) that has
    a single parent class C and that has no body enclosed in braces creates an instance of class C instead of an anonymous subclass of C.

This is the same distinction we make for new expressions, now generalized to object definitions.

With this rule, we have a way to express faithfully all forms of new expressions by local object definitions. It is proposed to deprecate and phase out new expressions in a language version after Scala 3.0, where the precise deprecation schedule still needs to be determined.

Why is it necessary to find faithful expansions for new-expressions, given auto-generated creation methods? There are two reasons. First, we need a way to replace anonymous classes. Second, sometimes an apply method is not generated because it would clash with existing apply methods. These competing methods then need a way to create the instance without the ability to resort to an auto-generated method.

Aside: The dotc compiler would desugar an object definition

object x extends C[Ts](es)

to

lazy val x = new C[Ts](es)

This re-introduces new expressions but now only as a lower-level implementation detail.
There is no need to expose this functionality in the source language.

5 Likes
83

I think a colon may then as well replace extends.
We could then write:

object: A, B {}

With the colon that would be:

object x: C[Ts](es)

Maybe I am missing something, but for now I think the colon would be a good replacement for extends.