SIP: Make classes `sealed` by default

[odersky]

Scala extends Java’s basic class system, using the same notions for class, abstract class and final class. In hindsight, I believe the default in this system that every class is extensible is problematic. It leads to straightforward code being vulnerable to unforeseen overrides and requires final modifiers in too many places.

This pre SIP is a proposal to change the default class extensibility to sealed. It can be summarized as follows:

1. A normal class definition implies that the class is sealed, I.e. if it has subclasses, they must be in the the same compilation unit.
2. To have a class that can be extended in other compilation units, the class must have the open modifier. open is a soft modifier, like in Kotlin.
3. abstract implies open.
4. A sealed, non-abstract class C (i.e. the default) is allowed to have abstract members. It is then checked that all non-abstract direct subclasses of C implement these members. This is easy, as all these classes are in the same compilation unit.

Why Propose This Now?

Scala 3.0 is already over-full with changes. Can this not wait for a future version?

The problem with waiting is that we want to roll changes that “require the books to be rewritten” all into 3.0, so that docs stay valid for a long time afterwards. Changing the default for class extensibility is so fundamental that it would affect every tutorial.

How To Get There?

Since this is a fundamental change, changing existing code bases will be a hassle. Some measures to soften the impact are:

• The Scala 3 compiler will treat all Java classes and all Scala 2.x compiled classes as open. So they can be freely extended in application code.

• Extending a default class in another compilation unit will give a deprecation warning. We will wait to see how migration goes before turning that into a hard error. EDIT: Or maybe use some other warning instead of a hard error. The warnings should be turned on only in Scala 3.1. That way, we still enable cross compilation between Scala 3.0 and Scala 2, which does not allow the open modifier.

• We should think whether some automatic rewrites are possible. This is not trivial however, since simply slapping open on every class would probably be wrong in more cases than it is right. I believe that most classes are in fact not intended to be open and just did not get a final or sealed modifier out of laziiness.

Why is the default sealed and not final?

Writing simple class hierarchies is common. E.g.

  class A
  class B extends A
  class C extends A

Requiring a sealed in front of class A means the information that’s already present in the extends clauses has to be specified again, which is unnecessarily pedantic. Indeed, B and C will be classified as final anyway since they are sealed and do not have any subclasses in the same compilation unit.

Is sealed still needed as a modifier?

Not really, The only remaining use case is sealed trait. But if we talk about a single parent of a finite number of alternatives, making the parent a (sealed by default) class is more appropriate. Otherwise, we’d be looking at a set of classes that have different parent classes, yet all implement an additional trait, which furthermore is restricted to be extended by just these classes. I believe that use case is very uncommon. So, in the interest of not offering unnecessary choices, we should deprecate sealed in a future version of the language.

EDIT: it looks like sealed is still needed for traits. An example is an ADT for an abstract syntax tree, where all cases extend one base class, but certain cases extend some additional classification trait. The trait should be sealed to give pattern matching exhaustivity.

[Jasper-M]

Might be worth linking to this previous discussion about this design space: Make concrete classes final by default

[smarter]

@odersky

The problem with waiting is that we want to roll changes that “require the books to be rewritten” all into 3.0, so that docs stay valid for a long time afterwards. Changing the default for class extensibility is so fundamental that it would affect every tutorial.

Alternatively, we can introduce the open keyword in Scala 3.0 but without giving it any semantics, it would only be used to document that a class is intended to be extended from the outside. Then 3.1 can introduce the deprecation warning, and 3.N can make it an error. This avoids cross-compilation headaches with versions of Scala that do not understand the open keyword, and still allows books to be written for 3.0 that use the keyword.

[MarkCLewis]

@odersky, the last paragraph gives me the impression that this sealed by default rule would only apply to classes and not traits. Is that correct?

[Tom_Poczos]

IIRC C# answers a similar question by making methods non-overridable by default. Might be worth considering combinations of these:

sealed by default classes + non-overridable methods by default
sealed by default classes + overridable methods by default
open by default classes + non-overridable methods by default

[odersky]

Yes, this is a direct continuation of the previous discussion. Thanks for pointing that out. The main changes are:

• The proposed default is sealed instead of final.

• We now support soft modifiers, so using open should not be a problem.

• Instead of outlawing extensions of non-open classes outright, we emit a deprecation warning (it could be some other warning, this is a point to discuss), So the new system is more like the proposal by @nafg that evolved in the previous thread in that there are now three levels:

  • open: all extensions are allowed
  • default: extensions in other compilation units give warnings
  • final: no extension allowed

  This also means that tools like Mockito can still be used in the same way as today since we would not generate a final Java modifier for a default class.

@MarkCLewis Yes, traits are not sealed by default.

[Jasper-M]

@odersky

A sealed, non-abstract class C (i.e. the default) is allowed to have abstract members. It is then checked that all non-abstract direct subclasses of C implement these members. This is easy, as all these classes are in the same compilation unit.

So this would mean:

// foo.scala

// ERROR: concrete class Foo defines abstract method foo
class Foo {
  def foo: Int
}

// bar.scala

// OK, Bar is implicitly abstract
class Bar {
  def bar: Int
}

class Bar0 extends Bar {
  def bar = 0
}

class Bar1 extends Bar {
  def bar = 1
}

I’m not sure how I feel about that part yet.

[tarsa]

@odersky

• This also means that tools like Mockito can still be used in the same way as today since we would not generate a final Java modifier for a default class.

If there’s direct support for Mockito then why not add direct support for other types of “aggressive test code”, i.e. overriding methods freely? By direct support I mean language flag like scala.language.openByDefault. I would add such option to compiler options in test configuration and be free to override methods for testing purposes (e.g. replace globally observable side-effects with local ones).

[RichType]

@odersky

The problem with waiting is that we want to roll changes that “require the books to be rewritten” all into 3.0, so that docs stay valid for a long time afterwards. Changing the default for class extensibility is so fundamental that it would affect every tutorial.

I’m actually reasonably sympathetic to this particular proposal, but trying to ram it through with the hope of not having to modify educational material seems wrong to me for several reasons. First I think our slogan should be “Developers, developers, developers, developers.”

1. Obviously the needs of the Compiler writers and the Standard Library should carry the heaviest weighting.

2. Then the developers of the major open source libraries.

3. Then the big business Scala users that help promote and expand the Scala eco system.

4. All the other Scala developers.

Basically I’m saying that we should prioritse the needs of the developers who have already made a commitment to Scala over those that we hope to attract / educate / train in the future. I certainly think that

1. Learnability
2. Transferabilty

should be considered axis of scalability and should be priorities in the long term evolution of Scala. That Scala should aim to scale from a language accessible for beginner hobbyists, playing around on their home computers without profession tuition, to the top software houses employing the best graduates with in house training in advanced functional techniques.

I also think it should be accessible to those experienced in C/C++ Java and JavaScript, allowing them to quickly become productive and get an early pay off from adopting Scala, rather having to spend months or years mastering idiomatic “pure” functional Scala before Scala adds value.

So yes education is important but it must be part of the long term evolution of Scala, but not some year 0 revolution. With all due respect, perhaps Education establishments need to think about some sort of Agile paradigm. Perhaps they need to think about continuous development when it comes to courses.

As for books, they are out of date by the time they’re published. That’s just life. But they’re are still very useful. And I’m very happy to make a personal promise. If you bring out a new edition of “Programming in Scala” each year I will buy them all. I would also be happy to buy a couple of advanced Scala book editions every year.

So I think Scala Programming 101, students need to be told Scala will change. Scala is very good, but it needs to keep changing, so as it can keep getting better. But also change is also slower than it first appears. Its still worth learning even if it changes. In programming as in the rest of life, you can never fully understand the present without understanding the past. I would suggest that if you read the first edition of K & R back in 78, there’s still a significant amount of that learning from that which you can leverage programming Scala in late 2019.

[nafg]

Tangentially… I agree with prioritizing the happiness of existing developers over new developers, because I think that the best way to attract new developers is for them to see that Scala developers love Scala.

Traditionally this has been lacking. (It’s been suggested that it’s because Scala attracts people that are language perfectionists – they see it as the least bad language! I don’t know how much truth there is in that theory.) Whatever the reason, I don’t often see people talking about how much they love Scala they way I see Ruby, Python, Rust, and Typescript developers doing.

[dwijnand]

@odersky

Is sealed still needed as a modifier?

Not really, The only remaining use case is sealed trait .

Seems sealed is needs for abstract class too, if abstract implies open.

[Ichoran]

I think extensibility by default is preferable. Restrictions disempower programmers, which is okay if the power is mostly used to make trouble. But subclassing is often used to fix problems in original designs, adding hooks etc. where there should have been but were none.

Additionally, assuming sealed would be quite annoying for those codebases that have subclasses in different files. I expect that this is pretty much all of them.

I would rather focus effort on whole-program linters that can warn you “hey, this thing looks like it could be final/sealed/whatever”, and can make the change for you if you agree.

[dwijnand]

@Ichoran

I would rather focus effort on whole-program linters that can warn you “hey, this thing looks like it could be final/sealed/whatever”, and can make the change for you if you agree.

Such as GitHub - rorygraves/ScalaClean: Full program static analysis for Scala.

One problem I have with this proposal (which I have in general about sealed, but is exasperated by this change) is that sealed invites pattern match usage, which with separate compilation would mean a MatchError in your app when I decide to add class D extends A in my library.

[soronpo]

@odersky

open : all extensions are allowed

Is it possible to refine the open scope as open[SomeScope] class Foo?

[bmeesters]

I would very much be in favor of this change. I never extend a normal class. Either I design for extensibility and use an abstract class or a trait, or I design for final and use a final class (or a normal one if I am lazy).

[arkban]

Personally I’m against upstream barriers to inheriting and overriding. I realize I’m some nobody with no name recognition but I worked in C# for about 8 years and the decision in .NET to make non-overridable the default was a huge pain.

It was a huge pain because if an library author didn’t use interfaces consistently and mark everything as they thought would be useful to override as virtual. If the library author didn’t then the library consumer was out of luck. Even the best library authors aren’t perfect and will forget to use an interface or mark something virtual.

IMHO the crux of the problem is that a concept like open requires perfect forward knowledge by the library author. They have to be able to accurately predict every potentially useful modification someone might want. We all know that’s not possible – if we did we’d be much happier and probably richer than we are right now ; )

Let me ask this another way: how many times have you worked with a library and thought “I need to do X, if only the library exposed Y or did Z this way, this would be easy”? I’ve done that a lot. And I don’t blame the library author because I know there’s no way they could predict yesterday what I might thinking today.

TL;DR Making things non-overridable by default takes away significant power from the library consumer, and can make it very hard if not impossible to re-balance those scales. To be fair I understand many (all?) of the counter-arguments. I simply making stuff non-overridable by default swings the pendulum too far the other way.

[cchepelov]

there is another semi-legitimate residual use case for “sealed trait”:

sealed trait T {
   def publicApi(param: Int): String
}
abstract class TypicalImpl extends T {
     final def publicApi(param: Int): String = {
       (implementorsSpi(param+5).toInt -5).toString
     }
     protected def implementorsSpi(paramPlus5: Int): String
}
class SpecificCase extends T {
   def publicApi(param: Int): String = param.toString
}

Edit: on the proposal itself, why not. There will be a set of habits to break/migrate anyway (not just the code base). On the other hand, the wider the rift, the higher the chance to get bitten by the 3000 pythons down there.

[soronpo]

There is something I don’t understand. sealed is limiting the extension to the same file and not the same compilation unit. Is this expected to change in Scala 3?

[odersky]

source file = compilation unit. Sorry if that was not clear.

is it possible to refine the open scope as open[SomeScope] class Foo ?

Yes, or maybe rather sealed[SomeScope], which follows the convention used with private.

[nafg]

@odersky

Why Propose This Now?

Scala 3.0 is already over-full with changes. Can this not wait for a future version?

The problem with waiting is that we want to roll changes that “require the books to be rewritten” all into 3.0, so that docs stay valid for a long time afterwards. Changing the default for class extensibility is so fundamental that it would affect every tutorial.

Sorry for being so repetitive, but I think this is more evidence against this argument.

Languages need to evolve because we are human and we cannot anticipate everything in advance. There is no point at which we have thought of everything.

If we don’t accept this then two things will happen. (1) Until dotty has been feature-frozen, every time a new feature is thought to be important, the release date of Scala 3.0 will slip more and more. (2) Once Scala 3.0 is baked, there will be a strong resistance to adding new features, encouraging a “feature winter” like the last 8 years.

(This is besides the arguments I’ve made previously, that not releasing early and often greatly increases risk. “Preview releases” don’t count; they’re more comparable to a company’s internal manual QA process.)

Of course I have not offered any solutions to the Book Problem. I’m not sure I have any off the top of my head but I guess it would be for a separate discussion in a separate thread.

My point here is that I don’t think this change (among others) should be in 3.0.