PRE-SIP: open and closed classes and methods/fields

relates to:



I think it makes sense to introduce closed as new extensibility modifier and kind of reactivate the intended behaviour of open. Yet, I suggest modifications to the current documentation.


IMO this proposal would achieve the following points

extensibility modifier on class level

I suggest to introduce a new extensibility modifier closed next to open which will lead to 3 main states of extensibility:

  1. final => cannot be extended from and methods/fields without extensibility modifier are thus implicitly final as well

  2. closed => is not intended to be extended from but one can (how will be outlined in section “extend closed types” below).

  3. open => is intended to be extended

The current documentation states that omitting the extensibility modifier equals to closed. IMO it is better to have an explicit modifier as it allows to re-interpret the omitted modifier differently (see suggestion for finalFirst in QA). Nonetheless, I also propose to use closed as default if an extensibility modifier is omitted.

extensibility modifier on member level

Using open on methods/fields/inner type produces currently an error. Yet, omitting the extensibility modifier on a method/field/type results currently in open semantic.

I propose that one can use closed on the member level in the same way one can use final. Moreover, I suggest that omitting the extensibility modifier on member level is the same as using closed (also for inner classes/types). As consequence, I suggest that one can use closed and open modifier for members to open them up. abstract members (including inner abstract classes/traits) are implicitly open and I suggest we generate a warning if one specifies both modifiers (e.g. open abstract def) as we do already for open abstract class.

This leads to the following behaviour

  1. final => this method/field/type cannot be overriden in a sub-type (this inner class/trait cannot be extended from)
  2. closed => this method is not intended to be overriden but one can (how will be outlined in section “override closed method” further below)
  3. open / abstract => this method can be overriden and it is intended to do so (e.g. open override in case of the class adapter pattern).

I am aware of that this is quite a drastic change and I can imagine that many don’t feel comfortable with going through all libraries which one wants to release with the new scala version containing this change. Thus, I suggest to introduce an annotation on class/trait level: @memberExtensibility with values "final", "open" which has the effect, that members without extensibility modifier are either final or open instead of closed (members of inner classes/traits kind of inherit the behaviour). This way the migration should be easy IMO (regex search and replace) .

The extensibility modifier on member level would give the possibility, that one can:

  • define that a member is final in a class / trait without @memberExtensibility or in a class / trait which has @memberExtensibility("open")
  • define that a member is closed in a class / trait which has @memberExtensibility("final"/"open")
  • define that a member is open in a class / trait without @memberExtensibility or in a class / trait which has @memberExtensibility("final")

To sum up:

  1. omitting an extensibility modifier on a member results in the member being closed
  2. allow to set a memember to closed or open (and final which is already possible)
  3. introduce the ability to modify the default extensibility modifier via @memberExtensibility on the class/trait
  4. remove the current warnings regarding using open for methods/fields/types

extend closed classes

I would take a similar approach as outlined in the documentation but with slight modifications

Classes that are closed can still be extended, but only if at least one of two alternative conditions is met:

  • The extending class is in the same source file as the extended class. In this case, the extension is usually an internal implementation matter.

  • The language feature adhocExtensions is enabled for the extending class. This is typically enabled by an import clause in the source file of the extension:

    import scala.language.adhocExtensions

    Alternatively, the feature can be enabled by the compiler option -language:adhocExtensions.

suggested addition/change

The doc states:

If the feature is not enabled, the compiler will issue a “feature” warning.
the feature warning for ad-hoc extensions is produced only under -source future. It will be produced by default from Scala 3.1 on.

I don’t know what the difference is between a feature warning and a regular warning. I guess it is related to -source future. If this is the case then I suggest we still emit a feature warning in 3.1.x (even though the documentation states the compiler will emit a regular warning in 3.1) and a regular warning starting from 3.2 or later.

Yet, I propose that we do not emit a warning if the feature is absent but an error because warnings can easily be overlooked and IMO it should be opt-out, not opt-in. If one has enabled the language feature adhocExtensions then I would still emit a specific warning for extending a closed class. One can suppress the warning per class, per file or even globally (e.g. for test sources). Maybe I am just not aware of it, in case a language feature can somehow be enabled on a top level class (and not for the whole file), then we would not need to emit a warning.

To sum up:

  • compile error when extending a closed class
  • specific feature warning if adhocExtensions is enabled in 3.1 (only if -source future activated)
  • specific regular warning if adhocExtensions is enabled in 3.x

override closed members

Members that are closed can still be overriden (extended from in case of an inner class/trait) but only if the adhocExtensions feature is enabled in the corresponding file or globally. If the language feature is enabled, then a specific feature warning (a different one than the one we emit when extending a closed class) is emitted in 3.1 and a regular warning in 3.2 or later

One can suppress such a warning per member, per class, per file or even globally.

Relation to suggested implement solution.

Personally, I see the point that override is not the best chosen name in case a sub-type implements an abstract member. This proposal does not address this point. Neither that omitting override in a sub-trait in case it implements an abstract member does not lead to an error. However, since omitting an extensibility modifier on member level makes it closed we cover at least part of it (see compile error if adhocExtensions below).

trait A
  final def foo1()
  final def foo2()
  def bar1()
  def bar2()
  closed def bar3()
  closed def bar4()
  open def bar5()
  abstract def zulu1()
  abstract def zulu2()
  abstract def zulu3()
  abstract def zulu4()
  abstract def zulu5()

trait B extends A :
  def foo1()          // compile error, foo1 is final
  override foo1()     // compile error, foo2 is final
  final def bar1()    // compile error if adhocExtensions is not in scope otherwise warning (no warning about missing override though)
  def bar2()          // compile error if adhocExtensions is not in scope otherwise warning
  override def bar3() // compile error if adhocExtensions is not in scope otherwise warning
  closed override def bar4()  // compile error if adhocExtensions is not in scope otherwise warning
  open override def bar5()    // OK, as bar5 is open in B
  final def zulu1()           // OK (no warning about missing override)
  def zulu2()                 // OK (no warning about missing override)
  override def zulu3()        // OK
  closed override def zulu4() // OK
  open override def zulu4()   // OK

trait C extends B :
  def bar1()   // compile error, bar1 is final in B
  def bar2()   // compile error if adhocExtensions is not in scope otherwise warning, bar 2 is closed (extensibility modifier missing in B)  -- no warning about missing override
  override def bar3()   // compile error if adhocExtensions is not in scope otherwise warning
  override def bar4()   // OK
  def bar5()            // OK (no warning about missing override)
  override def zulu1()  // compile error, zulu1 is final in B 
  def zulu2()           // compile error if adhocExtensions is not in scope otherwise warning  (no warning about missing override though)
  override def zulu3()  // compile error if adhocExtensions is not in scope otherwise warning
  override def zulu4()  // OK


Why not final-first approach?

I see from the previous discussions, that a final first approach will not have many advocates, thus I propose a closed-first approach instead as already proposed by the documentation.

Personally, I think it makes sense in application code to take a final first approach (i.a. to mitigate security holes), meaning that if one omits an extensibility modifier on a class then it should not be implicitly closed but final. Same goes for members which are currently implicitly open.

One could argue that closed is kind of final if no adhocExtensions is present but the resulting jvm byte code is still vulnerable to attacks using non-final classes/methods as attack vector. In times of increasing supply-chain attacks, we think it would be better if we took a security-first approach and thus use final in case one omits an extensibility modifier (for classes as well as for members).

The closed-first approach might be a better fit for libraries sometimes but I suggest to have “end-users”, in the sense of application code writers, as first target audience of scala and library authors as second target. I think even in libraries it’s most of the time better to use final per default and only open up if really required/requested. I am aware that this is controversial in the scala community but I believe that if a library author prefers closed or open classes/traits, then it’s not much work to add the corresponding modifier to the class/trait, use @memberExtensibility respectively.

Yet, I guess this proposal would lose ground if I really go with final-first because I saw from the previous discussions, that a final first approach will not have many advocates, thus I propose a closed-first approach instead as suggested in the documentaiton.

Nevertheless, I am open (pun intended) to add a language feature flag finalFirst – or maybe something like -security:finalFirst on (with default off)
If the security feature is not in scope, then the default extensibility modifier for a class/member without one is closed as already advertised in the docs.

Do jvm modules not solve the same problem as final first?

Only part of it. You still want (or accidentality) expose concrete classes and those should be final per default.
Also, it seems to me that the scala ecosystem (still supporting jdk 8) is not yet fully ready for jvm modules.

Are there other languages which use a final first approach?

  • Kotlin uses final first on class level but not for members
  • C# uses final first on member level but not on class level
    => IMO scala can do it even better and combine both approaches

This change will put a lot of work on library authors of existing libraries

…in the sense of, if they want to release their libraries against the scala version including this change, then they need to go through all libraries and have a look at each class and each member and re-decide if it should be open-ed up again or stay closed.

Personally I don’t think so, the migration path is smooth:

  • if you don’t want to spend time (which is fair enough), just add open to existing classes. This can easily be done with regex search and replace. Same same for adding @memberExtensibility("open") to classes and traits
  • you can (but don’t have to) step by step transition away from using open to closed or final at your own pace.

This will make mocking difficult

One can use adhocExtensions globally for test sources as already advertised in the documentation.

In case we should go with final-first, then IMO it is not really difficult but less easy. IMO one should not mock a class but its abstract class or trait. And in case the implementation which one wants to replace in the mock is in the concrete class then it is easy to use scala 3’s export capabilities (I guess this feature did not yet exist when the discussion about final by default was held). If there isn’t any contract (abstract class or trait) but just the concrete class and one does not own the code, then one is out of luck. Personally, I cannot remember when I had the need to mock a concrete class without contract.

This will make it difficult to patch-up a library in case of a bug

I think more or less the same arguments as for mocking applies.

… also if we should go with a final-first approach.

If you use a concrete class of a library and you can patch it up currently and pass the sub-class instead of the library class to some method, then in all cases where I had this desire, the method does not expect the concrete class but an abstract class or trait. i.e. patching up just means implement the abstract class or trait use the concrete class as composite and delegate everything (with scala 3’s export mechanism) to the concrete class except the methods you want to patch.

but I need to patch a protected member of the concrete class

If we go with the closed-first approach, then you can still use the adhocExtenions feature. If we go with the final-first thought, then you are out of luck. But honestly, if someone defines a method protected in a concrete class then this person will most likely use open or closed for this method as well once this change is implemented.

I still want some mechanism to monkey patch final classes of libraries

Fair enough :wink:

I am new to scala 3 so might be I don’t have the full picture. I guess it should be possible to re-compile a library based on the tasty byte code included in the jar of the library. If a tool (lets call it “monkey patcher”) allows to modify the tasty byte code in such a way that one can modify the extensibility modifiers before compilation, then this would be an alternative way which would still allow that one can patch up also a final class (of course you loose things like singed jars and the monkey patcher would need to be crafted carefully not to become an own source for security holes). Btw. I think a monkey patcher could also be interesting for the opposite way: kind of harden a dependency, set all classes/members to final with some exceptions (the ones we want to override).

but the monkey patcher does not exist yet

Indeed, and I guess I won’t have the time to implement it alongside with this SIP. As a middle ground between final-first and closed-first, we could introduce a 4th extensibility state. If one omits an extensibility modifier, then it is quasi-final. The class as such will behave like closed (i.e. one can still extend it via adhocExtensions) but it is basically syntactic sugar for creating two classes under the hood:

  • abstract class => this class will contain all definitions and will be used everywhere the class is used as type (so that passing another sub-class works)
  • final class => this class is empty (final class Xy extends _BASE_Xy) is used to instantiate the class

E.g. if one sub-classes a quasi-final class via class Xy extends Foo then it actually sub-classes the abstract-class. For the scala user, this should be transparent (setting reflection aside).

I am sure this is not well thought through and certainly a pain to implement but maybe gives someone else a hint for even a better idea :slight_smile:

Why closed and not sealed?

See previous discussion SIP: Make classes sealed by default . I guess the core team has dropped the idea mainly because mocking and patching isn’t easy, but a core team member would have to respond to this question.

Since that discussion, Java’s sealed classes/interfaces have landed as full feature in jdk 17 and I don’t know if there are plans to map Scala’s sealed classes to Java’s on a byte code level. Java’s sealed classes are final and a sealed class is always abstract in contrast to scala where we can have a concrete sealed class.

Why closed as default on member level

There are several points which speak for this IMO:

  • basically the same arguments as for classes apply for members. If you open up a class for extension via adhocExtensions then you most likely do this for some specific reason (setting mocking aside) and you don’t want that someone (including yourself) in the future override stuff you did not intend to extend adhoc without a warning.
  • it makes sense to have a symmetry between class and member modifiers as it is easier to remember/learn
  • not being able to state that a member is open even though it implicitly is indeed, make some people nervous

Future improvements

Possible improvements for the future (not yet thought through):

  • allow to widen closed to closed[package] as it is very common in a project making use of jvm modules to put the concrete implementation in an sub-package impl so that one does not need to expose it via module-info. For instance, put an abstract class in package com.example. this abstract class defines a default implementation marked with closed[com.example.impl] and in the impl package there are two implementations where one of them overrides the method. This override would not need the adhocExtensions feature in scope due to the package definition in closed.
  • allow to widen the closed (and sealed could follow) with a permits clause similar to Java’s sealed class allowing that certain classes outside of the file and even in a different package hierarchy are allowed to extend the class without the need to use adhocExtenions
  • I see Implement keyword (instead of override) as a good supplement including emitting an error if one overrides without override modifier.

There is currently a warning for ad hoc extensions but you still need source future. 3.1 implemented very few of the things we planned for the future, so source future is still relevant.

Just as info: I don’t get a warning also on 3.1.1 with -source:future.

Looking forward to feedback on the proposed changes

For some reason, sbt doesn’t report that there was a feature warning; you must ask for it.

For good measure, also -Werror.

scalacOptions ++= Seq("-source:future", "-feature", "-Werror")

Edit: for some reason, it took me 20 minutes to create an issue.

Note that it works normally from the command line. I didn’t try scala-cli.


Thanks for the hint. I tried out -feature as well at some point. Turns out using sbt -client does not show feature warnings (created an issue => my bad, I figured connecting a client to an existing server without client would reload automatically if necessary as the usual warnings were not there).

Thoughts about the proposed additions/changes?

Hm… I thought there would be at least one negative voice regarding my proposed addition/changes but I guess it’s just not of interest at all. In case I have not make it clear, I am willing to implement all missing pieces but I first want some green light from the compiler team as I don’t intend to just waste my time if in the end the answer is along the line of “not interested”.

What is the further process of a PRE-SIP?

The Scala Center is currently working on rebooting the SIP process. There were multiple meetings about it this week and last, so gears are turning. Not sure if there’s a target date. There will probably be a new committee with some of the same people and some new ones.

I did see your proposal, and I didn’t reply because I feel like Scala 3 just landed a huge amount of language change that the community is still absorbing and although your proposal seems interesting and well thought-out and presented, my gut feeling is that the status quo on the whole open/closed area is fine for now. I just can’t work myself up into seeing it as a pressing issue where change is truly needed. This is consistent with my response in 2019/20 to Martin’s proposal to add open to Scala 3, which I opposed on similar grounds. It’s not necessarily strong or permanent opposition, just “I don’t see why it’s important to tackle this now” type mild opposition.

It’s hard to interpret silence but perhaps the near-silence with which this was greeted here is because others have similar feelings.


A compromise to the open/closed principle is to leave classes “ajar”.

That is a suitable pun on “jar” files.

The semantics might be: only people who know what they’re doing can contribute extensions to the jar.

That would be effectively “sealed with respect to the jar file”.

I don’t think this proposal is going to get through. For the most part, the arguments that are put forward were already well known, examined and taken into account by the SIP Committee when Scala 3’s open classes were designed. I don’t see anything here that would invalidate the reasoning that was done back then, and hence that would change the outcome.

I am going to comment on a few specific aspects.

closed versus no modifier

What is proposed here for closed classes is essentially what is already available in Scala 3 with -source:future -feature for classes without any modifier. The proposal also states that no-modifier would be equivalent to closed.

There does not seem to be any incentive to explicitly mark closed something that is already closed by default. This is especially true given the original motivation for open classes: that most of the time, we don’t really think about the extension contract of a class, but we also don’t make it final. The motivation for open classes, and closedness by default, is precisely the lack of deliberate decision about a class’ extension contract. Explicitly marking a class as closed would … deliberately choose not to make a decision. It doesn’t make sense.

If you actually make a decision, the only two meaningful possibilities are open or sealed (with final being only a special-case of sealed which does not really make a difference from an outside API perspective). With sealed, you lock it down to yourself and don’t provide any extension contract. With open, you must think about your extension contract.

Open/closed/final for members

I start again from the reasoning about the lack of deliberate decision. We have 3 possibilities for a class:

  • Intentional decision to be sealed/final. In that case, it doesn’t matter what level of openness the members have; they will never be overridable by third-party code.
  • Intentional decision to be open. In that case, you must be carefully planning your extension contract. The nature of an extension contract is such that all members must be fully specified wrt. how they interplay with each other in the context of an extension. A closed member within an open class therefore makes no sense, as it would invalidate the very possibility of the open class to justify its own extension contract. There remain only final and open as possibilities, and we already have that.
  • Lack of deliberate decision. In that case, if you’re not making a decision about your class as a whole, what would it matter that you make some decision about any specific member. Marking a member as open within a closed class would be misleading at best, since you could not guarantee anything when the other members are going to call that member or not. So again, the only real choices are closed and final, and that is what we already have for the members of a closed class.

Final by default

Here, as you identified, this was debated quite a lot, and in the end closed by default won because we couldn’t get enough people on board with final by default. I don’t see anything here that would alter the decision if it had to be done again today.

The only possible thing is the security aspect. But that means you’re in a context where there is code that has been compromised. That’s bad, very bad. I don’t think making a few things final will protect anything in a threat model where there is some compromised code in the system. You’ll have much bigger problems. You would really have to bring up a concrete threat model and accompanying examples of attacks that would be prevented by final to use security as a real argument for final by default.

1 Like

Thanks for your feedback :slight_smile:

As I wrote, this allows to re-interpret the omitted modifier differently. But I agree, if we don’t want that omitting a modifier can also mean final by default then it is the same and we don’t necessarily need closed as class modifier but it still handy (see next answer)

I agree that it does not make sense if you only think in two categories but there is a third. If I, as library author, would make a class final but know that there are people out there which still want to override at their own risk etc. then closed is a meaningful choice. And making it explicit shows to consumers that a choice was made where omitting doesn’t tell anything.

Basically the same arguments as above apply IMO. Turn a few final members within an open class into closed members is currently not possible. This SIP would address this.
Also, open in a closed class (even if the modifier is omitted) makes perfect sense if you want to state that this member is designed to be overridden (e.g. a default-implementation). I really don’t see how this could be confusing, can you give an example? With the current implementation, one cannot open up a member within a closed class without requiring that the consumer need the adhoc extension etc. and a consumer should not need this by default

I am not at all convinced by this third category. What other language feature was ever designed to support a case of “I, as library author, would like my API to look like A and be specified as is, but still let people who want to abuse it like B which is conflicting with A”?

Same argument as above for me as well: I am absolutely not convinced by this third category.

Let’s say you have the following definition:

closed class A {
  def foo(args...): Foo = ???
  open def bar(args...): Bar = ???

A is closed, so you don’t provide an extension contract for it. foo is closed by transitivity, so you also don’t provide an extension contract for it. Now bar is open, so you have to provide an extension contract for it. How do you do that? You can’t, because you have no way of talking about the behavior of foo wrt. bar: does the behavior of foo depend on bar or not, and if yes, how? You don’t know, and you can’t know, because foo is not subject to an extension contract.

So in general, it makes no sense to talk about the extension contract of a method (and hence to declare it open) without talking about the extension contract of all the other methods of the class. And once you do talk about the extension contract of all the other methods of the class, then the class can be open itself.

The feature as such is already in the language on the class level (currently behind a feature flag). We are basically only talking about whether we want to allow that developers kind of think in three categories or not via a keyword or not. closed on the class level is not a must in this sense, the feature is already there. But, IMO it would be more consistent that folks can express their intention via a keyword and not only via e.g. documentation.

I guess you think too much in side effects and depending method calls. If your method is side effect free, then you can mark it as open without problems as it does not matter in what order you call it. To come up with an example, you might have a method which e.g. merely creates another class (protected factory method) and the sub-class can override to choose a different implementation. This method is open where other methods within this closed class remain closed as they are not intended to be overridden.
I already hear you “but in such a case the class should be open” and I totally agree. An open member in a closed class should be forbidden, you’re right. If one intends that a member should be overridable then the class as such needs to be extendable.

I realise know, that there is a misconception of my proposal. The class extensibility modifier has no impact on the member modifier. I suggest to treat them separately. I.e. an omitted extensibility modifier is always equivalent to using closed explicitly. In your example foo is closed because the extensibility modifier is omitted and not because of transitivity.
(as side question, just because I am curious: in java, the same applies to final in the sense of non-final members in a final class are not marked final in the resulting jvm byte code they are only quasi final. Is scala different here?)

In your first answer you wrote:

I basically agree with most of the outcome, I propose (and I see this as an improvement) to go a step further. As you wrote

The same applies to members. It’s not like developers would make deliberate decisions on the member level if they do not on the class level. I give an example. Let’s say we have the following in the code base

class A {
  def foo(args...): Foo = ???
  def bar(args..): String = "implementation"

Some months later some developer writes (in the same code base)

class B extends A {
  def bar(args...): String = "other implementation"

=> compile warning

Unless class A is declared ‘open’, its extension in a separate file should be enabled by adding the import clause ‘import scala.language.adhocExtensions’ or by setting the compiler option -language:adhocExtensions. See the Scala docs for value scala.language.adhocExtensions for a discussion why the feature should be explicitly enabled.

I propose that we turn this warning into an error. Taking into account that we deal with lack of deliberate decision this warning might just be ignored. A compilation error on the other hand cannot be ignored and the developer needs to make a first decision. Use open or adhocExtensions.

Let’s say the developer decides to use open

open class A {
  def foo(args...): Foo = ???
  def bar(args..): String = "only implementation"

=> still a compilaton error as bar is closed

open class A {
  def foo(args...): Foo = ???
  open def bar(args..): String = "only implementation"

foo remains closed, if someone tries to override it then we emit a compilation error again and force the developer to think a bit more. Which means, even if bar is not side effect free or requires a certain order in which other methods are called within the method, then this can be documented as part of the extension contract of bar. But there is no need to have an extension contract for foo as this method is still not intended to be overriden. IMO a great improvement compared to the current implementation and well aligned with the idea behind it.

Let’s say the developer decided to use import scala.language.adhocExtensions. In the current implementation, the warning/error would be gone. I propose that the compilation error is gone but that we emit two different warnings instead, one for the class extension and one per overridden member. Why still a warning? Because the risk is high enough and developers should think twice, so IMO it deserves a warning. If one really does not care then one can suppress the two warnings globally

import scala.language.adhocExtensions

class B extends A {

  def bar(args...): String = "other implementation"

And again, foo remains closed.

I am not talking about side effects. Let’s take your example:

class SomeData(val x: Int)

open class Parent {
  /** Computes `x + 1`. */
  closed def computeSomething(x: Int): Int = {
    val data = factory(x)
    data.x + 1

  open def factory(x: Int): SomeData = new SomeData(x)

class Child extends Factory {
  override def factory(x: Int): SomeData = new SomeData(2 * x)

println(new Parent().computeSomething(5)) // prints 6
println(new Child().computeSomething(5)) // prints 11 !

As you can see, even though there is no side effect here, the behavior of computeSomething depends on the behavior of factory. Therefore, one must specify the result of computeSomething in terms of factory, and that becomes part of the extension contract of the class.

Once again, it makes no sense to have both open and closed members in the same class, because just one open member demands an extension contract for the entire class, and hence for all its members.

And that’s perfectly fine, if the result of factory should be a SomeData with x = 5 then it should have been part of the extension contract of factory. If not, then the doc of computeSomething needs to take factory into account. I really don’t see a problem here, the one who openend factory is to blame for not updating the doc of computeSomething or the one who wrote computeSomething if factory was already open by that time.

What you totally miss though. computeSomething can still not be overriden in the subclass, so there is no need to define an extension contract for computeSomething. It could even contain nasty side effects, calls which require a certain order etc. etc. doesn’t matter, because a subclass cannot change it. That’s a big advantage IMO and quite a difference to the current implementation where once you open a class, everything is open. As already outlined in the previous post, it makes perfect sense to have closed and open members in the same class. In the end closed is a light version of final. And it is not unusual to see final members in an open class (e.g. abstract class). For instance, it is standard to use final in the context of a template pattern where at least some members are open as well.

Any thoughts about:

  • turning the warning into a compile error?
  • emitting two different warnings for overriding a class and overriding a member?