Pre-SIP: Support java.util.concurrent.atomic.Atomic*FieldUpdaters

Currently it is impossible to use to juc.atomic.AtomicReferenceFieldUpdater et al. with Scala classes, as even @volatile var members are wrapped with getters + setters (the actual field has the name mangled and is set to private).
This has led to the unfortunate pattern of creating dummy Java base-classes for high-performance concurrent code, even within the Scala standard library (cf. for example scala.collection.concurrent.CNodeBase).

I propose a minimally intrusive change to support inline var as syntax for telling the compiler to leave a variable unwrapped + unmangled. A demo implementation is available here: https://github.com/lampepfl/dotty/compare/master...elfprince13:support-java-util-concurrent-atomic-updaters?expand=1

It would also be possible to use a new annotation instead of inline, but this would require modifying the standard library as well as the compiler, and I believe that ā€œdo not generate a wrapper method for this variableā€ is fairly close to the spirit of inline.

2 Likes

Iā€™d also like to understand why Scala generates getters and setters for vars in the first place. Iā€™m sure there are good reasons but it seems a bit unnecessary to me.

For public and protected vars, because of ABI stability. You can change from a var to a pair of def getter and setter, and conversely, without breaking the ABI and therefore without breaking binary compatibility.

For private vars, there is no other explanation than: it has to do it for public and protected, so it is easier for the compiler to also do it for private, as it removes special cases in the compiler. Fewer special cases ā†’ fewer bugs.

4 Likes

Iā€™m not convinced. We do not emit any bytecode at all for an inline val (it only shows up in .tasty), whereas here the explicit intent is to emit some specific bytecode, so a @publicField annotation seems more appropriate to me (and as a bonus, users can just read the documentation of the annotation to figure out its precise semantics).

Implementation-wise, Iā€™m impressed that just setting JavaDefined works :), but Iā€™d be wary of keeping that since thereā€™s many code paths in the compiler specific to JavaDefined symbols, so a big potential for unforeseen interactions.

Finally, itā€™d be good to consider how this would affect other backends: can we give a useful meaning to @publicField var in a scala.js class for example?

2 Likes

In addition, vals need to be getters because they can be overridden. Emitting getters and setters for vars is more consistent.

2 Likes

Also - even with private vars, thereā€™s a useful idiom for mutable recursive data-structures of passing getters and setters as function arguments to emulate a C-style ā€œpointerā€ to your own parentā€™s field containing you. I actually made an issue in the other direction from this proposal about a year ago, because Scala 3 changes to private broke compatibility with some code I was porting over that relied on member_= for private vars.

If we go the annotation route, @bareField would be more appropriate, I suspect, since in most cases these arenā€™t going to be public fields (only needs to be accessible from the companion object).

Anyway, my gut reaction is that needing to modify the standard library to add an annotation that triggers special compiler behavior is icky, but I realize thatā€™s how a bunch of other annotations also work, so ĀÆ_(惄)_/ĀÆ

Fair enough. I was nervous that if I just turned off emitting of getters and setters without setting JavaDefined, there would be code elsewhere assuming the presence of getters and setters, but thinking about it further, this is presumably not the case since PrivateLocal has similar behavior.

The Scala.js IR has a notion of field, and a notion of method. All fields are public in the IR, even when they are private in JVM bytecode. @publicField would have the same effect as on the JVM, except it wouldnā€™t change the fact that the field itself is already public, but that has no consequence.

Cleaning up old repos recently, I found my first Scala code from ten years ago. It used XML and Swing, for some reason.

You remind me that I kept learning (or obfuscating) Scala and posted a StackOverflow Q&A on AtomicReferenceFieldUpdater. It says I was reading twitter util. The answer says to use private[this] and @inline as you propose, and cope with mangling.

It looks like I was impressed that you could define a like-named accessor with parens and invoke it without parens. Those were the good old days.

Iā€™ve definitely stumbled on that SO thread more than once thinking there had to be a way to do it.
Anyhow, Iā€™m not sure if the OpenJDK implementation changed or if Scalaā€™s implementation of companion objects changed in some important way, but the reflection that powers the Atomic*FieldUpdaters checks to make sure the code creating them them would have permission to access the field the normal way, and doesnā€™t recognize companion objects as having access to private fields.

The answer says the compiler loosens access for you, which scalac does for many things that are nominally private. (See the tweak below.) (was: I havenā€™t tried that code recently.)

Edit, this works, with semicolons left over from the original java:

package scala.collection.concurrent;

import java.util.concurrent.atomic.AtomicIntegerFieldUpdater;

object CNodeBase {
  private final val name = "scala$collection$concurrent$CNodeBase$$csize"
  private val updater = AtomicIntegerFieldUpdater.newUpdater(classOf[CNodeBase[_, _]], name)
}

abstract class CNodeBase[K, V] extends MainNode[K, V] {
  import CNodeBase.updater

  @volatile private[this] var csize = -1;

  @inline private final def tweak() = csize

  def CAS_SIZE(oldval: Int, nval: Int) = updater.compareAndSet(this, oldval, nval);

  def WRITE_SIZE(nval: Int) = updater.set(this, nval);

  def READ_SIZE() = updater.get(this);
}

Edit: regarding my previous self-deprecating joke about the like-named accessor, in the SO answer, a method had a reference parameter used to access the field. If tweak were named csize, the following works because the field is not visible and ā€œempty applicationā€ supplies the parentheses:

def f(cnode: CNodeBase) = cnode.csize

You shouldnā€™t rely on this, the compiler is free to choose any name-mangling scheme it wants for things which are not part of the public API.

Not saying itā€™s a good idea even modulo mangling.

Also, Scala 3 inline does not widen access to the field, but generates an accessor inline$csize, so the scheme breaks down. Not sure if that is a feature, as it is not inlined. Same for access to private field from companion, but accessor is named like mangled field in Scala 2.

I was expecting something "fairly close to the spirit of inline" as the OP puts it.

Somehow your snippet quote lost a dollar.

So, uh, what are next steps here?

1 Like

According to the SIP submission docs, it sounds like @darja is supposed to advise on next steps?

1 Like

Sorry to say this, but the SIP process is pretty much dead. If you want to advance anything, implement it under the experimental flag and create a PR for the dotty repo.

Instead of AxFU-support Iā€™d prefer something like an @atomic annotation on vars which would then compile down to VarHandles or AxFUs based on target JDK-version and expose atomic methos on the variable.

7 Likes

Hi everyone!

The absence of an ability to say to scalac ā€œemit unmangled public fieldā€ is what is blocking me from using VarHadles.
I will be really glad to see such a feature.

AFAIK, Scala makes fields only private to help maintain ABI compatibility.
So, IMO, annotation with a name like @ABIUnsafe or @BinaryUnstable will have more sense.

Also, I believe that SIP processes have already been restarted.
Therefore, this proposal can be brought back to life.

Glad to see continued interest in this proposal! Hoping someone picks it back up!