Extending Scala with modern concurrency primitives

Scala Platform
1

This thread is part of the projects covered by Sovereign Tech Fund’s investment in Scala.

For the full context, please read the announcement blog post.

On this forum, each project supported through this investment has its own dedicated thread.

This thread covers the work package Modernization and extension of the Standard Library/Core Library Modules and will be used to share the project overview, a roadmap with key milestones, ongoing progress updates, and opportunities to engage—so we can hear ideas from the community and encourage contributions.

This project covers an investment to support direct-style asynchronous programming in Scala, based on new primitives for structured concurrency including task groups, cross-task communication, and cancellation.

An important goal of the work is cross-platform availability, including developing APIs for asynchronous I/O operations on devices such as files and sockets.

Eventually there should be a framework for high level streaming API built on top of these primitives.

The work will either be integrated with the standard library, or in a separate library and likely included with the Scala Toolkit.

Current Team

Jamie Thompson (lead), Nguyen Pham

Communication

This thread is for updating the community on progress that has been made; and to seek feedback from the community on requirements for a useful library, and discussion on design proposals.

15 Likes
2

Would this work include some implementation of continuations/coroutines?

3

Scala Native already has a continuation API, JDK’s own raw continutation API is not accessible without --add-opens=java.base/jdk.internal.vm=ALL-UNNAMED - although can be simulated with virtual threads. Scala.js not unless using the JavaScript Promise Integration for WebAssembly.

the idea is to put a higher level abstraction over these

1 Like
4

Hmm, but there has been discussions on implementing some sort of compile time transformations like what Kotlin does with coroutines, right? Or has that idea been ditched? Otherwise it sounds like something that would be very useful with regards to the cross platform effort of this endeavour.

5

I hope this is being designed in a way that can potentially integrate with the existing stacks? Out in the enterprise world we have massive installed bases using mature stacks like cats-effect and ZIO, which have highly-optimized engines by now.

An architecture that would allow interoperation with these stacks, so that we could have common async / await syntax when desired and use the full power of our chosen stacks when that is insufficient (more complex concurrency situations) would probably be a big win.

(Whereas something completely decoupled would probably just foment further ecosystem fragmentation – probably a net loss IMO.)

10 Likes
6

That was ditched. With the exception of non-Wasm JS, all platforms now have native support, in one way or another, for couroutining. That leads to implementations that need no allocations of closures or other wrappers on top of what you write in the source code.

7

This is great! I’m a big fan of direct style. I’ve created a lot of my own tooling to support it, both in cases where it is essentially equivalent to a monad and cases where it isn’t.

It gets really interesting, for me at least, when it isn’t just about async, but general-purpose direct style control flow.

val text = Fu:
  val lines = p.slurp().?
  val metadata = Fu:
    loadMetadata().?
  process(lines, metadata.ask().?).?

This creates a future (using Loom threads), in which it creates an inner future, handles three failure-prone operations safely, and in just a handful of lines. To me, it reads better than the equivalent IO for loop, is easier to understand, is more efficient (I’ve benchmarked), and is more flexible.

Ox also has some nice direct-style features. I would probably just use Ox (wider adoption) if I hadn’t already created mine first. One could pick various default features or different implementation details. It’s very direct. You focus on doing what you want to do, with the ceremony of various boundaries and error handling and such staying visible but tidy.

Anyway, the point is, it gets really nice once you have control flow, error handling, and async all working together seamlessly. One still has to pay attention to thread boundaries, but possibly with capabilities Scala will help check for that.

In the meantime, it would be really nice if one could create opaque subtypes of Label that were optimized the same way as label but with more type parameters or somesuch. This allows a corral system where control flow constructs cannot jump through thread boundaries. I implemented one, but alas, it didn’t optimize to direct jumps when it could. (But it was nice in that you could use it to make sure your control constructs have to stay within the future you create them in.)

2 Likes
8

I hope prior art and research on this topic - refined in cats-effect and ZIO over many years - will be considered when designing this new system. In particular, relying on Java thread interrupts / InterruptedExpection for cancellation simply does not work - an interruption signal has to be parallel, not based on exceptions and not recoverable. Otherwise a cancellation system becomes too brittle to be useful.

7 Likes
9

for integration with java libraries, support for thread.interrupt is a must, imo. what else can we do to promptly end some operation and release blocked resources?

10

Few java libraries properly handle thread interrupts either. Both cats-effect and ZIO do support sending thread interrupts to blocking I/O operations - but on a strictly opt-in basis.

2 Likes
11

there are tons of java libraries. if some library doesn’t properly handle thread interruption then probably some alternative library does. with virtual threads supporting thread.interrupt i guess such support in third party java libraries will grow. virtual threads supporting thread.interrupt and many other functionalities from original java threads is also a reason why java standard library (providing various i/o primitives) won’t add much ‘reactive’ api, but instead will focus on synchronous style with interrupts.

what are the alternatives to thread.interrupt when calling java code (from java stdlib or third party libs) and wanting immediate stop to an ongoing operation for quick release of all resources associated with that operation? regular interrupt from zio waits for a blocking operation to stop, which can take ‘forever’ if the underlying java library waits for either i/o response or thread.interrupt and neither is coming. of course the control flow can progress using other threads (i.e. we can fork in zio or disconnect the control flow by other means), but putting stuck blocking operations to background and ignoring them doesn’t magically solve resource exhaustion problem.

12

Great news! Is there any rough plan or details about this direction? Also is it related to the gears library or a totally different thing? Note gears project has no update for 4 months.

1 Like
13

The alternative is to use thread.interrupt, but only when there are no other options, which ZIO supports via ZIO.attemptBlockingInterrupt. But – this is an opt-in operation, for large synchronous blocks of code. ZIO itself does not use thread interrupts for signaling - they’re extremely unreliable - they can be caught, they can be recovered, they can be swallowed accidentally or deliberately (due to that they’re anti-modular), they can be faked by throwing InterruptedException without actually setting the interrupt flag, they can be set and unset from within and without the thread itself. Also, to work around the unreliability of Java thread interrupts, ZIO.attemptBlockingInterrupt keeps sending Thread.interrupt to the target thread forever in a loop - just to make sure it actually unwinds. Interrupts and exceptions should be separate, we’ve spent decades fighting the mistake of merging them (a mistake which isn’t unique to JVM btw, Haskell made the same mistake with AsyncException - but in Scala we actually solved it in our effect systems, because they’re implemented as libraries, while Haskell’s runtime is bolted on and can’t be fixed).

8 Likes
14

As someone currently working on a large codebase in java with virtual threads, I cannot stressed enough how broken thread interrupt is and how unreliably useless it is, so thanks @kai for bringing it up.
Java moving to sync apis is a mistake given that they never fixed interrupted exceptions.

3 Likes
15

I’ve been working on YAES (GitHub - rcardin/yaes: An experimental effect system in Scala 3 that tracks effects through context functions) for a year now. It’s a Scala direct-style effect system based on Java virtual threads, which means it supports one-shot runtime continuations.

The Async effect was there since the beginning, and Channels and Flows were added a few months ago.

Ox by Softwaremill has similar concepts.

@raulraja proposed a similar change to the language called and made a PoC in a library called Unwrapped (GitHub - xebia-functional/Unwrapped: Unwrapped is an effects library for Scala 3 that introduces structured concurrency and an abilities system to describe pure functions and programs.).

Just to say, we’re already trying to move in this direction. If you need some hints or want to share some experience, just drop a message :smiling_face:

8 Likes
16

Thanks @rcardin , I should say that we are not going totally our own way, and of course community feedback, collaboration, and advice is much welcomed - we wouldnt want to have a blind spot

1 Like
17

I, too, hope that authors will look at prior art, such as Cats-Effect, ZIO, but also … Kotlin’s Coroutines.

Java’s interruption is a mixed bag, but it kind of works, and it’s getting better, because part of bringing Virtual Threads to the JVM, they’ve also made blocking socket I/O interruptible. And because of the work being done to bring “structured concurrency” utilities to the standard library, there’s this push in the ecosystem to make interruption work. For instance, I’ve noticed JDBC drivers modified to work well with virtual threads and with Thread.interrupt.

That many people catch InterruptedException is a reality. It’s bad that it’s a checked exception, so people end up catching it and ignoring it. But at least you recognize the well-grown functions in Java when they expose InterruptedException in the API. To tell you the truth, I think Java libraries are better behaved, in regard to interruption, than Scala libraries that are NOT based on Cats-Effect or ZIO, i.e., anything that came before, based on Future. Whenever I see a Scala API using Future, I search for the Java equivalent.

Speaking of exceptions, unfortunately, I don’t think they can avoid it. What they are proposing is to make use of the platform’s own call-stack. How Cats-Effect and ZIO work in regard to interruption is that they treat interruption on its own separate channel, so like a different kind of exception that can blow up the stack and that can’t be caught and ignored. I think Cats-Effect and ZIO have been doing great, but Scala is a JVM + JS language, and it’s cursed (or blessed) to be multi-paradigm, and it can’t afford to ignore the underlying platform. For example, I’d like to work on some libraries that do I/O, but I want maximum reach, and I’d rather work at a lower level, than bringing in heavy dependencies, with their own run-loops, thread-pools and way of dealing with exceptions.

An improvement to using exceptions in cancellation is what Kotlin has been doing… the way interruption manifests in regular code (in a try/catch/finally) is via a CancellationException, an exception that can still be caught. What’s good about it is that you still can’t uncancel a cancelled coroutine, so you can ignore CancellationException only until the next asynchronous boundary. And because in Kotlin calls to suspended functions are everywhere, ignoring cancellation is short-lived. I think this way of dealing with cancellation provides a compromise, because you can still work with the platform’s call-stack and use the language’s official constructs, such as try/catch/finally (no need for new DSLs to treat exceptions or finalizers).

5 Likes
18

I’ve been trying to one-up Kotlin coroutines (and Goroutines) for a while now. It’s architecturally tricky, but should be possible for the usual case of repeated channel access. One ought to be able to write brilliantly short, high-performance, automatically cancellable code (with the usual caveat that if you call a library that eats InterruptedException, you never have any way to see it).

If I ever get it working, I’ll try to mention it here. But it’s going to be JVM 21+ at least, so it’s probably not relevant when we’re targeting 17. And I’m not sure if it would port to JS in a way that had adequate performance–a lot of it is getting park statements in the right places, which obviously is very in the JVM weeds.

Incidentally, I also have discarded Scala’s Future in favor of an opaque-type-wrapped Java Future with Loom threads. (The virtual threads are the main draw–I can fire off an absurd number of futures and not worry about it.)

1 Like
19

Jamie would have to confirm/disconfirm, but my understanding is that the concurrency part of the Sovereign work won’t necessarily go in stdlib, which means it would be free to have a higher minimum JDK version, such as 21 (or even 25, if there’s some further benefit?).

Given the important of Project Loom in this space, I personally think it would make sense to target a minimum version of at least 21.

3 Likes
20

It’s better to target at least JDK 24 to avoid thread pinning :winking_face_with_tongue:

1 Like