Impact of Loom on “functional effects”

[mdedetrich]

@tarsa

Loom scheduler is over 4x times faster than the speed of Linux kernel switching threads. Alibaba reported over 10x speedup in their solution, so there’s a room for improvement (for Loom in the future). But again Loom already gives you scheduler that is over 4x faster than native one from kernel (in this benchmark at least). What’s not to like?

I am not trying to compare Loom threads to native OS threads, I am comparing Loom threads to other potential implementations of green threads in JVM. For me Loom VirtualThread’s being 4x faster than native threads in IO bound/high context switch scenarios is a moot point (I mean if VirtualThread’s wasn’t faster then something is seriously wrong).

Ultimately though, my 2 cents and prediction is that Loom threads will have close to zero impact in the Scala ecosystem apart from being an alternative backend to some IO types (and possibly the default depending on how good Loom turns out). Its already possible to create extremely performant IO bound services as is evidenced in the link I posted earlier about akka’s gRPC module. Put differently Loom is primarily solving Java’s problems.

@adamw Actually posted a great article on medium at Will Project Loom obliterate Java Futures? | by Adam Warski | SoftwareMill Tech Blog and I am quoting the relevant text from Daniel Spiewak in that article

In the performance tests that I’ve done, yes, Loom is a lot faster than a naive fiber scheduler based on opaque thread pooling, but you can reap massive performance benefits (often around an order of magnitude) in fiber libraries by making the thread scheduler fiber-aware. Loom actually defeats this optimization because it forces you into a generic executor-style interface. So Loom’s “native mechanism” is actually a lot slower than what can be achieved in higher level frameworks specifically because Loom is too low level to take advantage of what the higher level frameworks know about access patterns.

So at worst it may just end up being that for the current IO types that we are familiar with in Scala, Loom will be a downgrade.

[tarsa]

@mdedetrich

Ultimately though, my 2 cents and prediction is that Loom threads will have close to zero impact in the Scala ecosystem apart from being an alternative backend to some IO types (and possibly the default depending on how good Loom turns out). Its already possible to create extremely performant IO bound services as is evidenced in the link I posted earlier about akka’s gRPC module. Put differently Loom is primarily solving Java’s problems.

If you want to stick to IO monads then you won’t see a profound impact on your code (although Loom should still make your life easier by automatically, efficiently and reliably handling blocking that you forgot to handle explicitly). But I’m looking forward to Loom to embrace massively scalable imperative synchronous style so I will be able to freely mix control structures, higher-order functions, parallelism, awaits, nesting, etc without any need for specialized monadic combinators or any other sort of mental gymnastics and/or ugly syntax needed by monadic composition. However, @jdegoes said he will prove me wrong (well, of course he won’t focus on me, but on the comparison of effect systems ) and show how functional effects are easier.

@mdedetrich

text from Daniel Spiewak in that article

In the performance tests that I’ve done, yes, Loom is a lot faster than a naive fiber scheduler based on opaque thread pooling, but you can reap massive performance benefits (often around an order of magnitude) in fiber libraries by making the thread scheduler fiber-aware. Loom actually defeats this optimization because it forces you into a generic executor-style interface. So Loom’s “native mechanism” is actually a lot slower than what can be achieved in higher level frameworks specifically because Loom is too low level to take advantage of what the higher level frameworks know about access patterns.

So at worst it may just end up being that for the current IO types that we are familiar with in Scala, Loom will be a downgrade.

Loom should provide pluggable schedulers in the future. They seem to be missing in current preview in JDK 19 EA, but they are present in Loom design documents:
https://cr.openjdk.java.net/~rpressler/loom/loom/sol1_part1.html

…
Loom adds the ability to control execution, suspending and resuming it, by reifying its state not as an OS resource, but as a Java object known to the VM, and under the direct control of the Java runtime. Java objects securely and efficiently model all sorts of state machines and data structures, and so are well suited to model execution, too. The Java runtime knows how Java code makes use of the stack, so it can represent execution state more compactly. Direct control over execution also lets us pick schedulers — ordinary Java schedulers — that are better-tailored to our workload; in fact, we can use pluggable custom schedulers. Thus, the Java runtime’s superior insight into Java code allows us to shrink the cost of threads.
…
Virtual threads can use an arbitrary, pluggable scheduler. A custom scheduler can be set on a per-thread basis, like so:

Thread t = Thread.builder().virtual(scheduler).build();

or per-factory, like so:

ThreadFactory tf = Thread.builder().virtual(scheduler).factory();

The thread is assigned to the scheduler from birth till death.

Custom schedulers can use various scheduling algorithms, and can even choose to schedule their virtual threads onto a particular single carrier thread or a set of them (although, if a scheduler only employs one worker it is more vulnerable to pinning).

Probably the API of custom schedulers in Loom requires some serious rework, but from what I’ve seen, every time users asked for ability to define custom schedulers, Loom authors were saying that’s in the plans.

[jdegoes]

@odersky

Of course there are other sources of blocking than Futures. And of course Futures are used for other things than RPC. That has never need disputed.

I think it is important to be precise. Remote procedure call (RPC) tracking is radically different than anything proposed in the pre-SIP, where nearly all discussion focused on tracking async suspension (completely ignoring sync suspension, which is wrong at many levels). After all of this discussion, I think it should be clear that tracking async suspension is not a goal, it is not coherent in a post Loom world, and it is not strongly correlated with RPC.

I think if the conversation shifted to “tracking RPC” (rather than suspension, and particularly and arbitrarily, “async suspension”), then it would be more productive. Unfortunately, these threads reveal great confusion among Scala developers on everything from asynchronous programming, concurrent programming, platform threads, virtual threads, Loom, and the design, implementation, purpose, benefits, and drawbacks of Future, which is contributing to the generally unproductive dialogue.

@odersky

All I was ever saying is that if we want to distinguish local calls from crossing the internet in an RPC, we need some way to represent “waiting/suspending/taking an unbounded time to finish” in the type system. Just allowing arbitrary Future[T] -> T operations won’t cut it.

Lock#lock, ConcurrentBlockingQueue#offer, Thread.sleep, etc., and numerous other purely local (non-remote) operations can take an unbounded time to finish, suspend threads, and do not have anything to do with RPC. You cannot identify these methods from type signatures, and can only identify some of them through bytecode analysis (what’s the Java interop story?).

I think tracking suspension is a genuinely different goal than tracking RPC, and that precision is required to distinguish between them. And while I think the Remote[A] conversation would be more productive than focusing on “suspension”, I genuinely do not think lack of “suspension tracking” or lack of “RPC tracking” is on the top 100 list of problems to solve in any modern programming language, and if done in the viral fashion your comment above suggests (forbidding Future[A] => A), it will actually interfere with best-practice architecture, in which something like a UserRepo interface should not expose low-level infrastructure concerns inside type signatures, even though it does indeed depend on remote services, because the business logic or higher level code depending on such interfaces is not equipped to implement RPC-specific logic, which is best handled close to the edge, before it is eliminated at higher layers of the application.

[LPTK]

@jdegoes

I will quote JEP 425

Notice how even this document explains the difference between sync and async programming in terms of concurrency? For example:

This programming style [async programming] is at odds with the Java Platform because the application’s unit of concurrency — the asynchronous pipeline — is no longer the platform’s unit of concurrency.

I don’t understand why you want to argue that async is not about concurrency, when it absolutely is. If no concurrency concerns were involved, most people would just write everything in direct style. FP people would still write code in monadic style, but executing that code would have no reason to use more than one thread (be it a physical or virtual thread).

Anyway, disregarding this detail and your imprecise use of terminology, I think I understand better what you mean with the distinction of sync/async. Although you have not phrased this explicitly, would it be a good characterization to say that:

Synchronous programming deals with concurrency by using a fork/join model, where explicit blocking join points are used, while asynchronous programming deals with concurrency by always putting the rest of the computation in continuations (as “callbacks”, which is isomorphic to monadic programming).

?

This was touched on by people above: in async programming, the use of Await.result outside of the application boundary is pretty much an anti-pattern. In my definition, using Await.result extensively throughout the code would essentially bring you back to a synchronous programming style.

@jdegoes

The reason the industry adopted asynchronous programming, despite the fact that it is a more difficult style to program in (even with added layers), is scalability.

The full reason is scalability of concurrent programming. Again, if it were not about concurrency, async would never be used.

@jdegoes

However, let me be clear: when I say, Loom makes everything async, I am being imprecise: while Loom threads have an async (callback-like) implementation inside the Java runtime, to anyone building on the JVM, Loom code is synchronous code, written in a purely direct style.

I agree that you are being imprecise. Here you are mixing up the programming model, which is sync vs async, with the underlying implementation model.

@jdegoes

4. Just like asynchronous code may never resume (which corresponds to the callback never being invoked), synchronous code may never resume (e.g. while(true){}).

And this is a great example why using confused terminology is bad. You previously claimed that Loom would make sync code async. But that’s patently false: it’s easy to see that while(true){} will not become “async” in Loom.

@jdegoes

I take the shorthand form because the idea that “async is scalable, sync is not” is etched into everyone’s brains, due to decades of wrestling with JVM physical threads

You started off by purporting to rebuke what’s “etched into everyone’s brains”. Cf. you tweet saying “Scala’s Future has resulted in a generation of quite intelligent Scala programmers believing that […]”. You can’t do that effectively if you yourself are not using precise terminology.

[fanf]

All the insights in @jdegoes answer pretty match my experience (so: anectdotal, but one more data point), and especially the part about “why tracking RPC”:

@jdegoes

This way of approaching the “RPC problem” has the added advantage that it respects the fact that ultimately, some developer is going to want high-level interfaces like def getUserById(id: UserId): User, and does not want implementation details in such a high-level interface to leak (including a RemoteException, since if failure survives even after application of a retry policy, then it becomes fatal for the route handler, and users do not wish non-recoverable errors to affect static types).

As explained in previous comment, I more care about what errors my app can handle and recover from (ie the limite of the model) than which methods do RPC calls. Perhaps I don’t care at all about performance (toy script app?). Perhaps I deeply care, and RPC tracking is clearly not enough - what about that unserialization taking forever because there’s a JSON-bomb in it? And what error I want to track are totally dependant of my app intents, level of maturity, external constraints, etc.

And of course in an app, several sub systemq with different needs in error handling can interact. In the sub system dealing with remote call of some other service, it is likely that you will want to track errors that can happen, like timeout but also misconfiguration, total unavailabitlity, bad data serialisation, authentication or authz error, etc. But in another one, perhaps it’s totally useless because such error would be unrecoverable from. Things must happen that way else the app internal model is broken, and then, tracking these error cases in type system is just boiler palte, info cluttering, and giving bad clue to future devs.

All that said, if one wants to be able to handle error in a fine grained, compositional, boiler-plate free way and with a compiler checking that you didn’t missed any cases, yes there is huge benefits in tracking them in the type system.

But this is fine grained error tracking to match you model constraint. Not RPC tracking.

[mdedetrich]

@fanf

As explained in previous comment, I more care about what errors my app can handle and recover from (ie the limite of the model) than which methods do RPC calls. Perhaps I don’t care at all about performance (toy script app?). Perhaps I deeply care, and RPC tracking is clearly not enough - what about that unserialization taking forever because there’s a JSON-bomb in it? And what error I want to track are totally dependant of my app intents, level of maturity, external constraints, etc.

I think this line of arguing is deceptively (but not intentionally) fallicious because you assume there is a false dillemma fallacy when there isn’t one. By nature of being RPC there is a whole set of errors that you cannot experience in normal sync code and at least in my personal experience its specifically those errors that I care about because I have most issues with, errors in sync code tend to be easier to reason about/control (note that I am not advocating that we shouldn’t care about them, just that most of my hair pulling errors are typically not dealing with sync code).

Personally the reason why I care most about seeing which methods/functions are RPC and which aren’t is that on a practical level me knowing precisely which calls happen is immensely useful both in debugging problems and in reasoning about code.

@LPTK

Synchronous programming deals with concurrency by using a fork/join model, where explicit blocking join points are used, while asynchronous programming deals with concurrency by always putting the rest of the computation in continuations (as “callbacks”, which is isomorphic to monadic programming).

I would say that even this definition is not entirely “correct”. To me synchronous programming is imperative style of programming where you deal with OS thread’s as your basic primitive and any function/method calls will block on those threads. Your way of achieving concurrency here is by spawning OS threads.

Async programming is when your method/function calls do not block the underlying thread but instead immediately “return” a handle that represents the running computation in the background. Using this definition means it covers both callback based asynchronicity (old school Java) and the more useful Future/Promise abstractions that is used in modern Java and Scala.

Of course by nature of "return"ing immediately you can also achieve concurrency. But alas its hard to discuss details sanely when terminology is mixed up all of the time

[arturopala]

@LPTK

if it were not about concurrency, async would never be used.

@mdedetrich

its hard to discuss details sanely when terminology is mixed up all of the time

Synchronous in a general language means “happening or done at the same time or speed”, and its synonyms are “coexistent, simultaneous, concurrent”. Sic!

[seigert]

Anyone who tried to implement Future#zip for Go coroutines (as is ‘run two concurrently till completion, or capture first error and cancel another’) know that’s not easy: one would need couple of channels, latch, and a discipline to not to hang on anything. traverse/sequence and such are even more hideous.

Loom without ‘structured concurrency’ primitives does not do much for it either.

As a developer I wish I would be able to do not care about implementation details: loom, execution contexts, work stealing pools or “world most performant* nano-actor ZIO micro-kernel (*amidst ZIO micro-kernels as rated by ZIO developers)”. As a developer I wish I would write a clean readable code with known semantic and guarantees: things run sequentially, or things run in parallel, or things will be canceled if not computed in time, or resources will be freed when go out of ‘scope’ etc.

So in a world of ‘functional effects’ Loom would not do much.

[mdedetrich]

I wouldn’t agree with using Cambridge as a reference for terminology specifically with computer science, i.e. Synchronous vs Asynchronous

[arturopala]

This is not to say that we cannot just take a word and re-define it as we like for the purpose of our narrow domains, but rather to say that doing so without wider language context consideration does not help to communicate the meaning efficiently.

In the programming domain, the term asynchronous was coined to describe the situation when the program does not wait (as usual) for the result of operation A before proceeding with the next operation B. It means that the order of executing A and B can happen out of sync (async) with the current program flow, and implies not-blocking (not waiting-for) as seen from this program frame of reference.

[mdedetrich]

@arturopala

In the programming domain, the term asynchronous was coined to describe the situation when the program does not wait (as usual) for the result of operation A before proceeding with the next operation B. It means that the order of executing A and B can happen out of sync (async) with the current program flow, and implies not-blocking (not waiting-for) as seen from this program frame of reference.

Right, and this definition is largely the same as mine albeit higher level.

[jdegoes]

@LPTK

I don’t understand why you want to argue that async is not about concurrency, when it absolutely is.

As I have stated previously, and as anyone with a similar background in asynchronous and concurrent programming will reinforce (including the authors of Loom, for example), asynchronicity is orthogonal to concurrency.

One of these concepts refers to a continuation-passing style of programming, in which the results of computations are not returned from functions directly, but rather, are passed to callbacks, which are passed into those functions; while the other refers to the interleaved execution of two or more semantically independent chains of sequential instructions (“threads” / “fibers” / “strands” / etc.).

However, I believe I know why you (and maybe other Scala developers) are so confused: because while asynchronous programming is orthogonal to concurrent programming, the motivation for adopting asynchronous programming was in fact to make concurrency more scalable.

Stated differently, the reason we invented async programming was indeed related to concurrency: we wished to have more scalable concurrency than OS threads allowed. However, the reason for inventing async programming should not be confused with async programming, the concept.

We may do async programming for concurrency-related reasons (scalability), but async programming is a completely orthogonal concept to concurrency, and in a post-Loom era, we no longer need to reach for async in order to obtain scalable concurrency, because the concurrency primitive that Loom gives us (virtual threads) is already as scalable as anything we could emulate using async programming.

In summary, some may be confused that async programming has overlap with concurrent programming because our original motivation for doing async was to increase the scalability of our concurrency: but that does not, and will not ever, mean or imply that asynchronous programming (the concept) means anything other than “programming with callbacks”, which is a completely separate concept from concurrent programming (“programming with threads”).

(The confusion is probably made a lot worse in Scala because Future, many new programmer’s first introduction to either asynchronicity or concurrency, indeed conflates async and concurrent features, with some benefit, obviously, but with the side-effect of tangling these concepts in new developer’s minds.)

[jdegoes]

@seigert

Loom without ‘structured concurrency’ primitives does not do much for it either.

I think while, Loom solved a very real and hard programming challenge (scalable concurrency), in the way it should be solved, the challenges of resource-safe, efficient, non-leaking concurrency and parallelism remain, and while there’s no reason to reach for ZIO (et al) because of “async”, those who enjoy the high-level, resource-safe, globally-efficient concurrency model, together with error management, context, metrics, integrated logging, retries, scheduling, etc., will not really care at all about Loom, except maybe enjoy a bit of a performance boost in some contexts.