This is my favorite JVM project and I think it's going to be huge!
This model of concurrency is so much better than the async/await model used by many other languages. No more colored functions [1], or worse Completable<Future>, promises et al. Nice stacktraces, debuggers that work plus no need for thread pools anymore. I can't wait for this to be ready for production.
Only drawback seems to be when calling native code. I guess it's the same problem that Golang has. Good thing is that the Java ecosystem is not that dependent on native stuff, so I think it's a fair tradeoff to make.
That's a property of the function, whether or not you want the type system to help you with it. I can call a function myRedFunction, but if it calls a blue function, it's blue.
> or worse Completable<Future>
What is this?
Future<Result> f = e.submit(() -> { ... return result; }); // spawns a new virtual thread
> This model of concurrency is so much better than the async/await mode
ThreadFactory tf = Thread.builder().virtual().factory();
ExecutorService e = Executors.newUnboundedExecutor(tf);
Future<Result> f = e.submit(() -> { ... return result; }); // spawns a new virtual thread
...
Result y = f.get(); // joins the virtual thread
I don't get it. This just looks like async/await (submit/get) with extra steps.
Why not just write
Future<Result> f = async ( ... result; )
Result y = await (f);
> That's a property of the function, whether or not you want the type system to help you with it.
The problem is that in, say, C# or Kotlin, subroutines with the same semantics come in two different colours. The type system "helps" you distinguish between things that aren't meaningfully distinguishable.
> I don't get it. This just looks like async/await (submit/get) with extra steps.
You only need to submit if you want to do stuff in parallel and then join (also, there are no extra steps). The analogue to:
> The problem is that in, say, C# or Kotlin, subroutines with the same semantics come in two different colours. The type system "helps" you distinguish between things that aren't meaningfully distinguishable.
The only cases where you want the same function to come in two colour variants is when it's a higher-order function, and in that case what you really want is not to ignore colour but for the function to be polymorphic over colour. (i.e. higher-kinded types).
> The distinction is important though, because it affects the observable behaviour of other effects present in these functions.
Only if the language is single-threaded to begin with, like JavaScript.[1] There is no real difference in C# and Kotlin between, say, sleep and delay. It's a syntactic distinction between semantically equivalent terms (for all practical purposes).
[1]: Even then there are better ways to reason about effects.
Hmm. I find it hard to believe that whether a given block of code executes on a single OS-level thread or yields and is rescheduled across multiple OS-level threads is never semantically important (to resort to an extreme example, one kind of function can be used in a callback that's invoked by native code and the other can't) - particularly given how many languages (including Java!) started with a green threading model only to later abandon it. And so I'm very dubious about erasing that distinction in low level functions, even if it's not relevant in the vast majority of cases, because once you erase it it's impossible to ever recover it at higher level.
It is never semantically important (in a context that is multithreaded anyway) because you don't have more control over scheduling with async/await than you do with Loom's virtual threads. In either case, the code doesn't know if scheduling takes place on a single kernel thread or multiple ones. Nor does it know about the existence of other threads running concurrently. There could be some technical differences, like various GPU drivers allowing only specific kernel threads to use it, but Loom answers that with pluggable schedulers.
The difference in the native call case isn't semantic; it's, at worst, a difference in performance [1], and as far as I know, no one is making that distinction today, so there's nothing to erase. Moreover, as Java doesn't have async/await, there isn't even an artificial distinction to erase.
I don't know all the reasons Java abandoned green threads. One of them was that the classic VM that had green threads was simply replaced with HotSpot; it wasn't an evolution of a single VM. But more practically, and putting aside the fact that it was M:1, at the time there Java code relied on native FFI a great deal, whereas today it is very rare in general.
[1]: This is a little inaccurate. There could be a difference in liveness properties (deadlock) in that case, but neither async/await nor kernel threads, for that matter, make intrinsic liveness guarantees. You have to trust your scheduler in all cases.
> you don't have more control over scheduling with async/await than you do with Loom's virtual threads. In either case, the code doesn't know if scheduling takes place on a single kernel thread or multiple ones.
You do have that control: the value of having explicitly async functions is that it becomes possible to have non-async functions (just as the value of having explicitly nullable values is that it becomes possible to have non-nullable values, and the value of having explicitly-exception-throwing functions is that it becomes possible to have non-exception-throwing functions). A non-async function is nothing more or less than a function that is guaranteed to execute on a single native thread, whereas an async function might be executed over multiple native threads; in particular after `val x = f()` execution will continue on the same native thread as before, whereas after `val x = await f()` execution might continue on a different native thread.
I hope it works out, but there just seems to be so much opportunity for unforeseen edge cases; something very fundamental and global that every (post-1.2) Java programmer is used to knowing will no longer be true.
Everything on a thread always executes on the same thread, whether or not it's virtual; that's an invariant of threads. You cannot observe (unless it's a bug on our part or you're doing something complicated and deliberate) that, when running on a virtual thread, you're actually running on multiple native threads any more than you can observe moving between processors; if you could, that would indeed be a problem. The JDK hides the native thread the same way the native thread hides the processor.
That async/await allows this implementation detail to leak is not a feature, it's a bug. It forces a syntactic distinction without a semantic difference (well, it introduces a semantic difference as a bad side effect of a technical decision -- the semantic difference is that you can observe the current thread changing underneath you, which, in turn, makes the syntactic difference beneficial; in other words, it's a syntactic "solution" to a self-imposed, avoidable problem).
> That async/await allows this implementation detail to leak is not a feature, it's a bug. It forces a syntactic distinction without a semantic difference (well, it introduces a semantic difference as a bad side effect of a technical decision -- the semantic difference is that you can observe the current thread changing underneath you, which, in turn, makes the syntactic difference beneficial; in other words, it's a syntactic "solution" to a self-imposed, avoidable problem).
You're putting the cart before the horse. On the assumption that the programmer needs to be able to control native threading behaviour in at least some cases (if only a small minority), surfacing the details of the points at which evaluation may cross native thread boundaries is useful and important, and async/await is the minimally-intrusive way to achieve that. If it's really possible to abstract over the native threading completely seamlessly such that the programmer never needs to look behind the curtain, then async/await is pointless syntactic complexity - just as if it were possible to write error-free code then exceptions would be pointless syntactic complexity. But if there's still a need to control native threading behaviour then this is going to end up being done by some kind of magic scheduler hints that aren't directly visible in the code and will be easy to accidentally disrupt by refactoring, and in that case I'd rather have async/await (as long as I've got a way to be polymorphic over it).
I reject both assumption and conclusion. To the limited extent such control is needed, virtual threads with pluggable schedulers are superior in every way, and certainly less intrusive. There aren't any "magic scheduler hints", either. You can assign a scheduler to each virtual thread, and that scheduler makes the relevant decisions. That's the same separation of concerns that native threads have, too.
The only place where this could be important is when dealing with Java code that cares about the identity of the native thread by querying it directly (because native code will see the native thread, but it's hidden from Java code). But, 1. such code is very rare, 2. almost all of it is in the JDK, which we control, and 3. even if some library somewhere does it, then that particular library will need to change to support virtual threads well, just as we require for most new big Java features.
Having said that, I don't claim async/await isn't a good solution for some platforms, and in some cases there's little choice. For example, Kotlin has no control of the JDK, which is necessary for lightweight threads, and languages with pointers into the stack as well as no GC and lots of FFI might find that a user-mode thread implementation is too costly. But I think that in cases where usermode threads can be implemented efficiently and in a way thar interacts well with the vast majority of code, they are clearly the preferable choice.
> There aren't any "magic scheduler hints", either. You can assign a scheduler to each virtual thread, and that scheduler makes the relevant decisions. That's the same separation of concerns that native threads have, too.
I'm thinking of when you need to "pin" to a particular native thread. In a native-threads + async/await language you have direct and visible control over where that happens. In a green-threads language it's going to involve magic.
> Having said that, I don't claim async/await isn't a good solution for some platforms, and in some cases there's little choice. For example, Kotlin has no control of the JDK, which is necessary for lightweight threads, and languages with pointers into the stack as well as no GC and lots of FFI might find that a user-mode thread implementation is too costly. But I think that in cases where usermode threads can be implemented efficiently and in a way thar interacts well with the vast majority of code, they are clearly the preferable choice.
A priori I agree. I expected green threads in Rust to work out too. I just can't get past having seen it fail to work out so many times. Maybe this time is different.
would give you a factory for virtual threads that are all scheduled on top of one native thread (I didn't pick a specific one because I wanted to use something that's already in the JDK, but it's just as simple).
> I expected green threads in Rust to work out too. I just can't get past having seen it fail to work out so many times. Maybe this time is different.
Implementing usermode threads in Java and in Rust is very different as their constraints are very different. Here are some differences: Rust has pointers into the stack, Java doesn't; allocating memory (on stack growth) in Rust is costly and needs to be tracked, while in Java it's a pointer bump and tracked by the GC; Rust runs on top of LLVM, while Java controls the backend; Rust code relies on FFI far more than Java. So both the constraints and implementation challenges are too different between them to be comparable.
Not to mention that they have different design goals: Rust, like all low-level languages, has low abstraction (implementation details are expressed in signatures) because, like all low-level languages, it values control more than abstraction. Java is a high-level language with a high level of abstraction -- there is one method call abstraction, and the JIT chooses the implementation; there is one allocation abstraction, and the GC and JIT choose the implementation -- and it values abstraction over control. So whereas, regardless of constraints, Rust happily lives with two distinct thread-like constructs, in Java that would be a failure to meet our goals (assuming, of course, meeting them is possible).
There are many things that work well in high-level languages and not in low-level ones and vice versa, and that's fine because the languages are aimed at different problem domains and environments. Java should be compared to other high-level languages, like Erlang and Go, where userspace threads have been working very well, to the great satisfaction of users.
Having said that, I suggest you take a look at coroutines in Zig, a language that, I think, brings a truly fresh perspective to low-level programming, and might finally be what we low-level programmers have been waiting for so many years.
> No magic, just a custom scheduler for that particular thread.
But there's a spooky-action-at-a-distance between that scheduler and the code that's running on the thread. Code that's meant to be pinned (and may behave incorrectly if not pinned) looks no different from any other code.
> Not to mention that they have different design goals: Rust, like all low-level languages, has low abstraction (implementation details are expressed in signatures) because, like all low-level languages, it values control more than abstraction. Java is a high-level language with a high level of abstraction -- there is one method call abstraction, and the JIT chooses the implementation; there is one allocation abstraction, and the GC and JIT choose the implementation -- and it values abstraction over control. So whereas, regardless of constraints, Rust happily lives with two distinct thread-like constructs, in Java that would be a failure to meet our goals (assuming, of course, meeting them is possible).
Java isn't positioned as a high-level, high-abstraction language and that's not, IME, the user community it has. It's a language that offers high performance at the cost of being verbose and cumbersome - witness the existence of primitive types, the special treatment of arrays, the memory-miserly default numeric types, the very existence of null. I've heard much more about people using Java for low-latency mechanical-sympathy style code than people using it for high-abstraction use cases like scripting or workbooks. It's always been advertised as a safer alternative to C++ - rather like Rust.
(I'm all for trying to expand Java to be useful in other cases, but that's not grounds to sacrifice what it currently does well. For all the criticism Java attracts, it is undeniably extremely successful in its current niche)
> Code that's meant to be pinned (and may behave incorrectly if not pinned) looks no different from any other code.
Same goes for async/await. The decision to keep you running on the same native thread is up to the scheduler.
> Java isn't positioned as a high-level, high-abstraction language and that's not, IME, the user community it has.
I beg to differ. It aims to be a good compromise between productivity, observability and performance. Every choice, from JIT to GC, is about improving performance for the common case while helping productivity, not improving performance by adding fine-grained control. There are a few cases where this is not possible. Primitives is one of them, and, indeed, 25 years later, we're "expanding" primitives rather than find some automatic way for optimal memory layout.
> It's always been advertised as a safer alternative to C++ - rather like Rust.
I think you're mistaken, and in any event, this is certainly not our position. Java is designed for a good blend of productivity, observability and performance, and unless the circumstances are unusual, it opts for improving common-case performance with high abstractions rather than worst-case performance with low abstractions like C++/Rust. Roughly speaking, the stance on performance is how do we get to 95% with the least amount of programmer effort.
Anyway, regardless of what I said above, the constraints on the design of usermode threads other than philosophy are also very different for Java than for C++/Rust for reasons I mentioned. Still, Zig does it more like Java (despite still using the words async and await, but they mean something more like Loom's submit and join than what they mean in C#): https://youtu.be/zeLToGnjIUM
> Same goes for async/await. The decision to keep you running on the same native thread is up to the scheduler.
The scheduler decides what happens at each yield point, but code that doesn't yield is guaranteed to stay pinned to a single native thread. A non-async function is somewhat analogous to a critical section; the difference between async and not is a visible distinction between must-run-on-a-pinned-native-thread functions and may-be-shuffled-between-native-threads functions.
Can you give an example where this matters -- i.e. it's useful and allowed to move between native threads but not between well-known points -- given that the identity of the carrier thread cannot leak to the virtual thread?
> You only need to submit if you want to do stuff in parallel and then join (also, there are no extra steps).
I'm not comparing parallel to sequential. My point was that we're already doing parallel programming:
CompletableFuture<Result> fx = CompletableFuture.supplyAsync(() -> result);
CompletableFuture<Result> fy = CompletableFuture.supplyAsync(() -> result);
Result x = fx.join();
Result y = fy.join();
Parent doesn't like the existing parallel model. New model looks like:
// > plus no need for thread pools anymore.
ThreadFactory tf = Thread.builder().virtual().factory();
ExecutorService e = Executors.newUnboundedExecutor(tf);
// > No more [...] Completable<Future>, promises et al.
Future<Result> fx = e.submit(() -> { ... return result; });
Future<Result> fy = e.submit(() -> { ... return result; });
Result x = fx.get();
Result y = fy.get();
If going from the first to the second is an improvement, I don't understand the excitement. Especially when other languages look like:
fx <- async (return 3)
fy <- async (return 3)
x <- wait fx
y <- wait fy
> If going from the first to the second is an improvement, I don't understand the excitement.
The excitement is about going from my second example to the first.
> Especially when other languages look like:
It doesn't matter what they look like in the code. They suffer from all the problems I mention in the article: exceptions lose context, debuggers and profilers lose their effectiveness, APIs are split into two disjoint worlds. Virtual threads gives you code that doesn't just look synchronous, but behaves like that in every observable way.
What the code looks like on the screen is a very small portion of the problem we're trying to solve.
(1) That sounds like a recipe to unintentionally miss a ton of concurrency. Having to put an `await` there is a great indicator that you're forcing an order of execution.
(2) Can you give an example of an async and a non-async subroutine that have "the same semantics"?
The reality is that you rarely want to doA and doB concurrently, so optimizing syntax for that case is not useful, whereas you want to be able to call functions without having to worry about their color all the time, where "all the time" here is typically >1 time per function.
Many of you are perhaps scratching your head and going "What? But of course I concurrently do multiple things all the time!" But this is one of those cases where you grossly overestimate the frequency of exceptions precisely because they are exceptions, and so they stick out in your mind [1]. If you go check your code, I guarantee that either A: you are working in a rare and very stereotypical case not common to most code or B: you have huge swathes of promise code that just chains a whole bunch of "then"s together, or you await virtually every promise immediately, or whatever the equivalent is in your particular environment. You most assuredly are not doing something fancy with promises more than one time per function on average.
This connects with academic work that has showed that in real code, there is typically much less "implicit parallelism" in our programs than people intuitively think. (Including me, even after reading such work.) Even if you write a system to automatically go into your code and systematically finds all the places you accidentally specified "doA" and "doB" as sequential when they could have been parallel, it turns out you don't actually get much.
[1]: I have found this is a common issue in a lot of programmer architecture astronaut work; optimizing not for the truly most common case, but for the case that sticks out most in your mind, which is often very much not the most common case at all, because the common case rapidly ceases to be memorable. I've done my fair share of pet projects like that.
ParaSail[0] is a parallel language that is being developed by Ada Core Technologies. It evaluates statements and expressions in parallel subject to data dependencies.
The paper ParaSail: A Pointer-Free Pervasively-Parallel Language for Irregular Computations[1] contains the following excerpt.
"This LLVM-targeted compiler back end was written by a summer intern who had not programmed in a parallel programing language before. Nevertheless, as can be seen from the table, executing this ParaSail program using multiple threads, while it did incur CPU scheduling overhead, more than made up for this overhead thanks to the parallelism “naturally” available in the program, producing a two times speed-up when going from single-threaded single core to hyper-threaded dual core."
One anecdote proves nothing but I'm cautiously optimistic that newer languages will make it much easier to write parallel programs.
I want to emphasize that what was discovered by those papers is that if you take existing programs and squeeze all the parallelism from them you possibly can safely and automatically, it doesn't get you very much.
That doesn't mean that new languages and/or paradigms may not be able to get a lot more in the future.
But I do think just bodging promises on to the side of an existing language isn't it. In general that's just a slight tweak on what we already had and you don't get a lot out of it.
Well, the first two lines are usually written somewhere at the top level. You won't actually write that anytime you want something to be async. The last two lines are what you usually write, so most of the time, the number of lines of code are equivalent.
Second, Java doesn't have async/await today. If Java was to introduce it, it wouldn't be compatible with the code written today. The big benefit of project loom is that code written today will get all the benefits of async/await without changing any code. In fact, because it's not important to pool threads, it actually gets easier.
The big problem with async/await in languages like C# or even coroutines in Kotlin is that it doesn't mix with the "old" api. The get the benefits you need to do a huge refactor of your code, and you need to make sure that any library you pull in is compatible.
You don't. You can call synchronous functions from async functions just fine and vice versa. The syntax isn't identical but I don't understand this split the world meme.
In my experience with C#, I ran into apis I wanted to call that were async. I couldn't call them, even as Result foo = await async_thing(), unless my function was marked as async. I don't remember having a problem calling sync functions from async functions though.
But now you’re blocking the thread, so if this function is being used by an async function higher up in the function chain, it’s no longer async (unless that is changed as well.
There’s a split, because every step of an async process has to be async/await compatible or you lose all benefit. In a small code base, this might not be a big problem. Where I work it will never realisticly be done.
With project loom we won’t have to do anything or learn anything new. Just upgrade the java runtime and everything just works better :)
But yeah, the magic is: All code is now implicitly async (except native calls) - as if all code always had async in it's signature. Just - there is no need to now add this to the signature.
>I don't get it. This just looks like async/await (submit/get) with extra steps. Why not just write
because altering the underlying thread pool can convert all existing code to fibers without syntax changes. This is the crux of "colored functions", two different signatures for async/non-async.
The implications for improving the performance of existing code are enormous. Most Java API's give you at least indirect control of the thread pool used. Which means they can be converted to fibers without updating the library.
In Java, executors are basically thread pools. If you can update the executor to use fibers the rest of the code is using fibers now. Your second example only converts a single call site to fibers. A thread pool could be used in hundreds of places.
In practice, this is a huge difference. I can update my Streams to use a fiber pool, and all across my app the hundreds/thousands of streams are using fibers now. Same story with REST and DB access. There's usually a single thread pool for each. A few lines of code to update the pools and my REST and DB calls are all using fibers.
Can you maybe elaborate why this model is better than async/await? I have to admit, I don't really know Loom, but from a first glance, it looks like this is just (green) threads, and async/await is anyway an orthogonal concept to this, or not? But maybe I am confusing things here.
Can you maybe recommend a good overview over all the current concurrent approaches and concepts, like async/await, and alternatives, and their advantages and problems? I would love to get a more recent overview on this.
I had the impression that most recent languages follow on the async/await concept. At least JS and Python. Maybe also Rust? Go? Erlang? And there are probably libraries for C++ to do the same, or maybe it is already integrated? I have to admit that I did not fully follow all this development.
Async/await split up your functions at their "blocking points" to fragments that can be run fragment-by-fragment, in an interleaved way by an executor / event loop, thus enabling high level of concurrency within a single OS thread.
The problem is that your split-up async function ceases to be a normal function. It doesn't use the stack in a similar way than normal functions. Furthermore, the executor / event loop is required for running those async functions.
So you can't call async functions from normal functions, because they aren't. You need to create an event loop and then hand the task over for it to process it. Your world splits up into sync and async functions.
Go and Erlang are going with the similar "lightweight thread" approach as Java's project Loom. On the other hand, JS, Python and C# have gone with async. I'm not sure if these languages actually need async, going with something similar as project Loom would have also fit, and would have been simpler from the user perspective, perhaps? But hindsight is 20/20.
Rust has also gone async, and I'd argue it's the only one of the pack who really have async/await as a necessity - this is because of two design requirements that differ from Java, C#, Python and JS: 1) must have native-level performance when calling foreign code that expects a C-like stack. Something like Loom doesn't provide that. 2) must not have an implicit default runtime.
Python is popular in data science, AI & those field depends on a lot of native libraries: NumPy, SciPy, etc. Maybe that's why they chose the async await model
> The problem is that your split-up async function ceases to be a normal function. It doesn't use the stack in a similar way
Have you taken the CLR (a.k.a. .NET / C#) async await for a spin? Microsoft's done a great job of squaring away those details so the stack/thread discontinuity doesn't much matter.
As others have pointed out, the big problem of async/await is that it splits the world in two kinds of functions that can't call each other.
For example, in Python you have this http client library called `requests`. It's very easy to use, but it is synchronous and you can't use it in when doing async/await. Instead you have to use a similar library called aiohttp. And it's like that for every single library or function doing any sort of IO, you need two versions of everything. Same thing happens in Node.js. You have `spawn` which is asynchronous and `spawnSync` which is synchronous. And so on.
And it doesn't stop with functions, this division goes across the language it self. For example in Python you have a `for` loop and an `async for` loop. Same in JS, you have `for` and `for await`. C# also has it's own `await forech` construct. It's like splitting the world for no reason.
The great advantage of Loom is that you don't need a parallel language, you can use the same http clients, database clients, filesystem libraries that you were using in a synchronous world, but now you will be able to use concurrently in millions of virtual threads.
As the article mentions, the whole reason why async/await was invented, is because doing concurrency with kernel threads is very expensive. They are slow to create and they use a lot of memory. Using Promises and an event loop you can run multiple concurrent operations in a single OS thread. This gives you the chance to achieve much more concurrency than running one operation per OS thread. The problem is that this changes the way you program a lot. Your code is not sequential anymore, your error handling is different, even some conditional operations are different. Async/await is just some syntactic sugar created to make this looks like the regular programming model. It makes the function calls look sequentials (by adding an `await` prefix), it makes the errors look like exceptions, it makes the for loops look like regular for loops. But this is just appearance. It's something that looks similar, but it's different, and that's why you can mix it with regular code.
Project Loom provides the same benefit of async/await, but without changing the syntax in any way. In C# you now have a split API: blocking api's which rely on threads for concurrency, and async api's which require you to use async/await syntax. These api's are not compatible, so to take advantage of async/await, you have to refactor your code.
When project loom is implemented in Java, all code which makes use of threads today automatically gains all the benefits of the async/await model, without any refactoring.
Go and Erlang doesn't do the async/await thing. They only have one notion of concurrency (goroutines in Go, processes in Erlang). They don't require you to use async and await constructs, but they work similarly to async/await under the hood.
Couple specific examples you may be familiar with:
Coming from Java to JavaScript, I really, really yearn for the equivalent of ThreadLocals. For example, most Java logging libraries let you set a "context" object that can be used for logging a request ID or user ID anywhere down the call stack. You can't do that in Node, so instead you have to explicitly pass that context object down in all your function calls, or make use of brittle, complicated, hacky, not-fully-supported features like "continuation local storage".
Debugging stack traces in an async/await world is a mess because you don't get a nice "from the start of the request down to this call" view of the stack.
Perhaps most importantly, with Node at least you need to worry about starving your main event loop because there is only a single top-level main thread (but understand that's not inherent to the async/await issue in general).
> Coming from Java to JavaScript, I really, really yearn for the equivalent of ThreadLocals
While I understand why you feel that way today, you may find yourself some point in the future deeply, deeply detesting ThreadLocal. Sure it seems all convenient but all hell breaks loose when you need to use a threadpool, a future, an actor model...
This a Node limitation, though. E.g. .NET has had asynchronous call contexts, which encapsulate stuff such as "async thread locals", for a very long time.
Similarly, for the stacks, it's largely a tooling issue - if the debugger understands async, and the framework retains enough information about the origin of the promise, it can re-create the original stack trace, and combine it with the current one - .NET debuggers in VS and VSCode do this, as well, and WinRT even does this for C++.
Code that runs in goroutines looks no different from code that runs outside goroutines. You do regular blocking IO. The application code of a performant RPC / HTTP server never needs to refer to any kind of concurrency construct, unless it wants to make outbound calls in parallel. (Your HTTP or RPC library takes care of dispatching inbound requests onto goroutines).
As much as I want a more expressive language, all the explicit async/await, futures, callbacks, etc. junk needed just for your API handler to cooperate with concurrent requests (not even do its own parallel processing) keeps on sending me back to Go. I look forward to being able to write Java in the same style.
> Can you maybe elaborate why this model is better than async/await?
Others have given actual explanations, but I'll chime in with a bit of a tongue-in-cheek one: it's better than async/await because it doesn't have async/await (yet has all the benefits provided by them).
I wish I understood this better. If you're writing code in the same style whether it's async or not, how can you manage return values and reason about flow?
The flow is sequential, as laid out in the code, the same as for any old-fashioned code. It has the same semantics as the non-async code. The only difference is that OS thread can be released to do some other work while you wait for IO operation to finish.
For JS, cynical take would be that Javascript is just being consistent in making every wrong choice in language design. More serious (and more correct) answer is that, being single-threaded, adding any kind of threads would be pretty adventurous. Async/await as a sugar for promises is a decent improvement over using promises directly, so it's not so bad of a choice after all.
The acceptable alternative was, apparently, forcing to programmers to litter their code with words such as async/await/task in-between the words and symbols that actually describe the purpose of their program, should they want to take advantage of non-blocking IO. All the way throughout the call-stack, nonetheless. And you also get crap like this: https://github.com/dotnet/corefx/pull/4868
Win32 has fibers, which are essentially green threads. And for a while, .NET itself tried to support it. The problem is that there was never widespread support on the native side of things, and in .NET itself, user code has to not make certain assumptions to allow for fibers to work (e.g. that each managed thread maps 1:1 to an OS thread), which in practice nobody followed.
This points at the bigger problem that every implementation of green threads has: it breaks interop unless everybody buys into it. Async/await is fundamentally callback-based, which can be represented nicely even in C ABI - so literally any language that can interop with C, can interop with an async/await model, with execution flow asynchronous throughout the entire chain. Runtimes that choose to roll their own green threads, like Go, and I guess now also Java, are effectively stating that their convenience is worth the inability to interface well with async code outside of their runtime.
Win32 fibers would probably not be a good match for potential .NET fibers. From the link in my previous comment, they seem to allocate the stack of the same size as regular OS threads, making them not appropriate for the use case of keeping thousands or more fibers around.
> Runtimes that choose to roll their own green threads, like Go, and I guess now also Java, are effectively stating that their convenience is worth the inability to interface well with async code outside of their runtime.
You'd still be able to interop with async code outside of runtime, you can still model the API as callbacks for the interop purposes where needed.
But yes, interop has sam trade-offs. Which are, IMO, entirely correct trade-offs to make for Go, Java and .NET (and overwhelmingly so). But they are probably not for (as an example) Rust, according to it's goals.
They're definitely not correct for .NET - you forget WinRT. That ecosystem was much more heavily invested in native interop from the get go - COM interop, P/Invoke, various runtime features like unmanaged pointers and unions etc. So once native async became a thing, .NET developers expect to be able to interop with it, as well.
I been waiting for this project for a while. I saw an amazing demo of this on youtube video. In the demo a regular Jetty server running a simple endpoint that slept for 1 sec was DOSd with multiple concurrent requests. You can see as the number of concurrent requests increased, the execution time increased. The presenter then changed the code of the Jetty server to create a new Fiber vs a Thread. Then reproduced the test and this time there was no increase in execution time.
What I really like about this project is that we can keep Spring MVC applications in non-reactive form. Spring MVC came out with reactive framework for implementing APIs. But I'm not a fan of this style for a few reasons, plus there is millions of LOC doing it the non react implementation. By utilizing project Loom, we do not have to switch over to the Spring reactive way, and we can increase the performance of existing code.
> A virtual thread is a Thread — in code, at runtime, in the debugger and in the profiler.
To explain: if you debug async code on Kotlin right now, you cannot single-step in a debugger, because the debugger attaches to a thread. But coroutines get scheduled on different threads all the time!
I wonder what this means for ThreadLocals and Locks? Will they work as before? If so, that is huge! Because locks don't work with async code either, for obvious reasons. The consequence is that the whole ecosystem is split in two parts! You cannot just use a Guava cache in Kotlin coroutine-based code.
If Loom manages to avoid all of this, this is fantastic news.
This is outstanding. They solved the „what color is your function“ problem! The Rust ecosystem has a completely separate std-lib just for async. I think there is one other language that managed to avoid that problem too by using compile-time magic [0].
I mean, just because someone made a crate with std in the name doesn't make it another standard library. The actual standard library is still useful in async, as only the things that does IO or otherwise needs to sleep requires special handling.
I know of only one other, in erlang writing code to "just block" works as it should. In erlang processes are even fully preemptible too, not just at possibly-blocking points, although native code blocks the OS thread like in loom.
From what I understand, Erlang only preempts at function entry or exit, or at marked points in NIFs (C code).
Because functional programming requires loops to be written as recursive function calls, you can't do too much without calling/leaving a function, so a process can't avoid premption.
I don't understand, you can call a coroutine from a synchronous function and call a normal or suspend function from a suspending one.
By colorless we mean that you can use a suspend return type directly and not through the ugliness of a promise.
Kotlin did choose to have the suspend keyword but it's just an explicit type, kotlin could have had no suspend keyword and be officially colorless, it is technically colorless and is I believe the best of both worlds
> I don't understand, you can call a coroutine from a synchronous function and call a normal or suspend function from a suspending one.
Not without a syntactic distinction, and not without losing the performance characteristics you want coroutines to begin with.
> By colorless we mean that you can use a suspend return type directly and not through the ugliness of a promise.
That's not colourless. Colourless means that there is no syntactic distinction between sleep and delay. Two subroutines that mean the same but have a different syntactic colour.
> kotlin could have had no suspend keyword and be officially colorless,
It could not. You'd need to control the JDK to do that, or depend on class reloading, or pay a very high price in performance. Not to mention that it still wouldn't help when calling Java code (directly or indirectly) nor would it change how, say, JFR observes your program.
The problem is that Kotlin has very little influence over the platforms it targets -- Java, Android, JS and LLVM (well, maybe it has some influence over Android) -- and so must do its best to implement its functionality on top.
> I believe the best of both worlds
We don't want two worlds. We don't need two worlds. We want one that behaves as it should.
Green threads have been around for a very long time before Go, and there's nothing specific about Go that made them solve any problem they didn't solve before. It just so happens that this is a solution that comes with many downsides, which is why it's not universally adopted.
Indeed, Java itself had green threads (only!) in the first couple of versions.
Its downsides depend on the language. It works better in languages with GC than languages without, and it works better in languages that rarely rely on FFI than in languages that rely on it a lot. One of its major "downsides" is that it's just harder to implement than async/await, and requires control over the backend, something few languages have (even Rust is on top of LLVM). For example, Koltin just couldn't implement useful usermode threads because it has no control over the JDK.
Golang occupies an interesting spot here. They never had to migrate from a predominantly blocking, thread-based ecosystem to async. Does Golang really have two colors, is explicit threading a thing (I honestly don't know)? Or is it really just one color, namely the async one?
Golang doesn't really have a sync context. It has one color because everything is async. The `go` operation is not comparable to `await`, rather it is comparable to spawning.
There’s a difference between sync and async code in Go, which is why you have all the normal threading primitives like mutexes, semaphores and blocking queues/channels.
The point is that functions themselves don’t come in sync or async flavors. Just like in Java.
I don't think the existence of mutexes, semaphores and queues/channels imply that there is a sync version of Go. You can totally use those primitives in asynchronous Rust too.
You call the queues blocking, but they aren't really in the sense of "blocking" usually used when talking about async in Rust. The Go runtime can and will preempt your Go code in the middle of waiting for a channel to run some other task, and this preemption is what makes it different from a blocking Rust channel. An async Rust channel will also make the calling function wait for messages when you await the receive method.
Basically my point is that because any Go code can be preempted at any point, that makes all Go code async. The language not making you type await on everything doesn't make it sync.
Well, my point is that go has the same concept of sync and async as Java. All Java code can be preempted at any point, thats how threads work. Go is more efficient at scale as it uses a more light weight unit of concurrency under the hood, but from a developers standpoint the code functions in the same way.
So if you think Go is pure-async, then Java is pure-async, as it has access to the same primitives as Go for dealing with concurrency. It’s just that Java, at the moment, spawns a full thread wheres Go does something more light weight under the hood.
Unless, of course, you define async as doing something with coroutines/fibers. But I’d argue that is an implementation detail.
In any case. We are essentially agreeing. Go avoids the two-color problem by having single colored functions. Wheras Rust, JS, C# have two-colored functions.
Well I define async as being able to run many things without spawning a separate thread for each thing, by somehow swapping the current task every so often.
Call it an implementation detail if you want, but in my eyes, it is what makes the difference between all-async and all-sync.
I don't know about single step when debugging, but it seems intelligible stack trace for async code is available only recently, starting with .NET Core 2.1
It was a long while back, so I can't be sure, but I seem to recall single step debugging and good stack traces being available since the first days of async/await, although there are some ways you can get yourself into situations where stack traces are near to worthless.
It started out with just Kotlin/JVM. But now you also have Kotlin/Native and Kotlin/JS. Being multi-platformed has some drawbacks, for example, that it can be difficult to expose certain abstractions that are only supported on a subset of the platforms.
Kotlin can to lower it's IR to platform specifically intrinsics and have platform exclusive features to an extent. E.g inline classes are currently only available on the JVM
I was reading about Go threads just the other day and the article mentioned that Go uses green threads because they are more efficient.
I thought this was weird because IIRC Java 1.0 used green threads in Linux and it was a big deal when they moved to OS threads.
I’ve long believed that the IT world moves in cycles but this is a very clear example of exactly that. Java has gone from green threads to Posix threads and now back to green threads.
I do think it’s awesome (I love goroutines) and Threads in Java have become a but of a nightmare, made a little easier with executors and CompletableFutures. So this further improvement is great news.
But still... call me when the builder pattern is dead.
As the article explains (I know it's hard to comment after actually reading), Java moved away from using several green threads on top of a single system thread.
What Loom and Go do is to schedule green threads on a bunch of system threads and spawn more system threads when they get blocked doing synchronous system calls.
No need for snark, I did read the article. I was just pointing out the irony that it was a big deal when the JVM moved away from green threads and now it’s a big deal when it moves back. It’s a comment on the hype cycle. And, as I said in my OP, it’s a good change and I’m happy to see it.
(And if you want to be pedantic, IIRC the green threads were originally mapped to the Java process, not a thread, because threads were either unavailable or immature in Linux when Java 1 came out; I can’t remember which)
You're missing the point, which is what the person replying was trying to explain. We're not simply going back and forth, we're learning from past experience and improving implementations.
The green threads on the JVM were not the same kind as green threads in Go (and Loom), they would block on IO. Can't speak for Loom, but Go automagically reschedules your green thread when it blocks which allows other threads to run while waiting.
The point is they weren't rescheduled when they would block in the JVM, every process has a main thread.
It’s hard to understand how I can miss my own point... yes, I know that we are not simply going back and forth; I did read the article. I know that the new implementation of green threads is more sophisticated than the original implementation.
But I find the cycle - what I called the hype cycle - from internal scheduling to external scheduling and back again, interesting, and I wonder what, if anything, we as an industry can learn from this?
ISTM that Java 1.2 could have improved on green threads instead of moving to os threads. So, is there something we can learn from these two transitions that will help us all make better decisions in the future? The use of OS threads and all the complexity that this has caused has cost the industry hundreds of thousands of hours of developer time. If we can learn some lessons from this then isn’t that a good thing?
I don't think this is a matter of hype cycle at all. There are two things that changed:
1. Threading got much faster and lightweight. This is what Java was initially trying to work around, until it didn't have to any more.
2. The problem moved to handling as many sockets concurrently as possible. Even lightweight system threads are too heavy for scaling linearly with the number of connections (too much context-switching overhead, too much space for stack, etc.)
Green-threading has become a good idea again because we now have a kernel API that is used to multiplex a lot (but not all) I/O systemcalls.
Today Go runtime uses epoll/kqueue to read from a big bunch of sockets, whenever something new happens to any of them. This takes one system thread only.
The API model of epoll/kqueue implies some way to handle concurrency in your user code: this can either be callbacks (or async/await syntactic sugar) or green threads and CSP (channels and so on.) This is why green threading is having a comeback.
(Sorry for implying you did not read the article!)
That's OK, lots of people fail to RTFA, but I really like reading about this stuff.
> Green-threading has become a good idea again because we now have a kernel API that is used to multiplex a lot (but not all) I/O systemcalls.
OK, that makes a lot of sense. I had to read up about how epoll is different from select/poll (that's how long it's been since I worked in C :). Clearly epoll was needed to make green threads efficient, but from what I can see, by the time epoll and friends were widespread, the pthread model was entrenched in Java.
> I’ve long believed that the IT world moves in cycles but this is a very clear example of exactly that. Java has gone from green threads to Posix threads and now back to green threads.
No it’s not the same thing - Java was previously M:1, then became M:M, and now Loom is M:N and also preserves M:M at the same time.
Sure, but back when Java had green threads on Linux, almost nobody had multicore processors, so it’s not surprising that it started as M:1. What is perhaps surprising is that Java then moved to using OS (POSIX?) threads instead of continuing to manage them internally. I remember OS threads were touted at the time as being faster than green threads.
It’s a comment on the hype cycle. As a long time (since 1.0!) java developer, I like that Java has adopted a more efficient threading model. But they are certainly returning to a state where the JVM manages the threads rather than the OS, which certainly is the same as the original release, at least on Linux.
Java didn't deliberately move to OS threads so much as the JDK replaced the "classical VM", which was interpreted and had green threads, with HotSpot, that supported OS threads. Moreover, back then Java was young, and people still heavily relied on calling native libraries, which make green threads much less beneficial.
So it's more circumstance and necessity than hype.
Not really. They could have supported green threads under hotspot, but they chose posix threads, because posix threads were much faster than the previous implementation of green threads.
Today, green threads are faster. Have os threads become slower? Not that I know of. All that’s really changed is how green threads are implemented.
We went from “green threads bad os threads good” to “os threads bad green threads good”. Sure, the new green threads are better than the old ones, but in some alternate universe maybe we could have got here without os threads (and maybe saved hundreds of thousands of hours of developer time in the process).
To me, what’s interesting is that this appears to be a case of evolutionary “local maxima”. Java went from green threads to os threads because os threads gave better performance for the amount of effort expended. But now we’re moving back to green threads - at much greater effort, 20 years later. I’m sure I’m not the only person that wonders what lessons we can learn from this?
> because posix threads were much faster than the previous implementation of green threads.
Green threads employed only one kernel thread so couldn't take advantage of multicore, and at the time Java relied on native code so much more than today.
> but in some alternate universe maybe we could have got here without os threads
Maybe :)
> But now we’re moving back to green threads - at much greater effort, 20 years later
These aren't green threads but M:N, and I don't think it's a much greater effort than the effort that would have been required to add it to HotSpot back then.
Sorry, I'm using the modern meaning of green threads [0], which is simply threads scheduled in userspace.
The first green threads implementation, which was of course Java 1.0, was certainly M:1 - my memory is that we didn't even have pthreads in linux at the time - but the term today doesn't specifically mean a M:1 mapping.
On the other hand, of course JVM1.0 invented "green threads" which was M:1. So yeah this is confusing and I made it more confusing. Sorry. :)
So the point I was trying to make is that Java originally scheduled threads in userspace, then delegated scheduling to the OS, and has now taken it back. In hindsight it seems that delegating this key function to the OS has caused a lot of pain, and I'm interested in the drivers for using OS threads instead of just making the original green thread implementation better.
To be even clearer, I see this a lot in software. We delegate some key function of a product (like database access) to some inefficient and complex machinery (like an ORM) and then spend years working around the problems that decision caused. I've done this myself. I'd like to understand how to stop doing it.
You are largely correct about the "reversion" to userspace threading, but the decisions (some of them were forced moves) weren't because of fashions but very different circumstances. You could point out that it is interesting that different circumstances have led to userspace threads.
I don't know all the reasons behind abandoning green threads, but some are mentioned here: https://docs.oracle.com/cd/E19455-01/806-3461/6jck06gqe/inde... namely, no multi-core and bad FFI ("interoperate with existing MT applications"). Plus, a little later, the transition to HotSpot left little desire to reimplement userspace threading. There was no pressing need, there was other, higher-priority work needed, and FFI was still an impediment. In other words, making green threads better was neither an important requirement nor a reasonable possibility back then.
Why have we "gone back"? 1. the scale of concurrency required today is higher and resulted in the scaffolding I mention in the article, 2. FFI is no longer an impediment as Java code rarely relies on native code these days.
Nevertheless, Project Loom doesn't switch from kernel to userspace threads, but rather lets you choose which implementation you want for each thread.
Green threads are threads which are scheduled in userspace; Java 1 threads were scheduled in userspace, and Loom threads are scheduled in userspace. Loom is absolutely a reversion to userspace scheduling, and I think the intervening decision to delegate scheduling to the OS is something that's interesting to think about, because in hindsight it seems to have been the wrong decision.
(Also, IIRC Linux didn't even have native OS threads when Java 1.0 came out, so M:1 was literally and the only option available in Linux at the time. My memory is that we had to wait for pthreads before we could have hotspot on Linux, but it's a long time ago and quite tangental to my point).
> and I think the intervening decision to delegate scheduling to the OS
This is where you're getting confused.
This wasn't an 'intervening decision' because we haven't removed it and with Loom the OS is still able to schedule your threads. The user-space threads are opt-in. That's why it's not a reversion. Because Java was never like this before.
How Loom manages killing a vthread from another vthread? In Erlang you can do it safely and without needing to check a cancellation signal in the process to be killed. Also, Erlang has per-process heaps, an usually overlooked feature that gives soft-realtime capabilities, avoiding GC hiccups. JVM+Loom seems to be closer to Go than Erlang.
JVM has pauseless GCs now, in fact two of them! The Erlang multi-heap approach is much less compelling now GC pauses are finally vanquished as a problem.
I'm not qualified to say it copying is necessary at a language level, but in terms of implementation, BEAM always copies for message passing, AFAIK.
A small caveat is that Erlang has a Refc binary type that is reference counted, when sending a message with a refc binary to another process on the same node the content isn't copied, just the reference value (like a pointer) and the reference count is incremented.
There is an area where copies could potentially be avoided, but I don't think they are. Recent versions of beam have an optional off-heap message queue feature; some software patterns have a proxy process that accepts messages and sends them as-is to another process; if the message is off-heap for the proxy process, and it would send to the next process off-heap, it could be possible to avoid copying it, but it might be a bit tricky. (I don't think this is done, but that's why I said AFAIK earlier)
I'm really excited to see how this impacts the performance of akka and other stream-like things in the JVM ecosystem. I've spent far too much time tinkering with thread pools and digging through profiles with Unsafe.park everywhere.
The project has been in development for 3 years, only "early access" binaries are available now, and the parent article has multiple TBD notes. It's going to be another few years before this is released, IMO.
> A server can handle upward of a million concurrent open sockets, yet the operating system cannot efficiently handle more than a few thousand active (non-idle) threads.
Does anyone have a reference to some up to date notes on the state of scalability of Linux and/or free-/dragonfly-bsd scalability at the process level?
I don't doubt processes are a bit heavy for scaling to a million active threads of execution - but it'd be nice to see what one could expect on a low end server (say 16 cores, 64gb ram) today?
I'd also be curious about this. C10k was a problem at the turn of the century but world has changed a lot since, both in terms of hardware and kernel.
Best reference I could easily find was this experiment that had 100k connections to mysql which has thread per connection model. Seems to handle it well
Can Loom be used to simulate RTL & verification language on top of that (similar to SV or e)? I mean practically (since theoretically the answer is yes)
Why is there still no timeline? I don't see any previews, let alone GA features in JDK15, which means that by JDK17 probably nothing's going to be released. Which is sad, because that pushes the GA to the next LTS a extra few years down the line... (2023/2024 the earliest) :(
In case anyone else is confused about some of the internal links appearing to be broken (e.g. the Scope Variables link), the content is on the second page.
gevent has some serious drawbacks. Stack traces are incomprehensible, it works by monkey patching existing code, and it falls apart if you have a blocking operation gevent can't make async. Loom is a bit like monkey patching, but at the VM level, so I expect it to be much more stable.
Seems fine but I'm confused by the async/await hate. At least in C# it seems to me thee is already a superset of Loom through async/await.
Thread.startVirtualThread(() -> {});
Starts a new virtual thread that will run a synchronous method. Ok cool. C# already has:
Task.Run(()=>{})
This will run synchronously on the shared thread pool. Tasks are futures and can return objects. Same as the Loom proposal.
Optionally you can decide to opt into the await unboxing sugar by adding async to your method signature. There's arguably some double dipping with the async keyword on a method. It allows the await keyword to be used inside the method but also forces the caller into a different calling style. You can argue this shows API intent for cooperative threading. That said, you can still use async and sync interchangeably.
The syntax seems very similar to me.
Is there something else going on in Loom that I'm missing? Is it a matter of how the virtual threads are scheduled/preempted vs how other languages with async/await schedule their tasks?
Maybe I don't understand correctly what your saying but "Task.Run" just schedules something on a normal (kernel) thread by using a common thread pool. The Java equivelant is probably "CompletableFuture.runAsync" which does the same thing.
Loom with "Thread.startVirtualThread" will run something on a userland / green thread (ie. not a kernel thread so no context switching and blocking it "costs" practically nothing). async-await is actually a subset of wat Loom does in the sense that Loom allows for far more then just async-await. Most of the "hate" for async-await is probably because it leads to the "what color is your function" [1] problem.
This model of concurrency is so much better than the async/await model used by many other languages. No more colored functions [1], or worse Completable<Future>, promises et al. Nice stacktraces, debuggers that work plus no need for thread pools anymore. I can't wait for this to be ready for production.
Only drawback seems to be when calling native code. I guess it's the same problem that Golang has. Good thing is that the Java ecosystem is not that dependent on native stuff, so I think it's a fair tradeoff to make.
[1] https://journal.stuffwithstuff.com/2015/02/01/what-color-is-...