JEP proposed to target JDK 19: 425: Virtual Threads (Preview)
mail.openjdk.java.net
mail.openjdk.java.net
Achieving 5M persistent connections with Project Loom virtual threads - https://news.ycombinator.com/item?id=31214253 - April 2022 (142 comments)
"Whereas the OS can support up to a few thousand active threads, the Java runtime can support millions of virtual threads. Every unit of concurrency in the application domain can be represented by its own thread, making programming concurrent applications easier. Forget about thread-pools, just spawn a new thread, one per task. You’ve already spawned a new virtual thread to handle an incoming HTTP request, but now, in the course of handling the request, you want to simultaneously query a database and issue outgoing requests to three other services? No problem — spawn more threads. You need to wait for something to happen without wasting precious resources? Forget about callbacks or reactive stream chaining — just block. Write straightforward, boring code. All the benefits threads give us — control flow, exception context, debugging flow, profiling organization — are preserved by virtual threads; only the runtime cost in footprint and performance is gone. There is no loss in flexibility compared to asynchronous programming because, as we’ll see, we have not ceded fine-grained control over scheduling."
This seems so obvious in hindsight as the "right way". I wonder why we went down that whole async/await craze with so many languages?
https://github.com/python-greenlet/greenlet
has been available for quite some time in Python (gevent probably being its most used flavor).
Note that it it actually predates the async/await approach which was incorporated into the Python language (so, in Python it was implemented as a third-party library -- even async/await had an implementation based on Python 2 using yield and some decorators: https://pypi.org/project/trollius/).
I really do think Python should have instead adopted gevent as it's async approach instead of asyncio and async/await etc.
rust's rfc explains some of the drawbacks: https://github.com/rust-lang/rfcs/blob/master/text/0230-remo... (which basically explains why a preview took until jdk 19)
// Edit: also:
void handle(Request request, Response response) {
var url1 = ...
var url2 = ...
try (var executor = Executors.newVirtualThreadPerTaskExecutor()) {
var future1 = executor.submit(() -> fetchURL(url1));
var future2 = executor.submit(() -> fetchURL(url2));
response.send(future1.get() + future2.get());
} catch (ExecutionException | InterruptedException e) {
response.fail(e);
}
}
async Task<IActionResult> Handle()
{
var url1 = ...
var url2 = ...
try {
var requestTask1 = FetchURL(url1);
var requestTask2 = FetchURL(url2);
return Ok((await requestTask1) + (await requestTask2));
}
catch (Exception ex)
{
return BadRequest();
}
}
> In fact go code is harder to reason because of this, since you do not know exactly when a function spawns a new goroutine.
This is no different in the async/await world, as you never know what function will launch a new "task" (this concept exists in Rust/Python and I assume others). If you don't need a new "thread", both models allow waiting for multiple things simultaneously.
It is better for two reasons:
1. Virtual threads are threads as far as the Java platform is concerned, meaning you get the same troubleshooting (stack traces), debugging, and profiling support by the runtime and its tools as for platform (OS-backed) threads.
2. There is no split-world of APIs, with separate and largely incompatible flavours, that need to be developed for two constructs that are semantically (almost) equivalent.
You are correct, however, that implementing user-mode threads is harder than implementing async/await. While the latter requires only changes to the frontend compiler (until you want to add tooling support), the former requires deep changes to the backend.
I never had any problem with debugging and profiling Futures/async/await. Neither in Rust/C#/Scala. And stack traces like Exceptions were never a problem, in fact go panics were way more troublesome... (and still are, most often you dont run into them...) Most of the time depending how "concurrent" your program is, it also makes just no sense to debug them anyway, if you deal with tons of concurrent code you should learn to make good log statements, it's bread and butter, especially once your in production.
> 2. There is no split-world of APIs, with separate and largely incompatible flavours, that need to be developed for two constructs that are semantically (almost) equivalent.
well yeah... but the question is why do you try to hide it? that programmers don't see that the code might run concurrent? most goroutine bugs I've seen happened because of the hidden concurrency. and it's odd that an important concept gets hidden, for the sake of split-world apis. heck thats exactly why there is a type system, to express the meaning of code. with green threads it's not really obvious on first sight what the code exactly expresses.
(P.S.: I switched pretty early in my career to Scala, now to Rust/Golang/C# so maybe I'm more biased when it comes to a strong type system and I prefer a strong typesystem with all it's quirks, i've seen nasty bugs with goroutines and with async/await, so I doubt there is any real world benefit over the two worlds)
There are legitimate reasons to prefer a colored approach to asynchrony. But the two points are good points and you should prefer the colored approach only when said legitimate reasons outweigh those points given the language's design.
A panic in Go will always show you exactly where the problem is in the first lines of the panic.
When you profile a server written in the asynchornous style (with, say, JFR), the server can be under heavy load and yet the profile will only show idle thread pools. I don't know what you mean by having no problem profiling, but the Java platform offers no mechanism to profile asynchronous code.
> but the question is why do you try to hide it?
We don't. Virtual threads are threads, and just like threads today Java or Scala or Rust, you need some explicit operation to perform a concurrent task on some other thread.
> and I prefer a strong typesystem
Great, but that has nothing to do with threads. Rust and Scala support threads, too, and do so without any indication in the subroutine's type for blocking operations.
How many servers have a spare half-terabyte of RAM to do that though?
((512 * 1024 * 1_000_000) / 1024 / 1024 / 1024 = ~500)
Secondly, you don't need 512kb per thread. The kernel will only use pages that are actually used, so by default new threads will use 4kb.
And the point still-stands - hardware threads are heavier than virtual threads.
And even if it doesn't... those TLB entries are also not free.
Java just requests the memory and linux just smirks and always says “yes sure, whatever you need” - it can do that even while the OOM killer is trying to decide which process looks like it has completed the most useful work so far :-)
TLB entries dont really come into it (the size isnt an issue from our stack usage but even if it were, we could use larger pages).
TLB flushes could be an issue with that many threads floating around but that’s a different story and not really a memory usage so much as a performance and latency issue.
No you use the advisory bits when you map the memory.
It does this so it isn't unable to commit more memory while trying to handle an exception.
> Java just requests the memory and linux just smirks and always says “yes sure, whatever you need”
Linux doesn't ignore even LOCKED does it? And it certainly can't ignore actual writes to pages, which is what for example pretouch_memory does.
Is that really what’s happening when we create another thread? That doesn’t fit with my understanding here. When i create a new thread, i expect its stack to be allocated before the break so mlock wouldn’t come into it. Even on a jvm with Xms smaller than Xmx (i.e. so the jvm could have to request to move the break and grow the heap to accomodate my new threads private jvm stack), I’m expecting you would have to explicitly request this behaviour somehow (because it’s quite unneighbourly and probably counter productive for most apps). I don’t know how you would do that, maybe there’s a call available under the unsafe packages?
I’m conscious I’m saying the jvm - i mean hotspot.
>> so it isn't unable to commit more memory while trying to handle an exception
Again this doesn’t fit with my understanding - you can receive a further OOME while handling an exception and that’s just bad luck.
>> Linux doesn't ignore even LOCKED does it? And it certainly can't ignore actual writes to pages
Sure but I’m going on the expectation that doesn’t apply here
Why? Stacks are just memory like any other. You can put them wherever you want.
> so the jvm could have to request to move the break and grow the heap
The JVM allocates heap space using mmap, not by moving the break.
> you can receive a further OOME while handling an exception and that’s just bad luck
If you're being reasonable you don't - the JVM maintains emergency heap storage (pre-committed!) space so it can keep allocating even when out-of-memory.
The JVM commits a suitable number of pages for the stack. Stack growth can mean new page commits to expand the stack. After a thread is idle for some defined period of time, the JVM may release back some of the pages used to expand the stack.
If you actually want to take advantage of this behavior, you have to pay attention to which threads are being scheduled to do work -- because if you randomly choose a thread from a pool they're all doing work, and the JVM thinks none of them are idle. You need to focus on particular threads.
And at the end of the day if you're allocating a certain number of threads to do "task-like" work, you need to be prepared to deal with the expansion of thread stack/overhead.
So overall...I like this new proposal. Building cooperative scheduling deep into the libraries makes sense, as does breaking the 1:1 ratio to OS threads.
In other cooperative scheduling libraries I've seen, that 1:1 ratio is retained, and this inevitably leads to a schism in programming: Those who believe that the "lightweight" thread truly is, and those who have learned otherwise.
[1] https://man7.org/linux/man-pages/man3/pthread_create.3.html
We addressed this, albeit very briefly, in the Alternatives section of the JEP: https://openjdk.java.net/jeps/425
There are multiple reasons:
1. Languages that don't already have threads have an implicit assumption built into all existing code that program state cannot change concurrently. This is why scheduling points need to be marked explicitly, and why adding threads — whether user-mode or OS — might break existing code in some very tricky ways. That's the case of JavaScript.
2. Some languages target an IR without control over its backend, making an efficient implementation of user-mode threads difficult, if not impossible. async/await requires only changes to the frontend compiler. That's the case of Kotlin, and, perhaps to a lesser extent, Rust.
3. Some languages have technical features that make implementing user-mode threads efficiently more difficult than in others. Pointers into the stack and careful control over memory allocation make this more challenging in languages like C++ and Rust than in Java.
I.e. IMHO async offers very weak reentrancy guarantees that are better enforced via other means (rust-like lifetimes, immutability annotations, atomic contructs, etc).
The programmers should be seen as “users” of the language.
What you give here is a list of excuses on why system X doesn’t do what is best for its users.
“It’s difficult to do A given B”
As a user I don’t care about B. I just want A.
For your example, the concept of explicit pointers are orthogonal to threads. That’s why you can have OS threads with explicit pointers.
Just because it’s difficult to make them work together doesn’t mean they are incompatible as concepts.
It’s still an implementation detail.
For example, during Win95 one could argue it’s impossible for a crash program not to crash the whole system.
As a user I don’t care what’s going under the hood. I just don’t like it when my Windows 95 app can crash the system. It is an implementation detail
For languages like Python for example, this is a big issue, and reason why alternative concurrency patterns to async/await haven't made much progress.
Well, ironically, the exception that applies to all languages being: "does my code still work?" choices... which is what pron was addressing.
But the GP replied to a comment that was claiming threads of execution are better than async. So in the context of the reply the threads are the “best for the the user”
I don’t know why people confuse this so much
E.g. it might be visible to you that a certain operation runs quickly on certain inputs, or that a particular output is chosen for a particular input, even though the documentation does not specify the exact output.
Targeting a specific IR is an implementation detail.
Having explicit pointers and thread are not mutually exclusive concepts. It’s the implementation details that make them difficult.
HN can be so annoying sometimes
As an aside, Ron had a JUG talk 6 months ago which I found really helpful where he went into more detail about why they chose this approach: https://youtu.be/KmMU5Y_r0Uk (27m20s mark, and from 2m50 there’s a more general introduction to Loom).
I’m sure there are other videos/papers as well, but this was a pretty good overview of Java vs other languages’ approach to async.
For example if I want to implement a "sleep" with async/await I probably only need to store the wake time as state, if I want to make a virtual thread to do the same I likely need to allocate a large stack just in case I use it.
Of course this can be mitigated with stack caching, small stacks, segmented stacks or other tricks. But doing this is still more expensive than knowing how much "stack" you need up-front and allocating only that.
The entire mechanism is rather efficient because in Java we don't have pointers into the stack, so we don't need to pin anything to a specific address, and stacks can be freely moved around.
> The optimization should work with Project Loom when it becomes available.
Rather than saying it's a benefit for particular languages, I'd say it's a benefit in particular contexts, e.g. in contexts where you don't have a heap. Of course it's true that some (most) languages don't support such contexts at all (for a host of good reasons), but the languages that do are shaped by that decision.
Many languages/runtimes want just a single coroutine/continuation construct to cover both concurrency and generators — which is a good idea in principle — but then they, especially low-level languages, optimise for the less useful of the two. I've seen some very cool demos of C++ coroutines that are useful for very narrow domains, and yet they offer a single construct that sacrifices the more common, more useful, usage for the less common one.
There was one particular presentation about context-switching coroutines in the shadow of cache misses. It was extremely impressive, yet amounted to little more than a party trick. For one, it was extremely sensitive to precise sizing of the coroutine frames, which goes against the point of having a simple, transparent language construct, and for another, it simplifies small code that has to be very carefully written and optimised to the instruction level even after the simplification.
> Many languages/runtimes want just a single coroutine/continuation construct to cover both concurrency and generators — which is a good idea in principle — but then they, especially low-level languages, optimise for the less useful of the two.
Notably Rust appears to be the opposite here, as it is first focusing on providing higher-level async/await support rather than providing general coroutine support, but its async/await is implemented atop a coroutine abstraction which it does hope to expose directly someday.
I'm sure you don't need to be told most of this, but I bring all this up to help answer the more general question of why not every language builds in a green thread runtime, and why one approach is not necessarily strictly superior to another.
It might be interesting to try something like Mesh (https://github.com/plasma-umass/Mesh) to share pages.
At the cost of breaking : conceptual model of concurrency ; debugging ; performance analysis ; tracing ; logging
but yeah...great stuff.
While the "stack is optimally sized" argument exists, it might not always be true: E.g. implementations could require far more memory being required for the "virtual" stack than what is actually required due to implementation challenges. That for example applies in various situations in Rust. Then a more classical stackless implementation which allocates state for each callback on the heap (like if you manually write boost asio code) which have quite some allocation and memcpy churn. And besides that a "virtual stack" might be more fragmented and less cache friendly than a contiguous stack, which also impacts efficiency.
For single-threaded event loops because most of those languages did not want to put the concept of thread safety onto the developer (e.g. JS and Python). And even the ones that do concern the user w/ thread safety don't have an intermediate representation and VM to automatically sequence instructions (e.g. Rust).
Funny, I have never heard anyone describe threads as the thing that made concurrent programming easier, or more threads would make it easier.
What makes concurrent programming easier is thread interaction models that make it hard to lock the process.
async\await, give you concurrency without the need to for the developer to explicitly write lock code. Much easier than having to manually handle that interaction.
async\await in no way relieve the developer from needing to worry about locks. If you have shared mutable memory, you have locks. If you aren't considering that, then you've got broken code.
> What makes concurrent programming easier is thread interaction models that make it hard to lock the process.
Yes, the way you do that is by focusing on message passing and immutable data when you can get away with it and, ideally, prebuilt thread safe datastructures when you can't.
What async\await buys is lightweight threading when lightweight threading isn't available. It allows you to have millions of concurrent processes running at the same time. HOWEVER, the cost of this is the "colored function" problem. You HAVE to mark up your code to let the compiler/framework know "this is code that can block and thus needs to be able to give up control and resume". This problem is difficult because you can't simply call an async function from a non-async function. You have to do some wrapping/juggling to make everything play nice.
The reason lightweight threading is nice is because you no longer have the colored function problem. From any reference or context you can say `CompletableFuture.supplyAsync(()->calculateValue())` and have a new concurrent action spawned. If that action does IO or whatever, it doesn't hog a thread, it just yields (like async/await does) and lets another task move forward.
The only reason for async/await is OS threads are expensive and have pretty large negative implications on operating systems.
If you really wanted to, you could simulate this behavior in threaded languages by having a global lock you acquire before you run anything.
It's the python GIL problem [1]
Languages like kotlin, rust, C++, or C# with async/await MUST concern themselves with locks because they don't acquire whole application locks anytime they run a sliver of code.
But if you don't, then yeah, you sort of run right into the need for an actual lock since you can have shared memory in concurrently executing interpreters. Entire: Web Lock [1] Specifically designed to resolve this issue.
In typical JS programming this is not an issue, but if you are trying to really abuse that javascript VM then this is the way you'd do it.
[1] https://developer.mozilla.org/en-US/docs/Web/API/Web_Locks_A...
This isn't really true. For example, UI code often uses async/await for the concurrency but a single UI thread to prevent locking. Because this design is only concurrent and not parallel you don't need to worry about thread safety while still getting a way to yield execution to other code.
C# does have a thread pool that you can use with async/await with multiple threads but that would be different from a UI thread or its synchronization context.
It's also my understanding that if you stick to async/await and don't create your own tasks, the synchronization context will be the one of the UI thread thus never run in parallel. However, this doesn't prevent you from doing so.
At some point you realize that explicit continuation passing, threads, coroutines and async/await are all the same thing, and the only thing that async/await gives you is a static bound on your suspended stack size (by enforcing suspension of a single activation frame).
The huge difference is that futures (and thus async/await) change the primary concurrency concept you think about into "wait until this value is available" rather than managing the locks and shared memory yourself.
The latter is still available (and still as footgunny as ever) but aren't as much of a problem when better options are more ergonomic.
Futures also compose far better than threads do, especially when you want to wait for "any of X" rather than "all of X".
Java has had Futures for a while (and pretty good ones since Java 8's addition of "CompletableFuture")
That being said, the biggest issue with Java Futures has not been the futures themselves, but rather the management of the ThreadPools for when you want to do IO in the futures.
As pron points out, virtual threads make that WAY better to work with.
Hindsight? There are many of us that have been advocating against the async/await madness.
I said it before and I will say it again. Async is today what OOP used to be in the nineties.
But I am a Java pleb.
That’s basically what async swift does?
It is also orthogonal to async/await.
Async/await is a mechanism a programming language uses to allow you to write straight line code without having to write blocking code. You could resolve that by creating a hulking thread, but that just means you need to await a thread result :)
Also JS isn’t going anywhere, and it seems unlikely it will ever get multiple execution threads
And, to be fair, languages like Erlang and Go went this route LONG ago.
Even Assembly can be considered to have one, in case of microcoded CPUs.
So did java! It originally used green threads, before switching to exclusively kernel threads.
I don't think you could, for example, use this model of many virtual threads with a traditional GUI framework as designs are still predominately single threaded by nature.
Said another way, cooperative scheduling has its place and will continue to.
I'm excited to see some threaded UI designs shake out of this, though.
you can do that as a design choice in languages that aren't exclusively single threaded.
Async/await is in many single and multi-threaded languages so I'm not clear on what you're saying.
In a proper threaded model you can use exactly the same syntax that you would use with async/await (futures, future combinators, message passing what have you) except you do not have to randomly annotate your code with awaits.
You can set up the same limitation in a threaded program/language, and people have (see vert.x, clojure STM, and others).
Well no, not _all_ of the complexities because you have await and explicit yielding. It's much easier to pass data from one asynchronous task to another using await than it is to, say, manually code the locks or state machine necessary for the message passing and/or callbacks.
Does anyone know how Project Loom has gotten around these constraints?
[0] http://www.open-std.org/JTC1/SC22/WG21/docs/papers/2018/p136...
One big (and IMHO the only) advantage of the async/await model is that the stack size can be bounded. But if you control the VM like in the Loom case, you can expand and shrink stacks as needed, so it is much less of an issue.
My theory on is that it evolved as a consequence of people trying to solve problems on a level they were familiar and comfortable with. E.g. let's say there were C and C++ application programmers who wanted to scale their applications to handle more clients than a OS supported at that point in time. Their solution was to use event-driven OS APIs, and build abstractions on top of that (libuv, boost) that they found comfortable to use. Then we went one level further, and compiler and programming language experts got aware about the problem, which tried to solve it with language extensions (async/await).
I fully confess I'm guilty of this too, by having used/promoted/evolved various async frameworks in the last 10 years. It's a super interesting problem to work on, a bit on the research side, and it feels fulfilling to design some elegant solutions on top of the "i have to async paradigms" problem.
But ultimately the compiler magic on top of callbacks on top of event-driven OS APIs workarounds definitely doesn't seem to be best solution for regular application developers, since the abstractions are very leaky and now application developers need to be aware about how all those things work and work together.
I really like the Loom solution and look forward to it, since it means the application space doesn't have to deal with colored functions anymore. But ultimately I'm wondering in the meantime whether we should rather try to find an OS level fix for those OS level problems instead of trying to work around them on a higher level. There's probably 100x the amount of code written on async runtimes than for the kernel schedulers and IO subsystems themselves, which feels wrong.
Because it's a useful abstraction for managing N concurrent activities. In the threaded world, it's known as fork-join.
I really think C# & Python will be jealous of the languages mentioned and puts them in an odd spot from language design perspective.
Although the coloring problem is only a problem if red functions are harder to use. Not sure marking all functions with "suspend" would make it any worse, besides infecting the codebase.
https://elizarov.medium.com/how-do-you-color-your-functions-...
I am also not sure how this explicit marking would work with interfaces. Can you create an interface that can be implemented by both suspending and synchronous functions?
I do know you can at least specify suspend in an interface method to enforce its implementers to be suspending too. Note that this is e.g impossible in typescript..
You restated the idea of function colouring without further elaborating why it is a problem.
If your thread can afford to block and want to call a suspend function, use `runBlocking`. If your thread cannot block, having `suspend` in the type just saves you from a bug.
If runblocking can't allow non-suspending function then you got a problem because that mean a method implementing an interface can either be suspending or not and therefore for one of two implementation, fail.
`fun main() = runBlocking<Unit> {`
https://kotlinlang.org/docs/composing-suspending-functions.h...
Quoting the post:
> Having to mark asynchronous functions with suspend modifier is a small price to pay, but in return you get better insight into your code.
Also you can do `suspend fun main()`
Did you not read my second paragraph?
It felt like a question more than a complaint.
> create an interface that can be implemented by both suspending and synchronous functions
If interface has a suspending method, the implementations will also be suspending. But you can choose not to make any suspend calls in the method body. Not being able to say "this particular implementation of a red interface is blue" has never bothered me.
https://www.reddit.com/r/rust/comments/7x0icm/regarding_gree...
https://github.com/rust-lang/rfcs/blob/master/text/0230-remo...
> Initially, Rust supported only the green threading model. Later, native threading was added and ultimately became the default.
For green threads the stdlib would have to be designed around them (again) or they'd feel like something bolted on the side with two sets of APIs for everything. This is also the case for async/await but that means green threads don't have an advantage there. If you have to worry about Task vs Thread, make sure you all the right APIs from each, figure out what to do for TLS in Tasks, etc then you may as well just use the one that is more efficient.
In theory, such a system could be implemented on C# with the same sort of gotchas as Java virtual threads. I don't think there's a fundamental design conflict.
https://github.com/openjdk/jdk/pull/8166
1,140 files, (+98,553 −9,862 LOC) and those lines of code are mostly horribly fiddly low level assembly/compiler hacking. This is partly why it took years of development.
Most people don't realize this, especially because in the later stages of course many others contributed, but Loom is in some sense one man's journey. Before he worked at Oracle Ron Pressler spent years writing a library called Quasar which implemented fibers on top of the JVM using bytecode rewriting and some low level hackery with internal APIs. At some point he became available for hiring and Oracle brought him on board, as by that point he was not only an expert in fiber implementations but also the JVM. That was the genesis of Loom.
Something I've learned from following the intricacies of VM development is that what we get is very much a result of hidden human stories as well as technical decisions. Features happen or don't happen on the basis of who was available to be hired at the time, as much as cold calculations of performance impacts. In turn that depends heavily on the vagaries of personal lives. The skills needed to do this work aren't that easy to find on the open market and training takes a long time.
For other VM implementors to do this, and realistically only .NET has the sort of languages where it makes sense (JS doesn't), well, it'd take a long time even if they start today and there's no guarantee of success.
https://github.com/python-greenlet/greenlet
has been available for quite some time in Python (gevent probably being its most used flavor).
i.e.: just because it's not in the standard library doesn't mean it's not available (so, you can actually choose whether you'd like to use async/await or greenlets).
I must say that the main usage I personally had of greenlets didn't have it in mind initially (it was used in an existing application where making the coloring wasn't really feasible as it was a huge app already and adopting green threads was much less work).
There are also advantages in debug ability and performance tracing for threads.
It would be nice to have (virtual) threads that are lightweight and efficient enough that we did not need async for writing highly concurrent software.
I wonder if 20 years from now, programmers will look at async, kind of like current programmers look at segmented memory or manual register allocation —- something that was necessary for performance in a bygone era, but is now not needed.
If anything, this is yet another example of the decades how old concepts still take to become widespread across all major mainstream languages.
How does it know that an operation is blocking on I/O? Is this limited to stuff baked into the language or standard library?
What is the mechanism of suspension and resumption? They mention elsewhere that any platform thread could pick up any virtual thread so I assume they must be storing the stack somewhere. Is there a cost transferring stacks on virtual threads across platform threads? Does this introduce new security implications if there are less OS level restrictions on memory access between platform threads?
What happens in cases where an application has user defined platform threads? How does the system determine what platform threads are available to the virtual threading system?
I think this is probably the right decision for Java and I agree with all of their motivations. I personally like the explicitness of async/await, however I assume Java devs are very familiar with threading in general. I believe this allows an easier path to migrating existing Java code.
You shouldn't need to "transfer" the stack. There isn't really a "platform stack", that is just whatever your stack pointer is pointing at. So it is perfectly fine to allocate many stacks and switch between them within one OS thread just by changing the stack pointer.
Of course if you are allocating a "full stack" then the main question is what is the point? IIUC the biggest cost of threads is the memory allocated to the stack. So unless you are doing something clever you don't get much benefit. There are many approaches here but I guess it is up to the runtime to pick one.
Most libraries use the I/O primitives from the platform standard libraries, so the behaviour is going to trickle out from there. If you aren't using those libraries, the only other way to do I/O would be to use native code such as via JNI, and the runtime would schedule that on a thread pool and so it would tie up an OS thread for the duration of a function invocation.
It will be interesting to see how much of a coloring problem this creates, if any. I guess the great thing about having it be standardised and baked into the language / VM is it will quickly become de facto best practice to make any such library code compliant with virtual threads. And for all its various downsides, the java ecosystem has always leaned away from native code and treated it as a last resort rather than leaning into it like Python etc. So likely it will not be a huge issue the way the coloring problem is in the async/await situation.
Virtual threads are built on top of underlying delimited one shot continuations, and those store the stack. A large part of the engineering effort has been in making this as efficient as possible.
> Is there a cost transferring stacks on virtual threads across platform threads?
A stack always has to be at least partially copied to the carrier thread’s stack, and it doesn’t really matter which OS thread that is. I say partially because initially only the current stack frame will be copied as most threads will have a deep stack compared to the number of frames active in any operation likely to yield.
> Does this introduce new security implications if there are less OS level restrictions on memory access between platform threads?
The security model remains intact. A virtual thread performing some privileged operation will be privileged no matter which OS thread it is run on, and one which is not privileged will not be no matter the OS thread it runs on.
> What happens in cases where an application has user defined platform threads? How does the system determine what platform threads are available to the virtual threading system?
Virtual threads don’t just run on a random OS level thread. They run on a scheduler (commonly a fork join pool) and are effectively a series of tasks fed to that scheduler.
https://cr.openjdk.java.net/~rpressler/loom/loom/sol1_part1....
"All Your Blocking Are Belong to Us"
Not sure how updated the document is. But the section title says it all. (It's a reference to a meme.)
That is, should i organise my code so i create a single executor and use it across many operations, or is it okay to create and destroy executors all over the place?
And are the executors threadsafe - both for use from virtual threads, and from native ones?
As an aside, these are things which is important to know, but which are very rarely documented. We had some headaches a while ago because we wrote code which treated the new HttpClient as lightweight when it is very much not.
Executors tend to be units of management (await()/shutdown()) etc. and monitoring, so that'll be your main reason to use more than just one for everything.
The good news is that long term supported releases got more frequent in recent times and the next one is due in ... September 2023. So there hopefully won't be any delays for people who have to wait for an LTS release.
1. Promise constructor (to turn an existing callback-based or async operation into one that can be blocked with Loom). JS example:
await new Promise(resolve => someApiThatTakesACallback(resolve));
const [result1, result2] = await Promise.all([doSomething1(), doSomething2()]);For the second one, you can use an implementation of ExecutorService[1], which has the methods invokeAll and invokeAny.
Futures in Java are similar to Promises in Javascript, in that they represent a calculation that might not be completed yet.
1. https://docs.oracle.com/en/java/javase/17/docs/api/java.base... 2. https://docs.oracle.com/en/java/javase/17/docs/api/java.base...
`ExecutorService#submit` works with the latter.
I was expecting something like call/cc, or `suspend(Cancellable)Coroutine` in Kotlin. But judging from the list of preview items[1] it seems the only way is to have the callback write the result to a Future and blocking wait on it.
2.1 Promise.all() can be done with `.map(future => future.get())` But it is a bit more complicated than that. [2]
2.2 Promise.any()
This is somewhat relevant http://mail.openjdk.java.net/pipermail/loom-dev/2020-Februar...
[1] https://download.java.net/java/early_access/loom/docs/api/pr...
[2] https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutin... "This function is not equivalent to deferreds.map { it.await() } which fails only when it sequentially gets to wait for the failing deferred, while this awaitAll fails immediately as soon as any of the deferreds fail."
1. Same as with normal threads:
CompletableFuture<Foo> future = new CompletableFuture<>();
someApiThatTakesACallback(future::complete);
Foo result = future.get(); // this can take a timeout
ExecutorService executor = Executors.newVirtualThreadPerTaskExecutor();
Future<Foo> future1 = executor.submit(this::doSomething1);
Future<Bar> future2 = executor.submit(this::doSomething2);
Foo result1 = future1.get();
Bar result2 = future2.get();
For Promise.any:
ExecutorService executor = Executors.newVirtualThreadPerTaskExecutor();
Foo result = executor.invokeAny(List.of(this::doSomething1, this::doSomething2));
Note that in both cases, you probably already have an executor lying around that you can use for this.
This is overkill for a single task, but for your second example you could do a an invokeAny with something like this:
try (var scope = new StructuredTaskScope.ShutdownOnSuccess<String>()) {
scope.fork(() -> doSomething1());
scope.fork(() -> doSomething2());
scope.join(); // or if you wanted a 5 second timeout .joinUntil(Instant.now().plusSeconds(5))
String result = scope.result(); // This will return the result of either doSomething1() or doSomething2(), whichever won the race to finish first.
...
}
Not even close. The entire mobile space — Android and iOS combined — is drastically smaller than just Java alone (not counting Android). You can confirm this by going to job websites that allow you to analyse their postings. Java, together with its sister leading languages, JavaScript and Python, is so popular that it's in a completely different ballgame than most languages.
> Virtual threads are a lightweight implementation of threads that is provided by the JDK rather than the OS. They are a form of user-mode threads, which have been successful in other multithreaded languages (e.g., goroutines in Go and processes in Erlang). User-mode threads even featured as so-called "green threads" in early versions of Java, when OS threads were not yet mature and widespread. However, Java's green threads all shared one OS thread (M:1 scheduling) and were eventually outperformed by platform threads, implemented as wrappers for OS threads (1:1 scheduling). Virtual threads employ M:N scheduling, where a large number (M) of virtual threads is scheduled to run on a smaller number (N) of OS threads
I genuinely don’t know as I have never actually seen them being used, but they always seem to be described as a “lightweight” thread
Or as syntactic sugar for interacting with such a threadpool?
I guess I still haven‘t understood what this gives you as opposed to submitting lambdas to a threadpool. Besides maybe more usefull threadlocals.
Curious to see how Loom concurrency compares to FP languages like Erlang.
The JDK doesn't have this because many of the concerns involved are outside its remit. Java does not define a project layout, a build process, or a way to acquire or distribute libraries.
But the Java world does have Gradle and Maven, which are, for better or for worse, the options to satisfy abledon's desire.
e.g. to make a new console app.... ```
c#
dotnet new console
java
go install extra maven program (if your team uses maven)...
mvn archetype:generate -DarchetypeGroupId=org.apache.maven.archetypes -DarchetypeArtifactId=maven-archetype-quickstart -DarchetypeVersion=1.4