Java virtual threads hit with pinning issue | Hacker News Reader

Java virtual threads hit with pinning issue

infoworld.com

49 pointscempaka28 comments

This is not news, it's documented in the official description of the virtual threads: https://openjdk.org/jeps/444

OK. Good.

I’m glad someone else said this, since it means I’m not crazy. I was reading that thinking, man, this sounds like the way it’s outlined to work? Where’s the new issue?

It's news if you are are uneducated about synchronization/threading in general (it's extremely difficult to do well so it stands to reason most people won't even understand the issues).

It's also news that they are trying to provide some abstraction to help. Which is great, although as usual I suspect it will need to be applied per situation, not a one-size-fits all.

Nah, it's not normal or really documented. Normally when languages have async, they end up with dual primitives - the synchronous one and the asynchronous one, and everyone learns that if they do the blocking operation on the async thread, they deserve their deadlock. Generally you then end up with two variants of the API, with the standard function colouring problem where you can call lockBlocking() from a non-async context, lockAsync() from an async, and woe betide you if you get mixed up. Ideally your language can help you avoid these issues sometimes (e.g. with Rust you can't hold a non-async lock over await points).

Java instead did a big thing where they made all the primitives work fine in both cases with one API, something which is honestly really hard and really reflects the level of thought put into modern Java features. But there's a long tail of stuff that's still getting cleaned up (e.g. various weird I/O apis), and honestly I think it _is_ weird that an actual language keyword made it into the long tail.

It's extra fun that it's actually quite hard to hit performance problems as a result of this, as the JVM will actually detect that it's getting to this problem and boot up threads to compensate.

It's not news only if you carefully read that JEP (and even then, it's misleading, it says "Pinning does not make an application incorrect, but it might hinder its scalability" as if using "synchronized" could only lead to not getting the full performance advantage of virtual threads; there's no mention of potential thread starvation or deadlocks). But most people are not reading JEP pages; most people are getting their information from release notes and news articles, and AFAIK these didn't mention the important caveat that virtual threads can misbehave if anything waits within a synchronized block (which is very common, synchronized blocks and methods exist and are widely used since the early days of Java).

For instance, a quick web search led me to a release notes page (https://www.oracle.com/java/technologies/javase/21-relnote-i...) and a news article (https://www.infoworld.com/article/3689880/jdk-21-the-new-fea...), neither mention anything about synchronized blocks and methods. In fact, the later one says "Previously previewed in both JDK 20 and JDK 19, virtual threads will be finalized in JDK 21.", as if everything now works as expected with virtual threads on Java 21. Which is clearly not the case, if synchronized blocks are enough to make it misbehave. IMO, we can only say virtual threads are really "finalized" once synchronized blocks work as expected (releasing the carrier thread, like AFAIK ReentrantLock already does) when running on a virtual thread.

Using "synchronized" by itself and calling anything has the potential thread starvation or deadlocks. Virtual threads only makes it worse.

Kotlin has mutexes that presumably are compatible with coroutines:

  kotlinx.coroutines.sync.Mutex

Until Java sorts this out, it could be its "Rust async" watershed moment.

You can use Concurrent Locks from the standard library. And it has been communicated that you should not be using low level concurrency constructs for years.

But people just… don’t.

> And it has been communicated that you should not be using low level concurrency constructs for years.

Where has this been communicated? The first I heard that you shouldn't use synchronized statements or blocks was in these virtual threads discussions; and the popular IDE I use for Java at work doesn't flag synchronized methods or blocks as something to be avoided. The only thing I had heard about it was that you should avoid synchronizing on objects exposed to users of your library, since it would make it possible for code outside your control to also synchronize on that same object (but IMO synchronizing on arbitrary objects you got from somewhere is itself a bad practice, you should only synchronize on objects you control). And obviously, synchronized methods or blocks do not have the "fair" mode, which might be necessary (or good) in some cases.

(And before Java 9, stack overflows within the implementation of some concurrency constructs could lead to unexpected behavior, see https://openjdk.org/jeps/270 for details; this was never a problem with synchronized methods or blocks.)

I started Java over a decade ago and by then no one was writing synchronized blocks when doing multithreaded code.

You’re being downvoted, but I haven’t done Java programming in over twenty years and even I knew that!

I've been coding in Java for a bit longer than that, and in my memory it's always been known that ReentrantLock is _faster_, but synchronized has better language level support (as in, the language won't let you forget to unlock, whereas with locks you need to remember your finally statement). And then, unlike ReentrantLock, synchronized uses no extra memory, which is good for things that end up being uncontended, and then in both cases, the performance doesn't matter if the lock is uncontended, which most of mine were.

There of course are things like ErrorProne that statically check that you unlocked your lock, but there's still bugs possible.

I hope anyone who learned the horrible construction of double-checked locking would have used an alternative.

I think this one's weird because the language puts so much effort into making it hard to hit this that you won't notice until it's a gigantic problem. For me, this was enough of a problem that I wrote a Java agent which patches the method which blocks the carrier thread to just throw (ByteBuddy makes it easy!).

Like, it makes sense that when you synchronized on an object, there's room for contention. It seems weird that synchronizing on an _uncontended_ object can cause contention. But that's what this is. The behaviour is that you have target numCores carrier threads, and if someone synchronizes on an object and then does a non-blocking sleep, it's now blocking, because synchronizing upgraded the non-blocking I/O to blocking.

So basically when you hit this issue, it's because not only has the bad thing been happening, it's also been happening badly enough that all the compensation mechanisms have failed.

It's just weird that a whole language level keyword behaves this badly.

Alan Bateman's FOSDEM presentation that explains this issues and some solutions in the meanwhile:

https://inside.java/2024/02/17/virtual-threads-next-steps/

By the way, there was a recent HN discussion on a real-world example of deadlock caused by `synchronized` pinning ("We switched to Java 21 virtual threads and got a deadlock in TPC-C for Postgres") [1].

[1] https://news.ycombinator.com/item?id=39008026

The specific edge case which is most annoying here is that locking a Java object with synchronized converts all the non-blocking calls that Loom does further down the stack to be blocking ones, which then lock up your carrier threads. So you get this action as a distance thing.

E.g.

    for (int i = 0; i < 500; i++) {
        newVirtualThread(() -> synchronized (new Object()) {
             Thread.sleep(100_000);
        });
    }

will blow up your JVM (modulo some compensation mechanisms that work definitely kinda) and that's odd.

I didn't watch the preso (TL;DW) but isn't this just a case of good old fashioned Priority Inversion? https://en.wikipedia.org/wiki/Priority_inversion

If so, it's pretty much an intractable problem, and is the reason (for example) why Solaris abandoned the two-level thread model and went back to a single level one? https://flylib.com/books/en/2.830.1.14/1/

But then again, as I said, TL;DW ;-)

I think this is a bit more subtle and nasty than bog standard priority inversion. Specifically, in present virtual threads, if you take out a lock with synchronized, if you then do a normally-non-blocking asynchronous API call while you have the lock, you block one of your very few OS threads because the virtual thread can't now be migrated off the OS thread.

IMO the compound thing is what makes it be nasty. E.g. you have a function `doSomething` which does some RPC, and that's all nicely non-blocking. But someone called map.computeIfAbsent(x, k -> doSomething(k)), and that uses synchronized on the inside so now your non-blocking API calls all magically became blocking, no further action required.

Thanks for the explanation and - Ugh :-(

Reply on news.ycombinator.com