Hacker News Reader

> Virtual threads are nice for unblocking legacy code but they aren't without issues. There are better options for new code with less trade offs on the jvm as well.

The designers of Project Loom would say the exact opposite. The whole push behind Project Loom and similar models (Go's oft-praised "goroutines" runtimes being another one) is motivated by Threads being a much better fit for async behavior in a fundamentally procedural language like Java or Go than promise-based frameworks like async/await.

The whole motivation of Project Loom is to make the simple thing (spawning threads to handle blocking IO) the fast thing as well (by actually replacing the blocking IO with efficient async IO OS calls and managing the threads internally). Project Loom will be considered a full success if the next generation Java web server does something akin to "new Thread(() -> {executeHandlerFunc(conn); }.Start(); " for each incoming connection, just like the Go built-in web server.

I don't know.

My understanding is that that highest performance webserver is nginx. And it uses async internally.

IMO, virtual threads is a better general purpose language feature because it avoids function coloring and is generally easier to reason about, but it may not result in the highest performance Java webserver.

NGINX is a native C implementation, so it has to be carefully written to use the OS's native high-performance IO and native OS threads.

The purpose of project Loom is to abstract that away from Java application code. The runtime can use the most efficient IO for the given platform (ideally io_uring on Linux or IOCP on Windows, for example) even if the application code calls the old blocking File.Write(). The application can then use simple APIs and code patterns, but still get massive performance.

With Loom, you can easily have 20,000 virtual threads servicing 20,000 concurrent HTTP requests and each "blocked" in IO, while only using, say, 100 OS threads that are polling an IOCP. A normal Linux box can typically only handle around maybe 1000 threads across all running processes.

Servicing 20,000 concurrent requests on a single box where somehow threads are the bottleneck, is that not a problem that approximately no one has?

Most application webservers (by default) handle one request per thread. For mostly IO bound stuff (which many projects are), it makes sense to me that threads become a bottleneck in relatively ordinary scenarios.

The scenario where your IO could handle way more than a thousand concurrent requests if only the thread overhead was reduced? When does that ever happen?

Each OS thread costs memory. With the version of Java I have, the default is to allocate 1MB of stack for each thread. So, 10,000 threads would require 10,000 MB of RAM even if we configured ulimit to allow that many threads. In contrast, asking the kernel to do buffered reads of 10,000 files in parallel requires much less memory - especially if most of those are actually the same physical file. Of course, they won't be read fully in parallel.

For example, this program:

  var threads = new Thread[20000];
  for (int i = 0; i < 20000; i++) {
    threads[i] = Thread.ofVirtual().start(() -> {
      try {
        Files.copy(FileSystems.getDefault().getPath("abc.txt"), System.out);
      } catch (IOException e) {
        System.err.println("Error writing file");
        e.printStackTrace();
      }});
    }
  for (int i = 0; i < 20000; i++) {
   threads[i].join();
  }

Run as `java Test > ./cde.txt` takes about 4.5s to run on my WSL2 system with 2 cores, writing a 2 GB file (with abc.txt having 100KB); even this would be within the HTTP timeout, though users would certainly not be happy. Pretty sure a native Linux system on a machine beefy enough to be used as a web server would have no problem serving even larger files over a network like this.

1. You are not solving a real problem. The use case you describe (basically a CDN) is already exotic, the scenario where such a system would have already been implemented with Java and its basic IO seems implausible.

2. You did not compare against fewer threads to see if threads are actually the bottleneck rather than IO. Also, all your threads are competing for stdout.

>The designers of Project Loom would say the exact opposite.

Sure, but then again the designers of circa 2000-2010 J2EE also thought the verbosity and over-engineering was a good idea.

There might be some justification for comparing any one particular thing to the worst possible particular thing if those things have something in common. The only feature the two things you picked have in common is the word 'java'.

Also have in common the "appeal to authority": (the designers) as arbiters of good judgement

Appeal to expertise. Appeal to authority is a falacy when the authority is not an expert in the requisite domain. eg: we don't care what a policeman thinks about astrophysics, we do care what the astrophysicist says.

J2EE started as a Objective-C framework, before being rewritten in Java.

I think it's not that black and white. Clearly they made a choice to be backwards compatible. Not because Java Threads have a nice API (not even close) but because a lot of legacy code that will never be changed uses it. Including all the ugly bits that you shouldn't be using. Like a lot of the low level synchronization primitives that date back to the early days of Java. It's an impressive bit of work but they made some compromises to make things work. A new API would have been easier, would have had less overhead, and be nicer to use. But backwards compatibility with legacy code was a big goal.

It mostly works fine and it's an impressive bit of engineering. But it has some really ugly failure modes in combination with hacky legacy code designed for real threads. So, you can't blindly assume things to just work. Hence the deadlocks.

Many Java servers already work the way you outline. It's just that they are a bit tedious to use with the traditional Java frameworks. Which is one reason I like using Spring's webflux with Kotlin instead. Just way nicer when it's all exposed via co-routines.

There are two separate choices. One is the choice of whether to implement green threads in the JVM at all, or whether to use async/await, or some other type of concurrency primitive. The other is whether to expose the new concurrency primitive using a new API or an existing one.

You could say the second choice, the specific API, was done, at least to some extent, for backwards compatibility reasons. I wouldn't agree, but I think there is at least some argument to be made. Here is one of the designer's explanation [0]:

> We also realized that implementing the existing thread API, so turning it into an abstraction with two different implementations won't add any runtime overhead. I also found that when talking about Java's new user mode threads back when this feature was in development, and back when we still called them fibers, every time I talked about them at conferences, I kept repeating myself and explaining that fibers are just like threads. After trying a few early access releases of the JDK with a fiber API, and then a thread API, we decided to go with the thread API.

However, the choice of adding a new concurrency primitive to Java in the form of green threads instead of others was very very clearly not done for backwards compatibility's sake. Ron Pressler (who is active here as 'pron') has several talks on the advantages of green threads over async/await that you can look at [0][1]. The designers of Go also had the same belief, and also chose to add green threads as the fundamental built-in concurrency primitive in Go, obviously not for backwards compatibility reasons in their case.

[0] https://www.infoq.com/presentations/virtual-threads-lightwei...

[1] https://www.youtube.com/watch?v=EO9oMiL1fFo

The lack of support for synchronized isn't a fundamental or hard limit, it's just that the HotSpot implementation is complicated for performance reasons and they put off rewriting that code until later. They're indeed working on that now and in some future version I guess wait/notify and synchronized blocks will start to work. After all, you can easily transform such code into an equivalent that does work.

There are ways to find problem sections without having to trigger a full deadlock: https://openjdk.org/jeps/444

  The system property jdk.tracePinnedThreads triggers a stack trace when a thread blocks while pinned. Running with -Djdk.tracePinnedThreads=full prints a complete stack trace when a thread blocks while pinned, highlighting native frames and frames holding monitors. Running with -Djdk.tracePinnedThreads=short limits the output to just the problematic frames.

Was curious what it is "jasync". And man it hurts me to see documentation like this (when compared to classic javadocs)

https://github.com/jasync-sql/jasync-sql/wiki/API-Overview

From project WIKI (https://github.com/jasync-sql/jasync-sql/wiki)

Synchronized blocks are not a problem. Synchronized blocks that later don’t unblock the thread may sometimes be.