Feel free to burn me at the stake.
Feel free to burn me at the stake.
I think a large part of that for me is because early in my career I got involved in Unix kernel work, first writing device drivers and later doing Unix ports, including both porting a swapping Unix to paged hardware (and rewriting pretty much all process and memory management from scratch) and porting a paging Unix to swapping hardware.
That was followed by a few years largely doing embedded stuff on systems too small to have operating systems or memory management units. For those I'd write a small kernel that supported threads and scheduling (sometimes cooperative and sometimes preemptive) and then write the embedded applications much like I would have written them on Unix, using threads instead of processes.
So for me threads are easy and things like async/wait seem overly complicated and less clear because I came at concurrency from the operating system side of things and the hardware side of things (e.g., dealing with interrupts) where thread-like thinking is natural and perhaps required.
Maybe it is conceptually harder for people who come at it from the application side of things?
On the app side, "I want to do this thing that might take a while without blocking the UI, and then do something with the result, and I have zero need to communicate with the code that's running the task beyond giving it its arguments and getting its result back," is the most common use case. Task parallelism is tailor-made for that kind of problem.
Using a dedicated thread, on the other hand, gives you some raw material you can use to solve that problem, but leaves you to handle all the messy details on your own. And doing it the easy way - one short-lived thread for every thing you need to do in the background - is expensive, while doing it the correct way and implementing a thread pool is essentially just committing to Greenspunning your own personal task parallelism library.
E.g. GUI code -> async, because you must do everything on the UI thread. Heavy lifting in a GUI app -> threads. Traditionally you'd do this with a huge mess of callbacks, but these days it's reasonably easy to just write the GUI portion using async and have background computations run in threads signal their results through futures into the async code, which makes everything so much easier.
There are times when one might want async executors like node's event loop or rust's tokio/async-std, but in other situations one might want os threads or "green threads" with a RwLock or Mutex.
What about a few long running threads that use async internally? ;-)
Parallelism and concurrency are orthogonal concerns. Multiple threads can run on the same core, while concurrent awaits can run on separate cores (depending on the language). Or the opposite!
Threads definitely solve wasted time waiting on I/O, but OS-level threads are a very limited resource in many environments, which can cause problems more quickly in programs with unbounded concurrency, and they tend to have a lot of startup cost, so they have to be created judiciously to avoid hurting performance.
Languages like Go and Elixir give the programmer access to unlimited lightweight “threads”, and that works great for concurrency, without requiring developers to re-color their functions.
One additional note is that some OSes (including Linux) have struggled to support true nonblocking File I/O for many years, so you literally have had to use multiple OS threads for I/O concurrency... one for each file operation in progress. Otherwise you would block the async executor! Languages often handle this implementation detail for the programmer, so you just don’t realize you’re using a thread-per-await sometimes.
(io_uring seems to finally solve this in a meaningful way on Linux)