Should I Use Threads?

Besides, I think the author is missing the point. Sharing memory between threads is a huge part of what makes them fast. The difficulty doesn't lie in having to deal with mutexes - in fact I think mutexes and critical paths are easier to work with than message passing. Instead, the difficulty lies in trying to weave all these parallel moving parts together in a way that's both fast and correct. Synchronization primitives are easy, it's the actual synchronization that's hard, and multi-process doesn't fix that.

You have no choice.

Graphics drivers fire up threads. Audio libraries fire up threads. Xinput fires up threads. Your library will be used in multithreaded programs. "I have an executable!" you might object - but this will be rebuilt as a library to accomplish involuntary multithreading if need be, or just loaded like a DLL as-is. Anti-viruses will inject threads. Corporate monitoring tools will inject threads. Debuggers will inject threads. You'll need to be able to initialize COM cleanly without worrying what the impact on other COM APIs the current thread could be using might be. You'll need to turn a blocking API into an event loop or other asyncronous style. Android fires up separate UI and Rendering threads even if you don't want them, and IIRC prohbiits some IO on the rendering thread. WinRT's main UI thread isn't the "main" thread that ran the entry point.

You can avoid firing up a single thread and still end up debugging multithreading bugs. Given that you're going to have to go through the effort of making decent chunks of your code multithreading safe anyways, you might as well use some of the easier tools for the embarassingly parallel parts of your program for some easy performance wins.

A thread is virtual conterpart of the real hardware _thread_ of execution. Without a concept of threads you have nothing in the high-level side to map hardware to. Think again before posting this kind of material, it seems to me vaguely embarrassing.

Now going upper to the place that you probably find the most familar - a user-space. Truth be told if the app is a unikernel there is almost no overhead in switching threads (just switch the stack and update a bunch of registers). So for unikernel they are perfect abstraction, but you may want to add queues, channel etc on top.

Getting to the user/kernel classic OSes - again the problem is lack of trust between user-space and the kernel, the hardware can still switch things quite fast provided we can share most of memory mapping between the two. Nowadays kernel bypass techniques and eBPF running as trusted scripts in the kernel, I do see anything wrong eith threads per see.

The architecture pattern of just spawning one new process or thread per connection is bad only if the said processes or threads are expensive to context switch between, and that cost is a rapidly moving target.

TL;DR: please stop writing and start reading, there is a lot of homework to do here.

This is what I’ve heard through my career: don’t use threads, don’t use locks, don’t use pointers, don’t use languages with manual memory management. The list goes on: don’t use unsafe code, don’t use pointer arithmetic...

The bottom line is that these concepts can be used successfully if you understand the traps. Yes a sufficiently large codebase will eventually suffer from things like data races, memory leaks, etc but if you always take this advice and avoid the dark side of software development you’ll never develop an aptitude for how to deal with such code when you see it the wild.

Instead of saying don’t use threads I would say. Learn the good and bad of a threading model. Know your limits, and study the traps so you can possibly avoid them. Still there are no guarantees but all languages have warts if you look deep enough even high-level languages.

Take Go for example, it has a threaded model with m x n goroutines scheduled over threads. Guess what? Go can easily have data races if you aren’t careful.

It’s still a great language! Still worth learning.

Ok. Give me a non-thread-based solution in Python that can parallelize IO-bound tasks as easily, universally, and naturally as throwing in 3 lines of concurrent.futures.ThreadPoolExecutor and concurrent.futures.as_completed.

OP says:

> Look at your program's CPU usage. If the sum of your threads is <100% of one core, then there is no reason whatsoever for you to use threads.

Attacking a CPU-bound problem with parallelism is only one reason for using threads. Concurrency (even on one processor) has benefits.

> The shame is in believing ourselves super-human, and reaching for a tool that we don't need, in full knowledge that we're likely to shoot ourselves in the foot with it.

Although synchronization primitives require care, they're not that hard. You don't need super-human powers.

I took some time to understand pthreads and apply them to example programs. For a careful and balanced overview, see my article:

https://begriffs.com/posts/2020-03-23-concurrent-programming...

Threads are a fine tool for many jobs. You just need to understand what you are doing. Reduce your shared state. Use locks when more than one thread may write data simultaneously. Use queues/channels for message passing. Profile your lock contention when things get slow. Learn a functional language, so that you get the hang of writing no-side-effect functions and using immutable data structures. This will immensely help you with writing concurrent code.

You should use threads.

Sure a pipe works in the unidirectional case, but what if you need to communicate back? Signals are a thing, but they're worse than threads. Multiple processes and shm_open is a thing, but is that really easier than multithreading? (You still need mutexes, and such, but now they're across processes... yuck!).

Also, for many use cases, "#pragma omp parallel for" is often easier than anything else.

For instance, of a spreadsheet computing values.

A more common use-case, however, is of trivially parallelizable problems which are completely isolated. Consider Monte-Carlo simulation for instance, where the thread hand-off to workers is done by a queue.

Making categorical statements like this political and unhelpful, and is one of the things that annoys me the most about IT culture.

(I should also note that the tooling for debugging and analyzing multithreaded programs has gotten much better recently, and claiming that language designers are lying to you and pinning the blame on runtime bugs is somewhat bogus since such bugs are clearly way outside the realm of why you should not use something for almost all applications, and affect non-multithreaded programs as well.)

"Should one use arms when swimming? NO! Because that can make you swim faster, and that's dangerous because you'll get to swim where you wouldn't with the arms bound on your back."

Explanation: if you haven't learned and understood enough of computer architecture, computer programming, how the operating systems work, how the libraries you use work etc, yes, please, don't use threads. Most with that background, who do simply lack the required knowledge to use them.

And if you have knowledge and experience and there is a real problem having a reasonable solution when threads are used, you don't need a site like above.

Moreover, one can acquire both knowledge and experience. But, that process, I'm quite sure, does not depend on clicking on a site having the content like the above and believing it.

There's a lot of reasons why you want threads though that aren't your CPU hitting 100%. Less janky UI for some systems is one of them.

You’re doing a core–maxing CPU–bound computation and you want your UI to remain responsive at the same time.

Splitting the application into a master process and child workers does not mean your app can magically distribute its work across machines, or gain performance from doing so.

It might sometimes be easier to convert such an application to a distributed setup, as at least it contains some concept of IPC, but I wouldn't make any guarantees about that.

The proposed solution doesn’t work for every multithreaded environment, e.g. mobile. You should instead work with threads at the level of abstraction that your skill supports and your spec requires.

Some aspects of programming are complex and unintuitive, and that’s OK. If we just wall off those options then–absent a CS breakthrough–a whole class of programs would not exist.

> On a long enough time-scale, all programs will be proven incorrect. All code has bugs, including your compiler and language runtime.

That's a bit of a false dilemma. Kernel bugs exist too.

> then you would be infinitely wiser to use an event loop or coroutines.

Coroutines... if you're lucky that your PL of choice supports them in a non-hacky way. C# kind of does with async/await (it's customizable how they behave). Event-loop replaces one evil with another; rewriting otherwise sequential code as a series of callbacks, or (better, somewhat more maintainable) a state machine.

In both cases, one problem is replaced with two others: The first is you have to be careful to not block for a long time in event handlers (or even accidentally invoke blocking I/O that may be hidden in some library you happen to be using). The second is debuggability if your language and tooling doesn't have native support for coroutines; look up "stack ripping".

And then, to top it all, he forgot to tell you that most event loop frameworks allow you to run an event loop on each core, i.e., multithreaded, so you have both events and threads.

> A thread has access to the same address space as other threads in your process, and can do whatever they want with it.

The same holds with any event handler invoked by the event loop. An event handler can just as easily corrupt data belonging to a "wrong" session and the problem will manifest itself 10 minutes later.

> You have to take extreme pains to avoid accidentally doing the wrong thing. You'll probably mess it up, and the symptoms will show up in an unrelated part of the program 10 minutes later.

These sentences assume that whoever is reading this is an ignorant fool. There are tons of books and papers, which have also trickled into mainstream PLs, that teach about writing well-structured and safe threaded code, i.e., there exist patterns for many typical problems.

Actually, I'd argue that with Java and C#, and to a lesser degree modern C++, it's actually _easy_ to write multithreaded code if you think about structure/design first.

You're entering "experts only" area when you start thinking that "mutexes are expensive" and to replace them with interlocked operations.

> But, perhaps your problem is CPU bound. In that case — you still shouldn't use threads! Consider using multiple processes instead.

Oh yes, "great" advice. So you have to decide how these separate processes should communicate.

If you're going for shared memory, you're back to square one, except that sharing is cumbersome because data is mapped to different addresses in different processes, so pointer-based structures can't be shared... At least C++ has a Boost library that mimics the usual containers (vector, map, set) by using special offset-based "pointer type" so they can be safely shared. And you _still_ must synchronize access to shared data.

If you go for the no-shared approach, well, you have to send data through socket, pipe, or a message queue. So you have to design some marshalling and communication protocol (or use a library), but the major drawback here will be performance: each send/receive requires a user-kernel transition, AND, copying data between user- and kernel-space. Twice (once from sender -> kernel, the other time kernel -> receiver).

If the MQ implementation is clever (like on Solaris), the communication buffer will get mapped into both processes so you can get away with copying the data only once, provided the MQ allows you to "peek" and consume data without copying it out from the shared buffer first.

> This is not only much more robust, but it scales better, since you could distribute these programs across multiple computers later on.

This is presented as if distributing across machines is simple matter. What is not mentioned is that communication between processes on the _same_ machine will NEVER fail. As soon as you have network, random latency increases or network outages will kill performance or expose a whole new class of bugs.

> Second, your language designers are lying to you. On a long enough time-scale, all programs will be proven incorrect. All code has bugs, including your compiler and language runtime.

Sure. So go and write in assembly.

https://en.wikipedia.org/wiki/Green_threads

Threads main use case IMHO is responsiveness. You might want to avoid creating new ones for no reason. But in common desktop or mobile app you’ll find yourself in need for running some code NOT on the main thread (unless you love your users feeling the app is stuck).

I have certainly had some horrible experiences debugging multi-threaded C++ programs, and using non-blocking I/O made them safer and easier to reason about because only one thing happens at a time. The non-blocking I/O programming paradigm, however exposes unnecessary "plumbing" to the programmer.

What we really want is a concurrency model where we can handle each request as if it was the only thing happening.

To the extent that multiple requests need to share resources, we need ways to safely access state with concurrent access. That might be a relational database, mutexes, or software transactional memory.

I am currently in the middle of updating a large application written in Twisted Python. async/await and promises can certainly be complex. The Flask approach of using one thread per request is easier to understand. Everyone in the Javascript is in love with async/await, but I can see where that ends: just look at what Twisted was doing 10 years ago. And see that it basically failed due to complexity.

The model I like best is Erlang/Elixir. The runtime handles I/O using non-blocking I/O and threads to be efficient. The programmer works in terms of lightweight actors (called "processes", more like Java "green threads") that communicate using message passing. Each individual Erlang process only does one thing at a time.

As Joe Armstrong, one of Erlang's inventors, says: "We do not have ONE web-server handling two million sessions. We have two million webservers handling one session each." https://joearms.github.io/published/2016-03-13-Managing-two-...

Message passing handles concurrency without locks. If multiple client processes send update messages to a server, the server simply pulls them from its inbox in order and updates the shared state it is responsible for. The VM schedules processes preemptively, handling millions without breaking a sweat. It's safe, because one process can't mess with another process's memory. It is easy to reason about and debug.

It easily takes advantage of all the CPUs on the server. You can run a single VM instead of breaking up your application into multiple servers, each of which only uses one CPU (via containers or OS processes) and trying to coordinate between them. The VM has a built-in key/value store with efficient concurrency. It's like Redis but without the serialization overhead, so a read takes 1 microsecond.