Hacker News Reader

Comment by atemerev | Hacker News Reader

Yes, might have worked in 1995. Now, however, when even your lowly phone has 4 (or more) processor cores and a full-fledged GPU...

Learn concurrent programming techniques — or perish. Threads and sync primitives are low-level, but important, and you have to understand them to figure out what compromises and biases were taken in higher-level models.

And, frankly, it isn't that bad (debugging existing code is bad, but playing with monitors and semaphores and critical sections is easy, until code is small and isolated).

View on HN

Yes, and no. Concurrency and parallelism are important, but threads (lightweight shared memory processes) are not the only solution.

Successful, popular concurrent platforms like Erlang/OTP, Nginx, and node.js, eschew threads in favor a single-threaded, async/non-blocking code model. These platforms simplify application-level code by avoiding the issues of thread synchronization and contention in a shared memory space, and instead provide/require isolated processes to exploit CPU-level parallelism.

The threading "isn't that bad" viewpoint generally comes from a limited understanding of the things that can go wrong - for instance, add "memory barriers" to the list of things to understand.

There's a good explanation of the problems with a common Java idiom "double-checked locking" at https://www.cs.umd.edu/~pugh/java/memoryModel/DoubleCheckedL...

Of course, when things get complicated with lots of interactions between processes, or significant amounts of computation in a single-threaded process, some platforms require the user to manually yield, etc., basically trying to re-create preemptive threading provided by the OS.

Nginx and Erlang single threaded? Uhhhhh, that's just not true.

Nginx and Erlang most certainly use multiple threads by default. They both use thread pools. I'm not sure about Erlang but Nginx has a single master thread that does async select on the main socket. Once it receives a connection it hands off to worker threads. This is a traditional multithreading setup with control/worker threads.

Node doesn't support threads because JavaScript doesn't. Performance would certainly be better for many workloads if it followed a traditional multithreading approach like Nginx. Async != Single threaded. They're not related concepts. You get the best performance using both non-blocking calls and multithreading , an approach used by Nginx and Netty. They both use nonblocking IO on the main thread and thread pools.

Node requires you to manually yield by calling Response.end() because it doesn't support multithreading. This basically duplicates OS level pre-emption manually and is my single biggest criticism of node. The worker/master thread model allows the runtime to manage worker resources or just defer to the OS in a flexible way.

You are confusing threads with processes.

Processes are isolated and do not share memory. Threads within a process share memory. For example, see https://msdn.microsoft.com/en-us/library/windows/desktop/ms6...

Nginx is single-threaded, multiple process: "Each worker process is single-threaded and runs independently" https://www.nginx.com/blog/inside-nginx-how-we-designed-for-...

Erlang code is single threaded. Each Erlang "process" is an isolated, share-nothing single sequence of instructions, and you don't use semaphores, locks, or critical sections in writing Erlang code. The BEAM VM is implemented as a multithreaded process, but this is not exposed to the Erlang code, and Erlang code can be transparently distributed across multiple instances of the BEAM VM (processes) even on separate hosts.

You're correct on node.js, and "basically trying to re-create preemptive threading provided by the OS."

Nginx recently implemented thread pools for blocking operations. The distinction between threads and processes depends on the OS.

The "nothing shared" model used in Linux forking actually shares the underlying memory until the processes diverge (copy on write). The difference between this and threads is that threads expose the shared memory by default.

IMO the difference between threads and processes is somewhat academic.

I don't know much about BEAM

> IMO the difference between threads and processes is somewhat academic.

Shared memory vs. isolated memory is a huge difference. You use completely different sync/protection/IPC approaches with each, so programs look very different. And the copy-on-write optimization of logically isolated memory doesn't effect that. The beauty of it is that it speeds up forking and saves memory without changing your IPC-based programming model.

> IMO the difference between threads and processes is somewhat academic.

At a linux-implementation level the difference is almost irrelevant. At a code level the difference between your variables being shared by default versus "thread"-local by default is huge.

> Erlang code is single threaded. Each Erlang "process" is an isolated, share-nothing single sequence of instructions, and you don't use semaphores, locks, or critical sections in writing Erlang code. The BEAM VM is implemented as a multithreaded process, but this is not exposed to the Erlang code

The same is true for PHP.

No wonder Facebook developed HHVM for PHP and also uses Erlang with WhatsApp and other parts.

Well, Whatsapp servers have always been Erlang-based (well before the acquisition by Facebook) but your point stands.

"Nginx has a single master thread that does async select on the main socket. Once it receives a connection it hands off to worker threads"

Nginx has a master process and worker processes. Master process doesn't do anything, but manage worker processes in case they die or need to be restarted on configuration change. A worker process runs a single event loop that accepts connections. It doesn't hand off them anywhere, but just schedules connection handlers to be run in the same event loop. Worker processes don't share memory (almost, there are some sysv shared memory segments for a few things, but this is irrelevant). Thread pools in nginx were added only relatively recently to do blocking file I/O on linux, which is kind of slow otherwise in some scenarios.

Mentioned in another response, but Node.js does indeed use threads/threadpool... It's used internally for all async IO, and by a number of binary modules as well.

Node is single threaded only in Javascript land. Internally it does use threads !

>Successful, popular concurrent platforms like Erlang/OTP, Nginx, and node.js, eschew threads in favor a single-threaded, async/non-blocking code model.

Ugh. As someone suffering with a node.js project now I'm continually chafing under this limitation. It's great if you're making a simple CRUD website, but you'll be beating your head against the wall if you have a problem domain with centralized resource sharing that prevents you from spamming processes willy-nilly.

And node.js code is ugly.

>The threading "isn't that bad" viewpoint generally comes from a limited understanding of the things that can go wrong - for instance, add "memory barriers" to the list of things to understand.

Actually, threading isn't very hard to do right. It's more a question of choosing a reasonable architecture. Sure, you can get yourself into a lot of trouble trying to interleave threads in an attempt to use multiple processors to speed up a single operation. Don't do that.

>There's a good explanation of the problems with a common Java idiom "double-checked locking" at https://www.cs.umd.edu/~pugh/java/memoryModel/DoubleCheckedL....

Yes, and if you read to the end of the article you'll see the problem it's addressing was fixed 12 years ago in Java 5.

As someone who's implemented reliable, massively parallel threaded systems in Java, when I hear people say threading is too complicated to be used safely I feel like a blacksmith who's just been told horseshoes are impossible to make even though he's been making them his whole career.

As to the ugliness of node.js code, I find that if you're using a transpiler (babel) or the latest 8.x, that async/await with promises tends to clean things up a lot.

Yeah, the Koa2 framework for Node.js is a beautiful way to handle requests in my opinion and it's built on async/await functionality from the ground up.

Yeah, re-reading my comment above, it feels like I'm saying that single-threaded async is the one-true-way, and I don't think that. Both threaded and async/CSP models are useful, and both do have weaknesses.

I've also implemented reliable, parallel, `threaded systems in Java. But I've also found defects in code that I thought had been thoroughly exercised, especially when it has moved to a new environment or new level of parallelism.

There's often "one more thing" that can cause problems, and generally it can be resolved by one more concept or workaround (like adding "volatile" in Java 5).

And I'd suggest Erlang/Go, which are designed for CSP-style concurrency, for cleaner code than node.js. JavaScript-based platforms often feel like, "Wow, you can even make JavaScript do X" rather than, "X is really natural in JavaScript."

> Successful, popular concurrent platforms like Erlang/OTP, Nginx, and node.js, eschew threads in favor a single-threaded, async/non-blocking code model.

Erlang, the language, is threading model agnostic, but the main implementation has been, by default, M:N threaded on systems with multiple logical processors since, AFAICT, the R12B release in 2008. So characterizing it as single threaded is inaccurate.

Erlang the language provides single-threaded processes that do not have shared memory.

The BEAM VM hosts Erlang processes in a multi-threaded single process. This is an implementation detail/performance optimization.

The Erlang processes don't care if they are hosted in a single OS process or multiple processes or even multiple host machines. Erlang-the-platform simplifies application code by providing a single-threaded, share-nothing, async/non-blocking code model. (Edit: Yes, dfox, it's probably more accurate to say, Erlang provides a synchronous, blocking code model, since that's what the Erlang programmer sees)

> Erlang the language provides single-threaded processes that do not have shared memory.

Erlang processes share memory, and some message sends (local sends of binaries > 64 bytes in size) are implemented by sharing references to shared, reference-counted, objects. This is safe because (not considering the process dictionary, which isn't shared) Erlang processes have no mutable state, and all that is shared is immutable values, so if the implementation wanted to, all sends could be references to shared memory (for small objects, the indirection and reference counting overhead isn't worth the memory savings, so its not done.)

The Erlang processes don't care if they are hosted in a single OS process or multiple processes or even multiple host machines.

It's not possible to write a high-performance program without a good understanding of the costs of primitives. If something might vary in cost by several orders of magnitude, it's quite hard to design for.

Network transparency has never been a good fundamental for well-performing application architecture. Networks are inherently far less reliable than shared memory, and have massively more latency, particularly on the tail end of the distribution.

From programmers view, synchronous blocking IO and IPC is the natural way. That it is mapped by the VM onto some asynchronous/event-triggered loop is also an implementation detail.

Message passing in Erlang is asynchronous and this is one very important detail, that makes it good at concurrency. And equivalent to event driven programming to some extent.

Message passing in Erlang is asynchronous in the sense that messages can be queued, but from the point of view of the process receiving message is synchronous function call that happens on well defined place in the process, in contrast to truly asynchronous message receipt which would interrupt the process/start new thread at arbitrary places in code, which is the model used by some microkernels and similar systems (this can be emulated in userspace by starting background thread which runs some event loop, for example SDL, and optionally ALSA, uses this pattern for audio I/O).

In it's original application space the primary apeal of Erlang (as well as Ericsson's custom CPUs in their exchanges) was that user's interaction with the system (eg. one phone call) can be implemented by one thread/process running normal sequential synchronous code as opposed to as state in some complex asynchronous state machine (ie. what is done in node.js) that is orders of magnitude more complex to program and understand. Another solution to this problem involves inventing some monads-and-arrows-reinvented (async/await...) machinery that hides the state machine away.

Multi-threaded programs have to be concerned with low-level details like memory ordering and barriers only when they do synchronization without using OS-provided utilities or when they implement synchronization wrong. pthread_mutex_lock() is both memory and optimization barrier and that is probably all one has to know for vast majority of MT code.

> when they implement synchronization wrong.

Yes, but there are lots of ways to implement synchronization wrong because you don't understand memory barriers.

For instance, Doug Lea's article linked above includes this very simple Java code, which breaks because it wasn't aware of memory barriers:

  // Broken multithreaded version
  // "Double-Checked Locking" idiom
  class Foo { 
    private Helper helper = null;
    public Helper getHelper() {
      if (helper == null) 
        synchronized(this) {
          if (helper == null) 
            helper = new Helper();
        }    
      return helper;
      }
    // other functions and members...
    }

[dfox: can't reply, guess we're nesting too deeply. Per the above, the simple rule has to be modified to "do not touch OR READ shared state outside of relevant critical section". Easy to make errors...]

There is simple rule for that: do not touch shared state outside of relevant critical section.

To some extent the cited problem is unique to Java, because the whole monitor concept just creates confusion about what synchronized really means (and also creates significant overhead by making literally everything usable as mutex). Pretty common misconception is that synchronized is akin to some kind of read-write lock (in which case the code above would work as expected).

AFAIK Ericsson never had custom CPUs.

> Successful, popular concurrent platforms like Erlang/OTP, Nginx, and node.js, eschew threads in favor a single-threaded, async/non-blocking code model.

All these platforms have some problems. E.g. Nginx is PITA if your processes need to communicate with each other. Erlang/OTP has this nasty mailbox problem (O(N^2) behavior of the selective receive). The cooperative multitasking is a piece of Victorian-era technology that is so painfully bad for various reasons, but people still tend to believe that is solves something.

On the other hand, modern operating systems are awesome. The thread schedules are awesome. Thread schedulers can dynamically adjust thread priorities depending on the load and solve some nasty synchronization problems (like priority inversion or starvation) for you. It's not 1995 and OS schedulers can switch threads in O(1) and most server apps can use a thread per connection approach without any problems. The only thing you need to get right is a synchronization, but there are a lot of tools that can help (like Thread Sanitizer). It's much harder to get the synchronization right with cooperative multitasking (good luck finding that priority inversion on implicit lock caused by dumb channel use pattern) than with normal threads.

Double-checked locking has been solved in Java since 1.5, which arrived more than a decade ago. The article you link to even says it's going to be solved.

"Solved" if you remember to mark the static as volatile.

It still proves the OP's point that this stuff can be come complicated.

That doesn't seem that bad to me. It's an explicit, unambiguous recommendation, one that even be recognized by static analysis. It's not much worse than an API that requires that you call some particular method--the only part that's bad is that you can fail to notice the problem because the code will usually work.

What I worry about are things that can't be easily caught. I trust myself to write embarrassingly parallel code, and I've written some code that has real risks of deadlock. But shared data structures that have complex locking schemes still scare me.

It all depends on what problem you're solving. Note that you're focusing on libraries and languages for writing servers. But what about, say, linear algebra routines? How would you implement BLAS in a performant manner without threads?

Sometimes, threads are the preferable solution.

> Successful, popular concurrent platforms like [...] node.js, eschew threads in favor a single-threaded, async/non-blocking code model

Except that Node (by way of libuv) uses thread pools internally for things that it deems to be CPU-intensive enough to need it, like crypto.

Threads are necessary and pretty much unavoidable, but in my experience cooperative sharing (e.g., co-routines) results in systems that are much more reliable and far easier to reason about. Structured worker threads ("go away and tell me when you're done") with no shared resources also go a long way towards keeping systems reliable.

Don't get me wrong; I love a nice, concurrent and efficient system as much as the next guy. I've been in various kernels and embedded systems doing the whole spinlock/reader-writerlock/mutex scene, and it's fun as hell. But in the end, if I can spend a few more cycles and not have to worry about race conditions and debug hairy crashes all the time then I'm far more productive, and ultimate the customer is happier.

Are there any coroutine systems that present a sensible call stack and the rest of the familiar debugging experience? I've only encountered systems that do not, and found them an utter nightmare to reason about.

Threads aren't particularly low-level; they're more like a mid-level concurrency abstraction. The whole point of a thread is that you get to use abstractions (like semicolons, variable access, function calls, and program stacks) that are familiar from sequential programming in a concurrent setting.

At the machine level, these are all done with interrupts, shared memory, CAS & LL/SC instructions, and memory barriers, which actually tend to map more closely to the event model.

In the middle, there's a whole family of concurrent data structures - semaphores, producer/consumer queues, mutexes, transactions, etc. - which really are worth learning about.

Yup. Don't be scared of threads, just make sure to be diligent about using them properly. Learning decent basic multithreading techniques is something a lot of devs can pick up with some effort in a couple of weeks to months at most. It's like most things, don't dig yourself a hole and you won't have to dig yourself out of it.

Well, yes. I learnt memory management and can C/C++ like a boss, but for many types of applications, doing it yourself is a bad idea.

I've learnt and used low async primitives for years, but for many (most?) applications, as long as you know what you are doing (which i guess is your point), using them directly isn't a very good idea anymore.

You want to know what happens when you light up all 4 + GPU? Your battery life gets decimated.

You can't even do that on any modern mobile device without thermal limiting kicking in quickly(although GPU/SoC vendors love to turn that off in standard benchmarks).

Reply on news.ycombinator.com