Comment by chowells | Hacker News Reader

Comment by chowells | Hacker News Reader

Parent Full thread

The difference is quite useful and informative. In fact, most places don't seem to state it strongly enough: Concurrency is a programming model. Parallelism is an execution model.

Concurrency is writing code with the appearance of multiple linear threads that can be interleaved. Notably, it's about writing code. Any concurrent system could be written as a state machine tracking everything at once. But that's really hard, so we define models that allow single-purpose chunks of linear code to interleave and then allow the language, libraries, and operating system to handle the details. Yes, even the operating system. How do you think multitasking worked before multi-core CPUs? The kernel had a fancy state machine tracking execution of multiple threads that were allowed to interleave. (It still does, really. Adding multiple cores only made it more complicated.)

Parallelism is running code on multiple execution units. That is execution. It doesn't matter how it was written; it matters how it executes. If what you're doing can make use of multiple execution units, it can be parallel.

Code can be concurrent without being parallel (see async/await in javascript). Code can be parallel without being concurrent (see data-parallel array programming). Code can be both, and often is intended to be. That's because they're describing entirely different things. There's no rule stating code must be one or the other.

When you define some concepts, those definitions and concepts should help you better understand and simplify the descriptions of things. That's the point of definitions and terminology. You are not achieving this goal, quite the opposite in fact, your description is confusing and would never actually be useful in understanding, debugging, or writing software.

Stated another way: if we just didn't talk about concurrent vs parallel we would have exactly the same level of understanding of the actual details of what code is doing, and we would have exactly the same level of understanding about the theory of what is going on. It's trying to impose two categories that just don't cleanly line up with any real system, and it's trying to create definitions that just aren't natural in any real system.

Parallel vs concurrent is a bad and useless thing to talk about. It's a waste of time. It's much more useful to talk about what operations in a system can overlap each other in time and which operations cannot overlap each other in time. The ability to overlap in time might be due to technical limitations (python GIL), system limitations (single core processor) or it might be intentional (explicit locking), but that is the actual thing you need to understand, and parallel vs concurrent just gives absolutely no information or insights whatsoever.

Here's how I know I'm right about this: Take any actual existing software or programming language or library or whatever, and describe it as parallel or concurrent, and then give the extra details about it that isn't captured in "parallel" and "concurrent". Then go back and remove any mention of "parallel" and "concurrent" and you will see that everything you need to know is still there, removing those terms didn't actually remove any information content.

Addition vs multiplication is a bad and useless thing to talk about. It's a waste of time. It's much more useful to talk about what number you get at the end. You might get that number from adding once, or twice, or even more times, but that final number is the actual thing you need to understand and "addition vs multiplication" just gives absolutely no information or insights whatsoever.

They're just different names for different things. Not caring that they're different things makes communication difficult. Why do that to people you intend to communicate with?

That leaves out the (partial) ordering of "operations". Happens before/after/at the same time is just one important primitive.

Async JS code is parallel too. For example, await Promise.all(...) will wait on multiple functions at once. The JS event loop is only going to interpret one statement at a time, but in the meantime, other parts of the computer (file handles, TCP/IP stack, maybe even GPU/CPU depending on the JS lib) are actually doing things fully in parallel. A more useful distinction would be, the JS interpreter is single-threaded while C code can be multithreaded.

I can't think of anything in practice that's concurrent but not parallel. Not even single-core CPU running 2 threads, since again they can be using other resources like disk in parallel, or even separate parts of the CPU itself via pipelining.

> A more useful distinction would be, the JS interpreter is single-threaded while C code can be multithreaded.

...this seems like a long way round to say "JS code is not parallel while C code can be parallel".

Or to put it another way, it seems fairly obvious to me that parallelism is a concept applied to one's own code, not all the code in the computer's universe. Other parts of the computer doing other things has nothing to do with the point, or "parallelism" would be a completely redundant concept in this age where nearly every CPU has multiple cores.

But I've never heard of someone caring about their code vs lower-level code running in parallel, just whether or not there are multiple OS threads involved in the end. Like you have 32 CPU cores, your Python batch code is only using 1, and whether or not you use Python threads to fix this depends on if you're using something like Numpy that'll release the GIL.

Reply on news.ycombinator.com