Concurrency is not Parallelism. Rob Pike at Waza 2012 [video]

> So concurrency is about dependencies between tasks or the lack of them while parallelism is about actually executing independent tasks at the same time.

I understand what you're getting at, but I don't think that's a great summary of concurrency vs. parallelism. My version would be: currency is about non-determinism and multiple threads of control while parallelism is about doing multiple computations at once for performance reasons (possibly deterministically).

EDIT: sorry I see this has already been posted almost verbatim a few other places in the thread

Yes although the example given by Rob Pike highlights all the information that would have to be shared between the load balancer and the web processes in real time in order to make intelligent routing work well.

Very hard to enforce when you're running a heterogenous network trying to handle routing which may have any kind of app answering requests. If there was an easy solution to this, evidently Heroku would have put it in already, but I'd hope they're working on it.

Sounds like some of their customers have outgrown them though; given the amount they are paying per month they could easily host themselves and hire someone to look after their servers - at some point it becomes necessary to control your entire stack if you're looking at performance.

I listened to his presentation. IMHO the presenter laid too much stress on parallelism and concurrency being different, but never explained the difference clearly and succinctly. But from the presenter's side, I can see that he needs to sell his product(golang). Therefore he assumed that - these definitions are often confused, it is a big issue, and go is 42.

They're not so creepy compared to some of the artist's other work: http://reneefrench.blogspot.com/

There's a plush gopher too-- http://1.bp.blogspot.com/-ti6s2UsV8no/Tdn82ozbggI/AAAAAAAAAB...

A task is a future or a promise which can deliver a result later. A Goroutine in go is more general than a task because they don't have to terminate when they deliver a result. The subsume tasks in any way.

Goroutines are mapped onto (kernel-level) threads in order to make it possible to run more of them at once and thus provide parallel execution. Otherwise, you have a single scheduler of goroutines in a single process. You can still switch context between the goroutines, thus the system is concurrent. But it will not be parallel: One goroutine only at a time.

For a system in which most goroutines wait on IO or the network, it may not be a problem to run many goroutines on a single core only.

Suppose you have a process which is I/O bound on multiple I/O resources. If instead of blocking on each one, you suspend that sequence of execution and move onto the next, resuming when you get the signal that the call is unblocked, you are handling the resources concurrently, but not in parallel.

If you alter the program to spawn a separate thread for each, and let them all block, you will have a parallel solution.

Concurrency : appears to be running at the same time (possibly using the same core)

Parallelism : actually running at the same (using different cores)

They are language agnostic generic concepts.

Concurrency deals with dependency, or the lack of, among tasks and their running order. Task A must run after task B, B after C, etc. A, B, and C are not concurrent tasks. D can run at anytime. D can run before, after, or at the same time of A/B/C. The ordering doesn't matter. In this case D is concurrent to A/B/C. The concurrent tasks MIGHT run at the same time but it's not necessary. It's up to the OS/system scheduler. Concurrent tasks can run on a single core cpu serially and they are still concurrent tasks.

Parallelism has to do with efficiently cranking as many tasks as possible to actually run in parallel. E.g. given 10 cpu, how do I fully utilize all the cpu at the same time? Mostly it has to do with data partition to make sure the cpu can get evenly distributed non-depending data.

Concurrency and parallelism can be mixed at the same time. E.g. in the map-reduce pattern, the map part utilizes parallelism to distribute the data to process at all the cpu. The reduce task has dependency on the mpa tasks; running order means concurrency is in play there.

It's a bit more nuanced than that. Concurrency implies non determinism. For example if you have multiple threads then the result of the program is non deterministic: it may depend on the particular interleaving of the threads over time. Parallelism is about executing multiple things at the same time to improve performance. You can have parallelism without concurrency. For example a parallel map operation is parallel, but not non deterministic. It always gives the same result. Now, under the hood to implement parallel map, there is probably some concurrency involved, but that is not exposed in the interface of map. You can have concurrency without multiple threads, for example Node with asynchronous callbacks is concurrent, because the order in which those callbacks get called is non deterministic. But it does not necessarily need to run on multiple threads. Usually we don't want the end result of our program to be non deterministic, so we use synchronization primitives like locks and Go's channels to limit the non determinism in such a way that it does not affect the end result.

What most people don't realize is that for concurrency to be useful, there must be parallelism. But wait, didn't I just say that in Node you have concurrency without multiple threads? Yes, but the parallelism does not need to be in your CPU! If you do an asynchronous disk read, or an asynchronous network request, the disk and other computers on your network are working in parallel with your program. Concurrency allows us to exploit that parallelism, so that the CPU can do useful work while the disk is busy reading our data.

Now, you can write your program with concurrency, but if there is no parallelism anywhere in the system, that's not useful. So in practice it is the reverse: concurrency implies parallelism (but not necessarily on the CPU), but parallelism does not imply visible concurrency (though it might be hidden under the hood of a parallel construct like parallel map).

Sort of, but this explanation loses too much information. Concurrency is a binary relation on tasks, which looks like Task -> Task -> Bool; if true, the order in which the tasks may execute relative to one another does not matter. If false, the tasks must execute in a defined order. Astute readers will note the above function is isomorphic to the function defining whether there is an edge between two nodes in a graph (although the Bool value is flipped); indeed, task dependency graphs are a useful way of modelling concurrent systems.

Concurrency has absolutely nothing to say about parallelism. If you have a computer system which is able to execute tasks in parallel, it can take advantage of information from the concurrency relation to execute two concurrent tasks in parallel.

For the best implementation of this idea I've seen, I recommend taking a look at Intel's Threading Building Blocks template library.

Concurrency is when you have multiple threads of control. Parallelism is when you are doing more than one thing at the same time.

Multiple threads of control may do things in parallel.

Another good explanation: http://ghcmutterings.wordpress.com/2009/10/06/parallelism-co...

Parallelism does not require concurrency (at least not in your code). Parallelism can be handled by a library allowing your code to speed up with often minimal changes. One such example is the replacement of a sequential map function by a parallel map.