Comment by unblough | Hacker News Reader

Comment by unblough | Hacker News Reader

Parent Full thread

> Is parallel programming hard? Without any further details or specifics, yes it is. It is far harder to conceptualize code instructions executing simultaneously, than one-at-a-time in a sequential order.

If I program (map inc [0 1 2 3]) is it really any more difficult to conceptualize the (inc ) function performing on each element sequentially than in parallel?

I think the difficulty of parallel programming is less innate and more two fold:

1) languages often default to sequential so to do async requires introducing additional primitives to the programmer

2) knowing when to effectively use parallel programming

When I have a list or stream that I know has independent elements that require wholly independent calculations then parallel programming is straightforward

Where people get hung up is trying to shoe horn async where it is either unnecessary (performance is equal or worse than sequential) or introduces breaking behavior (the computations are in fact interdependent).

Most problems are not embarrassingly parallel.

(Fun fact: I once had someone call HR on me because they didn't know embarrassingly parallel was a technical term, and they thought I was belittling them)

Prefix scan is not embarrassingly parallel. Yet OP's statement still works when you change it to scanl (+) 0 [0 1 2 3]

That requires + to be associative. And scan is one of the core skeletons of parallel skeletons, so obviously if you express everything as parallel skeletons, parallel programming remains manageable.

I agree that if we define the individual instructions to always be wholly independent, then sure, it is more straightforward.

While I'd probably argue that it is still more difficult to conceptualize, the statement we're discussing is presented as broad and general. I'd call it far less misleading if it said something like:

There is a common myth in software development that parallel programming *has* to be hard.

The whole reason async is even a thing is due to slow, side-effect producing operations. Of course pure functions are easy to parallelize.

I don't think folks so much "shoe horn async where it is unnecessary" as the red/blue problem causes async code in most languages to spread.

Or by "async" do you just mean concurrent code? I'm reading "async" to mean lightweight coroutines or similar.

> Or by "async" do you just mean concurrent code? I'm reading "async" to mean lightweight coroutines or similar.

Yeah, my bad, I was utilizing a colloquial definition of a term that has a technical definition in a technical conversation. A lamentation lo the lossyness of language.

I guess I assumed we were talking about something other than in terms of red/blue because I'd argue red/blue's "hard"ness transcends myth to mathematical fact.

Reply on news.ycombinator.com