Is parallel programming hard? Without any further details or specifics, yes it is. It is far harder to conceptualize code instructions executing simultaneously, than one-at-a-time in a sequential order.
If I program (map inc [0 1 2 3]) is it really any more difficult to conceptualize the (inc ) function performing on each element sequentially than in parallel?
I think the difficulty of parallel programming is less innate and more two fold:
1) languages often default to sequential so to do async requires introducing additional primitives to the programmer
2) knowing when to effectively use parallel programming
When I have a list or stream that I know has independent elements that require wholly independent calculations then parallel programming is straightforward
Where people get hung up is trying to shoe horn async where it is either unnecessary (performance is equal or worse than sequential) or introduces breaking behavior (the computations are in fact interdependent).
(Fun fact: I once had someone call HR on me because they didn't know embarrassingly parallel was a technical term, and they thought I was belittling them)
I don't think folks so much "shoe horn async where it is unnecessary" as the red/blue problem causes async code in most languages to spread.
Or by "async" do you just mean concurrent code? I'm reading "async" to mean lightweight coroutines or similar.
Yeah, my bad, I was utilizing a colloquial definition of a term that has a technical definition in a technical conversation. A lamentation lo the lossyness of language.
I guess I assumed we were talking about something other than in terms of red/blue because I'd argue red/blue's "hard"ness transcends myth to mathematical fact.
While I'd probably argue that it is still more difficult to conceptualize, the statement we're discussing is presented as broad and general. I'd call it far less misleading if it said something like:
There is a common myth in software development that parallel programming *has* to be hard.
When you say "without further details or specifics" you're saying "using my default framework of a c/ c descendent world"
The author's point is that sequential programming is one type of simple programming, but it's not the only type, and it doesn't map easily to modern hardware
There's some irony to the fact that you re-interpret my opinion as being very specific to C and (indirectly) posit that - in that specific case - parallel programming is hard, and then yourself go on to select a very specific case where parallel programming is not hard, because some matrix operations are independent.
I agree that there are languages that are explicitly built to make parallel programming easy. But in general, and not just related to c or c descendant languages, parallel programming is hard.
The common myth - you're doing parallel programming? That sounds hard
It's not always hard. It really isn't! You don't need to be a genius or an expert to write parallel code.
Maybe where we're getting caught up is Cassie K's comment on ml engineering. You don't need to know how to build a microwave to use a microwave. In the same way, you don't need to be a genius or some deep expert in distributed systems to use abstractions that parallelize your programs
To write a parallel program does not require that you know what a mutex is. It just needs you to understand some simple algebraic (6-8th grade) properties about your functions (and, in fact, for library functions, they can be annotated as associative)
There is a broad spectrum of parallel programs. Somebody using a web server implementation? They've made a parallel application
Somebody running tensorflow or pytorch? Also parallel! Even for simple stuff!
You could be a beginner programmer and be taught to make parallel programs without understanding distributed systems. It's not always hard. It's not generally hard. The complex bits are hard. The simple bits use 8th grade math.
Sure, and even more people commenting appear to be of the mind that it is generally hard.
> That's also what the author is saying.
It's not what author is explicitly saying in the statement I'm addressing if you re-read my original comment. There, the author isn't saying that it's not always hard, they're implying that it 'in general' isn't hard.
From your arguments, it would seem you think anything that actually runs in parallel (regardless of whether it programmed as such) can be considered 'parallel programming' and from that perspective, sure, it is super easy. But with that kind of reasoning, you can argue that anyone who only knows how to drive cars with automatic gears is actually a gear-shifting expert and shifting gears is really easy, because it happens automatically for them.
That Erlang exists and people use it successfully does not mean that harder things aren't.
Or do you mean an individual can't do things in parallel?
Like.... Pushing all of those grocery carts in a long line is moving them in parallel
Or do you mean processing? Like thinking?