Hacker News Reader

Comment by wasd | Hacker News Reader

Excuse my ignorance but why is the first one parallelizable but not the second?

As a matter of fact, any implementation of the same algorithm has the same parallelizing possibilities. You just need a smart enough compiler.

But a "smart enough compiler" is what compiler people say when they are talking about their version of the abstract anthropogenic equivalent AI. The fact is that the first one is much easier to parallelize, so that real implementations actually do it, while the second one is not done on practice. And it's easier to parallelize because there are some extra guarantees on the signature of those functions that a for loop does not give.

In the first one, you are simply telling it to map and filter every element. Since you can do that to each element in an independent way and the code is more abstract, the map and filter can be done in parallel without you knowing (I'm not sure if would execute in parallel in Swift, I know that in java you can change the way it does it by using parallel streams).

But in the second example, you are telling it to iterate over each element, sequentially. Since you are telling step by step what needs to be done, it's not parallelizable unless you implement it to be so.

Put another way, the first example is about data flow. The required value is specified declaratively, in the form of an expression.

The second example is about control flow. The required behaviour is specified imperatively, in the form of a statements to be executed in a defined order, and the combined effect of those statements is to update the list until it has the required value.

In the second case, the programmer specifies more details explicitly. In this case, those extra details probably aren’t helping, but the optimiser still has to prove that transforming the control flow (for example, into a parallelised form) really will give the same observable behaviour. In the first case, the optimiser doesn’t have to prove the things the programmer never specified anyway, so it has more latitude in how it implements the underlying computation.

They're both parallelizable, but the second one is harder to automatically parallelize.

There are guarantees about the way a map function works. It doesn't mutate its input, it only has access to one element at a time, you can't access the data structure you're building, etc.

All of these traits are true of the imperative version as well, but it's a lot harder to write a program which understands that. Meanwhile, you know that's how map and filter work, so it's trivially easy to know that those guarantees hold.

As a matter of fact loops that mutate state are not parallelizable because you can't analyze what they do in order to infer intent and thus some imposed ordering with which you could efficiently distribute the required work while at the same time guaranteeing correctness - solve that and you'd solve the Halting problem ;-)

Can't be done.

Well, you can't solve it in the general case. But parallelizing compilers can and have been written that perform a conservative analysis -- they can prove some subset of all parallelizable loops as parallelizable, and for the ones they can't prove as such, they err toward 'remain serialized'.

For a simple example, code that performs regular array-based accesses, where index computations are simple affine functions (a*i + b) of a single iteration variable 'i', is a case that's received a lot of attention. See e.g. Polly for LLVM.

Maybe we aren't speaking the same language, but I never mentioned parallelism in my text.

We could talk about why the first example could be parallelized of course, but that's not what I wanted to talk about and you missed the point ;-)

Actually, I responded to the wrong post. I meant to post my reply to svachalek.

Reply on news.ycombinator.com