Hacker News Reader

Comment by jerf | Hacker News Reader

"the second example has a clear execution order. This one doesn't - it can all happen at once, from the program's perspective"

I'm not sure what you're getting at here. With Python list comprehension semantics, those are both the same, including execution order. I don't see any ambiguity or need for "assumptions about the order of results". Am I missing something?

Optimization of Python list comprehensions to run on a GPU would take some serious mojo to ensure independence of each clause. Not an impossible amount, but certainly not trivial, especially as you move beyond calling 'sin'.

View on HN

Sorry. I did some small clarifications (the original post got a little mangled during edition before my coffee kicked in) regarding order.

When you use a list comprehension, you may (or not) care about the order of the resulting items, but your program is completely shielded from the order in which the resulting list is calculated - unless your function is affecting a global state while the LC is being evaluated and each evaluation depends on the state changed by the last one. You can't insert a print in the outer loop, for instance, unless you explicitly nest the LCs.

This isn't Scala. In python list comprehensions are single threaded and have a very specific meaning (i.e., their computation order is deterministic). The pattern for working with parallel maps etc are done with separate map functions (see concurrent.futures.Executor.map).

Even if the result is in a certain sequence, why is the calculation required to be done sequentially?

With the map builtin deprecated because of LCs, wouldn't it make sense to exploit concurrent-like behavior with the nicer LC syntax? It makes a lot of sense to execute strictly in order for generators, but it doesn't make that much with LCs and I never saw code whose correctness depends on strictly ordered causation of side-effects.

Obviously, being that much multiprocessor-friendly makes little sense under a GIL, but CPython is not the only implementation of Python.

In general, Python is too flexible to make that easy. In theory, while you may never have seen a LC that depends on side effect order (and I believe that, I don't think I have either), the compiler can't assume that and doesn't. In general almost anything in Python could have a side effect, even though the culture is that it probably shouldn't. It turns out if you dig into it, this problem is ground deeply into the language.

Python is and probably ever shall be my favorite language of the OO-imperative 20th century/first decade of the 21st century style, but it will not be making the leap to the next generation of languages. And I sort of hope it doesn't even try; better to be the best of breed imperative-OO than a half-assed hybrid that does nothing well.

All that's needed to fix this is to place a "there is no guarantee the items of the list comprehension will be calculated sequentially one at a time" warning in the documentation. We don't need to change CPython to clear the way for other implementations.

Reply on news.ycombinator.com