Hacker News Reader

Comment by scorpion032 | Hacker News Reader

"List Comprehensions: Ok to use for simple cases (only)": http://google-styleguide.googlecode.com/svn/trunk/pyguide.ht...

Really, Google? I find the following far more convenient to read:

    result = [(x, y) for x in range(10) for y in range(5) if x * y > 10]

Than the alternative:

    result = []
    for x in range(10):
        for y in range(5):
            if x * y > 10:
                result.append((x, y))

View on HN

I found the alternative easier to read. I read it once. I had to scan yours twice. I'd say yours is borderline simple case.

Also, i think part of the difference can be found when you're dealing with large amounts of code in maintenance. I'd much rather read something dead simple (if somewhat verbose) than something that makes me think at all (another example here would be (in the ruby world) use of !unless).

Like anything that has to do with taste, to each their own (i prefer a vinegary bbq while you might like a smokier bbq)

I didn't like list comprehensions when I first encountered them, but after getting accustomed to them, I now strongly prefer to write and read a comprehension over a for-in loop.

I agree, but it depends on how complex the operation within the comprehension is.

It would be more readable to write it as:

    result = [(x, y) for x in range(10) 
                     for y in range(5) 
                     if x * y > 10]

Anyway, I'd call it a simple case.

Also, it has nice semantics - the second example has a clear execution order. This one doesn't - it can all happen at once, from the program's perspective and unless you make assumptions about order in which the items are calculated (as opposed to returned), they don't even need to be all ready before you start iterating on them. If you assume it can happen at once, the list comprehensions can be neatly mapped to parallel computations.

So, it shouldn't be that hard to optimize something like:

    data = [sin(x) for x in arange(0, pi, pi/20)]

to run on a GPU.

edit: small clarifications

"the second example has a clear execution order. This one doesn't - it can all happen at once, from the program's perspective"

I'm not sure what you're getting at here. With Python list comprehension semantics, those are both the same, including execution order. I don't see any ambiguity or need for "assumptions about the order of results". Am I missing something?

Optimization of Python list comprehensions to run on a GPU would take some serious mojo to ensure independence of each clause. Not an impossible amount, but certainly not trivial, especially as you move beyond calling 'sin'.

Sorry. I did some small clarifications (the original post got a little mangled during edition before my coffee kicked in) regarding order.

When you use a list comprehension, you may (or not) care about the order of the resulting items, but your program is completely shielded from the order in which the resulting list is calculated - unless your function is affecting a global state while the LC is being evaluated and each evaluation depends on the state changed by the last one. You can't insert a print in the outer loop, for instance, unless you explicitly nest the LCs.

This isn't Scala. In python list comprehensions are single threaded and have a very specific meaning (i.e., their computation order is deterministic). The pattern for working with parallel maps etc are done with separate map functions (see concurrent.futures.Executor.map).

Even if the result is in a certain sequence, why is the calculation required to be done sequentially?

With the map builtin deprecated because of LCs, wouldn't it make sense to exploit concurrent-like behavior with the nicer LC syntax? It makes a lot of sense to execute strictly in order for generators, but it doesn't make that much with LCs and I never saw code whose correctness depends on strictly ordered causation of side-effects.

Obviously, being that much multiprocessor-friendly makes little sense under a GIL, but CPython is not the only implementation of Python.

In general, Python is too flexible to make that easy. In theory, while you may never have seen a LC that depends on side effect order (and I believe that, I don't think I have either), the compiler can't assume that and doesn't. In general almost anything in Python could have a side effect, even though the culture is that it probably shouldn't. It turns out if you dig into it, this problem is ground deeply into the language.

Python is and probably ever shall be my favorite language of the OO-imperative 20th century/first decade of the 21st century style, but it will not be making the leap to the next generation of languages. And I sort of hope it doesn't even try; better to be the best of breed imperative-OO than a half-assed hybrid that does nothing well.

All that's needed to fix this is to place a "there is no guarantee the items of the list comprehension will be calculated sequentially one at a time" warning in the documentation. We don't need to change CPython to clear the way for other implementations.

Your first example is easier for me to read, but only in the case that I don't care about the order of (x, y) points.

If I care that (3, 6) comes before (4, 4), I find it much easier to ascertain that from the form that uses indentation for nesting loops.

List comprehensions iterate in the same order as the for loops. If it helps, stick a mental newline in there :P

    [(x, y) for x in range(10)
    for y in range(5)...]

Not only that, try running dis() on each of those and seeing what bytecode gets executed...

Right. List comprehensions even perform better.

Reply on news.ycombinator.com