Hacker News Reader

Comment by sifoobar | Hacker News Reader

I spent some time comparing Python's [0] map/filter and comprehensions against Snigl [1].

Comprehensions are just plain faster, especially if you perform say filter and map at the same time.

That and a lambda implementation that only a mother could love, lack of tail call optimization etc.

What bothers me about comprehensions is they come with a separate set of rules, like LOOP in Common Lisp or templates in C++; but I'm pretty sure the restrictions are part of what makes them fast.

[0] https://gitlab.com/sifoo/snigl/blob/master/bench/seq.py

[1] https://gitlab.com/sifoo/snigl/blob/master/bench/seq.sl

View on HN

Agreed, this made my eye twitch:

> filter(P, S) is almost always written clearer as [x for x in S if P(x)]

sure, Guido, if you think it's clear that the variable x is in the enclosing scope...

Python 3 changed this, iirc. Comprehension no longer leak.

It's still an assignment operation, and you can still get weird effects in the enclosing scope. I did something like this by accident:

    Python 3.7.1 (default, Nov 28 2018, 21:47:25)
    Type 'copyright', 'credits' or 'license' for more information
    IPython 6.4.0 -- An enhanced Interactive Python. Type '?' for help.

    In [1]: lookup = {}

    In [2]: [x for (x, lookup[x]) in ('ab', 'cd', 'ef') if x in lookup]
    Out[2]: ['a', 'c', 'e']

    In [3]: lookup
    Out[3]: {'a': 'b', 'c': 'd', 'e': 'f'}

It appears to work without list comprehensions as well:

    Python 3.7.1 (default, Nov  6 2018, 18:49:54)
    Type 'copyright', 'credits' or 'license' for more information
    IPython 7.1.1 -- An enhanced Interactive Python. Type '?' for help.

    In [1]: lookup = {}

    In [2]: a, lookup[a] = 'xy'

    In [3]: a
    Out[3]: 'x'

    In [4]: lookup
    Out[4]: {'x': 'y'}

I wasn't previously aware that this was possible, but I guess it's no different from

    a = 'x'
    lookup[a] = 'y'  # i.e. lookup['x'] = y

The whacky thing there isn't the list comprehension though, it's the implicit assignment of `lookup[x]`. If you desugar it some (which isn't really possible, but

    [(x, lookup.setdefault(x, y)) for (x, y) in ('ab', 'cd', 'ef')]

comes close), what's happening makes a lot more sense. It works exactly the same way as any other scope: first checks locally, then nonlocally or globally. You're modifying a nonlocal. The syntax you're using to modify the nonlocal is surprising, but that's the surprising part, not the ability to modify a nonlocal.

I'd desugar it like this:

    >>> lookup = {}
    >>> for (x, lookup[x]) in ('ab', 'cd', 'ef'):
    ...     pass
    ... 
    >>> lookup
    {'a': 'b', 'c': 'd', 'e': 'f'}

This makes is much more clear that there is just a double assignment, and that comprehension are pretty close to regular `for` statement.

(this was a great example btw -- I write python3 a lot, and I was staring at it for a long time before Icould figure it out)

I think I was personally surprised by two assumptions that I implicitly held, both of which disappear when you write them down:

1. The syntax of comprehensions led me to think that they introduce some kind of a variable binding, such as `let` in many Lisps, and that binding `lookup[x]` would have been a syntax error. Of course Python didn't introduce anything like that for comprehensions, and it's just a normal assignment operation, with an extra scope inserted in Python 3 to stop `x` leaking into the enclosing scope.

2. Because of the swap idiom `x, y = y, x` I thought tuple assignment happens somehow in parallel, so my code should have caused an undefined variable error, even if it is not a syntax error. But of course it just evaluates the right hand side first and then assigns to the left hand side in left-to-right order, which the language reference https://docs.python.org/3/reference/simple_stmts.html#assign... notes "sometimes result[s] in confusion" (near the end of the section).

Okay that's pretty sweet actually. I write a lot of legacy-compatible code, so I'm kinda behind on Py3. Maybe that's my 2019 project.

> Comprehensions are just plain faster, especially if you perform say filter and map at the same time.

Comprehensions are often slower if you're applying a function regardless, definitely so if that function is a builtin.

Reply on news.ycombinator.com