A collection of not so obvious Python stuff

I'm being a bit pedantic, but the last snippet under About lambda and closures-in-a-loop pitfall is arguably misleading when it says, "This, however, does not apply to generators…" Which is true in the example given, but not true in general. Generators don't actually provide a new environment on each evaluation, they just lazily construct their contents, which assuages the problem in certain situations. This snippet

    for l in (lambda: n for n in range(3)):
        print(l()) # prints 0, 1, 2

will alternately request an argument from the generator, return a lambda, call it, and only then request the next argument from the generator, which means at the time of the funtion call, n will not have been incremented up to its final value. We can see the previous closures-in-a-loop behavior return if we keep the closures around to execute again, at which point the loop environment contains the final value of n:

    funcs = []
    for x in (lambda: n for n in range(3)):
        funcs.append(x)
        print(x()) # prints 0, 1, 2
    for y in funcs:
        print(y()) # prints 2, 2, 2

So generators don't 'fix' the problem by constructing distinct environments for each loop iteration, but by deferring evaluation of their body until necessary.

What I love about Python is that these hangups are mostly relatively obscure, due to the puritanical adherence to principle of least surprise by Guido and the other language maintainers. No language is completely without quirks though, especially one as old as Python.

This bit bothered me slightly:

>But what if we put a mutable object into the immutable tuple? Well, modification works, but we also get a TypeError at the same time.

    tup = ([],)
    tup[0] += [1]

Of course that's an error. It's equivalent to:

    temp = tup[0]
    temp += [1]
    tup[0] = temp

But the given explanation is way too obtuse. This shouldn't be a mystery at all:

> If we try to extend the list via += "then the statement executes STORE_SUBSCR, which calls the C function PyObject_SetItem, which checks if the object supports item assignment. In our case the object is a tuple, so PyObject_SetItem throws the TypeError. Mystery solved."

The "consuming generator" problem is a good point, but their workaround misses a better solution. In Python 3, range() returns a range object, which has a __contains__ method, so you can use "in" just fine without consuming any generators. It's also much faster (it would be constant time/memory except for the fact that integers can have arbitrary size).

    l = range(5)
    print('2 in l,', 2 in l)
    print('3 in l,', 3 in l)
    print('1 in l,', 1 in l)

prints...

    2 in l, True
    3 in l, True
    1 in l, True

Also, the lambda/closure example is something that really irritates me about Python. Both dynamic and lexical scoping have this problem: they always reevaluate the variable inside the closure whenever it's called, the only difference is which parent scope the value is pulled from (the parent scope or the calling scope). It's not that the "last lambda is being reused", as the article says, but rather, there are five different lambdas, but they each refer to the same variable, which is set to 4 after the list comprehension. This sucks for metaprogramming, where you'd like to create a bunch of copies of a function with different parameterizations. There's two workarounds that I know of for this for dynamic scoping: creating another nested scope, or using keyword arguments. I'm not sure what you'd do for lexical scoping (but who uses that anyways?).

    for x in [(lambda i: lambda: i)(i) for i in range(5)]:
        print(x())

...or...

    for x in [lambda i=i: i for i in range(5)]:
        print(x())

In my purely-functional Python-like programming language, Mutagen, closures are static, so that they capture the value of variables from the enclosing scope at the point that they're defined. So this works as you'd expect, printing 0 and 1 (no list comprehensions yet, and lambdas are full functions, and of course lists are immutable so no +=):

    lambdas = []
    for x in [0, 1]:
        lambdas = lambdas + [lambda: return x;]
    for l in lambdas:
        print(l())

Even after reading this list, I'm comfortable forgetting everything (with maybe one exception) because these surprises really only come up when you use horrible coding practices. But it's nice to know it really takes some serious effort to find these kinds of things in Python.

Midnight is not False in Python 3.5 anymore:

https://docs.python.org/3.5/whatsnew/3.5.html

I somehow didn't know about nonlocal. With that, isn't python 100% comparable with JavaScript?