Understanding Python's augmented assignment (a += b) | Hacker News Reader

Understanding Python's augmented assignment (a += b)

stupidpythonideas.blogspot.com

72 pointsceronman15 comments

Nice article, though I already understood the matter. Thanks for the link to PEP 463[1], though; I can see it cleaning up some of our code. Do you know if there have been any developments into implementing it?

[1] http://legacy.python.org/dev/peps/pep-0463/

It died:

https://mail.python.org/pipermail/python-dev/2014-March/1331...

(the PEP author posts a couple messages in the thread too)

What this article says about C++ is not true.

"Not true" sort of undersells it. They seem to make up some hypothetical language entirely and call it c++ for rhetorical convenience.

It seems to me it describes pretty much what happens in C++, but with pythonized syntax, because explaining the exact syntax of all the sigils would be too much for such a short article.

Absolutely not. Take for example the part about t = [0, 1] followed by t[0] = 2. t[0] does return nothing at all, neither 0 nor any strange complex object. It just evaluates to the memory address of the first element of the array. The idea expressed later that C++ unlike Python treats every piece of code exactly the same no matter whether it appears on the left hand side or the right hand side of an assignment operator is fundamentally wrong.

Well no, for

    std::vector<int> t = {0, 1};

t[0] calls operator[], which returns an int&. This is precisely an lvalue reference, which is kind of like a memory address, but int& is a type in its own right. Calling it a "reference object" is not entirely correct, but it's also not entirely incorrect. And let's not get started on vector<bool>...

At the semantic level it evaluates to an lvalue. Or an lvalue reference. It depends on what the type of t is. Given the syntax, the author obviously meant some vector-like type, so it should, if you're following good style, evaluate to a reference. And that's what he was getting at - t[0] evaluates to something that can be assigned to. In C++, the thing that t[0] evaluates to handles the assignment (whether t is a pointer and t[0] "handles" the assignment by simple copy to an address, or t is an object and t[0] returns a reference or some object overriding operator= for the type that t contains is irrelevant - it's the same semantically), while in Python, the t object itself handles the assignment. Python's behaviour is akin to C++ having an operator[]=(size_t, const T&).

It all stems from the different behaviour of = in Python and C++ - in Python = binds the name on the left to the value on the right and you can have any number of names for a value; in C++ each name is exactly one value (even in the case of references - the reference itself is a pointer and there's no other name that signifies the memory in which that pointer value is stored) and = copies the value from the thing on the right to the thing on the left, in a way determined by the thing on the left (so for references the value is copied to the thing that the reference points to and other types can override operator= to do whatever they like), but the base concept is that = copies a value from one place to another (or moves it if it's an xvalue and you're using C++11 and what's on the left implements move-assignment).

P.S. Since C++ lets you do as you like, you can for instance implement operator[](size_t, const T&) as assignment and then t[0, 2] will assign 2 to t[0]. Try putting

  template <class T>
  void operator[](T* p, size_t i, const T& v) { p[i] = v; }

in a header file in your next project for fun. Then use t[i, v] for all your array assignments.

Why couldn't Python do something like:

    >>> _tmp = getitem(t, 0)
    >>> setitem(t, 0, _tmp)
    >>> try:
    >>>     _tmp2 = _tmp.__iadd__([2])
    >>> except AttributeError:
    >>>     _tmp2 = _tmp.__add__([2])
    >>> setitem(t, 0, _tmp2)

Then if the first setitem raises, the rest of the code will be skipped. Obviously this isn't perfect:

1. If setitem has strange side-effects, this will cause them to happen twice. 2. If setitem raises based on the value (rather than the key), this won't notice that.

But otherwise, it seems like a pretty straightforward improvement, no?

The example look ill-conceived to begin with. If you want to extend the list that is the first element of the tuple just use t[0].extend([1]). Using an augmented assignment operator makes no sense if the left hand side can not be assigned to. I don't know Python well enough to tell why it does not check if the left hand side can be assigned to before evaluating the right hand side, i.e. why it dos not check that __setitem__ is present. This would make the behavior more similar to a compiled statically typed language where this would be a compile time error and the right hand side would never get evaluated.

In fact, the second half of the article is about why it doesn't do that. Also, Python is not a compiled statically typed language, and doesn't have compile errors.

I ran into this when computing PSNR comparisons using a Numpy function that returned a numpy integer type:

    >>> import numpy
    >>> x=0
    >>> y=numpy.int32(2*10**9)
    >>> x+=y
    >>> x+=y
    >>> x
    -294967296

In later versions of Python this gives a RuntimeWarning so this is less likely to cause problems now.

Python 2.7.2 on OS X 10.8.5 behaves even more strangely. I get:

   >>> import numpy
   >>> x=0
   >>> y=numpy.int32(2*10**9)
   >>> y
   2000000000
   >>> x+=y
   >>> x
   2000000000
   >>> x+=y
   >>> x
   4000000000
   >>> x*x
   -2446744073709551616
   >>> y*y
   __main__:1: RuntimeWarning: overflow encountered in int_scalars
   -1651507200

I didn't think it was possible to get integer wraparound in Python. I guess it's because the "int32" from numpy "breaks" x, which subsequently no longer behaves like a python variable.

Official FAQ is easier to understand. I recommend reading that first before the article.

This is a wonderful explanation of a deeply confusing situation. Thank you for posting.

Reply on news.ycombinator.com