Python Oddities | Hacker News Reader

Python Oddities

twitter.com

Vast majority of these are not actually oddities. A few stuck out at me, though:

- since True/False are aliased to 1/0, you cannot have both 1 and True as dictionary keys. This could plausibly happen in real code.

- there's no distinction between args/kwargs, at least when it comes to a function's __defaults__ attr. An arg can be turned into a kwarg by messing with the __defaults__ attr.

- you cannot have a nested tuple/dict/list with more than sys.MAX_RECURSION levels, because each list access apparently counts as a new call frame? (This could plausibly happen when generating a dict from json or html or xml)

> there's no distinction between args/kwargs

At the python level, all named args[0] are kwargs. The reverse is not true in Python 3, although it is in Python 2. At the C level, arguments can be exclusively one or the other even in Python 2.

> at least when it comes to a function's __defaults__ attr. An arg can be turned into a kwarg by messing with the __defaults__ attr.

No, an arg can be turned optional by manipulating __defaults__, that is a different (though not quite orthogonal) axis, especially in Python 3 (which supports python-level required keyword arguments).

> - since True/False are aliased to 1/0, you cannot have both 1 and True as dictionary keys.

They are not aliased (`True is 1` will return False) but they are equal and hash identically.

[0] *args notwithstanding

I’ve got one! First, some context:

This is normal assignment unpacking:

    >>> a, b = 1, 2
    >>> [a, b]
    [1, 2]

(Mismatch gives errors)

    >>> a, b = [1]
    ValueError
    >>> a, b = [1, 2, 3]
    ValueError

Extra parentheses works too:

    >>> (a, b) = [2, 3]
    >>> [a, b]
    [2, 3]

Unpacking a single value:

    >>> (a,) = [4]
    >>> a
    4

Works without parentheses too:

    >>> a, = [5]
    >>> a
    5

No values does not work:

    >>> , = []
    SyntaxError

Neither does empty parentheses:

    >>> () = []
    SyntaxError

Now, the oddity:

    >>> [] = []
    >>>

This raises no error! Neither does this:

    >>> [] = ()
    >>>

This isn't doing what you think though. There are container objects underneath.

For instance "(a, b)" is "a tuple object containing two things, the values named "a" and "b" so you can still refer to them separately. "[a, b]" is a new list with those same elements, not an alias for "(a, b)".

The oddity is that unpacking an empty iterable works when the target is a "list", but not when it's a "tuple". In fact it so doesn't work it actually fails during parsing.

And there are no collections underneath the LHS, it compiles to an UNPACK_SEQUENCE opcode in both cases:

    >>> @dis.dis
    ... def foo():
    ...     (a, b) = bar()
    ... 
      3           0 LOAD_GLOBAL              0 (bar)
                  3 CALL_FUNCTION            0 (0 positional, 0 keyword pair)
                  6 UNPACK_SEQUENCE          2
                  9 STORE_FAST               0 (a)
                 12 STORE_FAST               1 (b)
                 15 LOAD_CONST               0 (None)
                 18 RETURN_VALUE
    >>> @dis.dis
    ... def foo():
    ...     [a, b] = bar()
    ... 
      3           0 LOAD_GLOBAL              0 (bar)
                  3 CALL_FUNCTION            0 (0 positional, 0 keyword pair)
                  6 UNPACK_SEQUENCE          2
                  9 STORE_FAST               0 (a)
                 12 STORE_FAST               1 (b)
                 15 LOAD_CONST               0 (None)
                 18 RETURN_VALUE
    >>> @dis.dis
    ... def foo():
    ...     [] = bar()
    ... 
      3           0 LOAD_GLOBAL              0 (bar)
                  3 CALL_FUNCTION            0 (0 positional, 0 keyword pair)
                  6 UNPACK_SEQUENCE          0
                  9 LOAD_CONST               0 (None)
                 12 RETURN_VALUE

It’s a good point that Python is smart enough not to build a mutable list on the right-hand side when not needed (e.g. for "x = [1, 2]" it will and for "y, z = [1, 2]" it will not). A right-hand tuple however is still a separate type, and if you disassemble the original cases you can see that "[] = []" is more wasteful than "[] = ()".

  >>> import dis
  >>> 
  >>> @dis.dis
  ... def f():
  ...     a, b = 1, 2
  ...     (a, b) = [2, 3]
  ...     (a,) = [4]
  ...     a, = [5]
  ...     [] = []
  ...     [] = ()
  ... 
    3           0 LOAD_CONST               6 ((1, 2))
                3 UNPACK_SEQUENCE          2
                6 STORE_FAST               0 (a)
                9 STORE_FAST               1 (b)
  
    4          12 LOAD_CONST               2 (2)
               15 LOAD_CONST               3 (3)
               18 ROT_TWO             
               19 STORE_FAST               0 (a)
               22 STORE_FAST               1 (b)
  
    5          25 LOAD_CONST               4 (4)
               28 STORE_FAST               0 (a)
  
    6          31 LOAD_CONST               5 (5)
               34 STORE_FAST               0 (a)
  
    7          37 BUILD_LIST               0
               40 UNPACK_SEQUENCE          0
  
    8          43 LOAD_CONST               7 (())
               46 UNPACK_SEQUENCE          0
               49 LOAD_CONST               0 (None)
               52 RETURN_VALUE

Hmm, why is that an oddity? Unpacking a list of N items into N variables works fine, where N >= 0.

It works when you use a "list literal" as target, but with a "tuple literal" it only works for N > 0.

This appears to no longer be the case in Python 3:

    $ python2
    Python 2.7.13 (default, Jul 21 2017, 03:24:34) 
    >>> () = []
      File "<stdin>", line 1
    SyntaxError: can't assign to ()
    >>>

    $ python3
    Python 3.6.2 (default, Jul 20 2017, 03:52:27) 
    >>> () = []
    >>>

> This appears to no longer be the case in Python 3:

It was fixed in Python 3.6 specifically, the error still occurs under 3.5.

Some of these are pretty simple and not odd, e.g. immutability of tuples, iterator behaviour or modifying collections during iteration.

—————————————————————————————————————

Somewhat related video: https://www.youtube.com/watch?v=qCGofLIzX6g

Good thing CPython is the spec for the language.

That said, many of these "oddities" are disingenuous to the point of lying, many of the error messages are omitted. Or the functionality converts to list before doing an operation. Not that it is good library or language design, but this is a great way to inflate bug counts.

I think you may be taking the word oddity different than I meant it. These are things that trip up my students when they're learning Python.

I left out error messages (and occasionally other things) because Twitter.

I don't think any of the ones I put in this moment should be considered bugs. Many of these are core language features.

Some of them seem like litmus test for a Python programmer interview, others feel like WTFs Python LoL! It is hard to disambiguate them. Standing on their own in that twitter feed, with little context, there is next to no enlightenment.

Lots of the oddities could be explained by

    In [1]: list({'a':1, 'b':2, 'c':3})
    Out[1]: ['a', 'b', 'c']

When people would expect

    In [2]: list({'a':1, 'b':2, 'c':3}.items())
    Out[2]: [('a', 1), ('b', 2), ('c', 3)]

Lots of Python makes sense. About 3% does not. It is good to know the 3% so it doesn't bite one in the ass. Maybe switch to image posts? 1 megapixel limit.

Am I the only one who hated the mobile pop-out slideshow Twitter uses for this?

Also, the one about appending to arrays didn't seem too crazy.

Here's one from https://stackoverflow.com/questions/15171695/whats-with-the-...:

  In [1]: a = 1

  In [2]: b = 1

  In [3]: a is b
  Out[3]: True

  In [4]: a = 257

  In [5]: b = 257

  In [6]: a is b
  Out[6]: False

Yes, because the smaller ints are 'pre-constructed' in Python (hence all 1's are the same "int(1)" object)

Aha, three ways to concatenate arrays! I see Python conforms to the TIMTOWTDI principle. :)

There should be one _obvious_ way to do it. The obvious way is using +, and is sufficient for 90% of programs.

Reminds me of WTFJS: https://github.com/denysdovhan/wtfjs/blob/master/README.md

    >>> a = [1]
    >>> a += (2,)
    >>> a
    [1, 2]

    >>> b = (1,)
    >>> b += [2]
    TypeError

It's an oddity that an immutable type cannot be mutated in place?

This one is an odd one because initially most people I show this one to assume the oddity is that immutable types cannot be mutated in-place.

But tuples can be mutated in-place: >>> b = (1,) >>> b += (2,) >>> b (1, 2)

The odd thing is that lists can be in-place added (+='d) to any iterable, while tuples can only be in-place added to tuples.

I think it works this way so that += on lists works consistently with the extend method on lists, which also accepts any iterable.

> But tuples can be mutated in-place: >>> b = (1,) >>> b += (2,) >>> b (1, 2)

That's not an in-place mutation any more than

    >>> a = 1
    >>> a += 2

is.

The oddity is that most people assume `a += b` is the same as `a = a + b`, but Python allows overloading both separately and list.__iadd__ is an alias for list.extend, hence being able to += any iterable onto a list whereas `list + non_list` generates a type error.

The original intent is an optimisation (to avoid copying the subject list) but it's a broken one as it behaves very inconsistently with regular concatenation/addition: it mutates the subject in-place and allows a different (wider) set of parameters.

cpython oddity: The following program (call it "20.py") takes a minute to run, but the reported time for the "x=..." line is a few microseconds.

  import time
  if 1:
    t1 = time.time()
    x = ((((((((0,)*20,)*20,)*20,)*20,)*20,)*20,)*20,)*20
    t2 = time.time()
    print(t2-t1)

  % /usr/bin/time python 20.py
  3.09944152832e-06
         70.91 real        70.91 user         0.24 sys

That's probably ARC.

ARC? The GIS language? Age Restricted Content? Advance Reading Copy? Activity-Regulated Cytoskeleton-associated protein?

Automatic Reference Counting, i.e. garbage collection. That's why the timer wouldn't attribute the CPU time to the line you mention; the GC cleans the structure up when the "x" goes out of scope, so after the timing has ended.

Thanks. No, it's not due to reference counting, at least, not according to the tracker issue which reports this oddity.

Add:

  import os
  os._exit(0)

to the bottom of the code and it still takes a minute to run, even though there's no final garbage collection.

Also, that code creates only a few hundred objects, and the 'x' data structure is acyclic.

Best use of Twitter Moments I've ever seen.

First use of Twitter Moments I've ever seen. What is this? A collection of tweets? I'm confused because they all seem to be by the author, did he tweet them himself and collected them?

The page's UI looks weird, I'm not sure if Twitter had a redesign or if I'm blocking some of their CSS.

I think they're pretty much a collection of tweets. This was the first time I used moments and I'm not sure where I saw them used before I made this.

They're mostly by me because I made a goal for myself to tweet out one of these every week for a year. Others have shared their own and I've tweeted my own since then, but Twitter's hashtag search is pretty bad so I made a moment.

I think it works quite well, I just hadn't seen the feature before. I disagree with you a bit, though, in that many seemed pretty regular, and weren't really oddities, in my opinion.

There's no oddity in being able to extend an array in several ways (.extend, += other array, etc)

Why does the _MyClass__number example work the way it does?

https://docs.python.org/3/tutorial/classes.html#private-vari...

Within class bodies, symbols prefixed with two underscores get "mangled" (prefixed with the class name).

This is intended to avoid risks of collisions for classes built specifically for inheritance so you'd usually use it with attributes e.g. `self.__name = name` (that way if a subclass defines its own "name" things will work as expected for both). I was unaware this also worked for straight local variables.

I was unaware it happened for all symbols, but I guess it makes sense as the Python compiler does very little analysis.

Reply on news.ycombinator.com