Hacker News Reader

Comment by NelsonMinar | Hacker News Reader

namedtuple is particularly weird. When you create a new type of namedtuple, Python creates the source code for a new class from a string template and passes it to exec() to build the class. It's a clever and relatively straightforward bit of metaprogramming, but it's not a suprise it's not fast. https://hg.python.org/cpython/file/b14308524cff/Lib/collecti...

(BTW if you need a quick and dirty way to make Python start up faster, see if you can use the -S flag for your application. It skips the site-specific initialization and has a significant impact on startup times.)

View on HN

I'm curious, given the presence of a ".pyc", shouldn't these kinds of costs be paid primarily on first invocation "ever"?

Not in a general case, because eval/exec accepts an arbitrary string which could change at runtime.

No, because the string given to eval will not be stored in the pyc – it has to be compiled on every import.

It can't be cached? Is not caching a dependency issue?

It can't, because lol dynamic dispatch. Exercise: what does this code do?

    import collections
    collections.namedtuple = lambda _0, _1, _2=False, _3=False: "Where is your god now?"
    import b

b.py:

    import collections
    print(collections.namedtuple('Point', ['x', 'y']))

And this is not even particularly 'evil' code, in the sense of relying on internal implementation details; that's just your bread-and-butter Python language semantics. It prevents (or at least complicates) many other potential optimisations as well.

You should link to the GitHub repo instead of Hg. They put effort into the move.

https://github.com/python/cpython/blob/v3.6.2/Lib/collection...

Cool use of eval. I wonder if there are metaprogramming aimed libraries for generating eval'd code.

Why would you want that when you have actual metaprogramming on code-as-data in the standard library?

Is this a joke on the namedtuple trick or is there actually such a module in the standard library?

Not a joke, but "code as data" was hyperbolic.

You can do some pretty amazing metaprogramming in Python with metaclasses, and the inspect and ast modules give you access to the guts of the language.

Really? Huh. I would have expected a metaclass.

While using a metaclass is indeed cleaner, it's slower than generating code on the fly and exec-ing it.

This video talks about it: https://youtu.be/sPiWg5jSoZI?t=1h37m18s

I guess that's how Python 3.6's typing.NamedTuple works, which is frankly a much nicer syntax for named tuples.

It's not actually; typing.NamedTuple is just a wrapper that calls the original collections.namedtuple. It does provide some additional features, like default values and methods.

> It's a clever and relatively straightforward bit of metaprogramming

Wat? It's neither of those things.

Should we take an informal survey?

Reply on news.ycombinator.com