CPython internals: Codewalk through the Python interpreter source code (2014) | Hacker News Reader

CPython internals: Codewalk through the Python interpreter source code (2014)

pgbovine.net

137 pointsavinassh16 comments

I've been hoping to come across something like this ever since Pat Shaughnessy's Ruby Under a Microscope (http://patshaughnessy.net/ruby-under-a-microscope). This looks fantastic!

Professor Guo writes great articles. His "PhD grind" is great not only for PhD students but also for people in early-stage startups.

I love this walkthrough! I've been going through it except with the python 3 interpreter.

I too am interested in doing it with the python 3 interpreter. Can you share the relevant link

Looks interesting.

Pedagogically, I wonder what the tradeoffs for using Python are. I haven't looked at it, in part because I've heard comments to the effect that it's a big hairy mess inside.

On the upside, students have a better chance of actually needing the knowledge than with Lua, for instance.

[author here] it's not too bad, actually; i mean there's the usual hairiness of anything implemented in C, but the interpreter doesn't try to do much in terms of clever optimizations, so the code base is fairly readable for a production-grade system. check it out!

I actually have it open, and so far, it's not as much as I thought.

Still, the interpreter I've previously looked at is Lua, so it's a lot bigger.

I agree with you that the code is pretty readable, and well-commented. But in my experience with the code-bases of MRI (Ruby), CPython, and Lua, IMO CPython is by far the hairiest.

Here is what I mean by "hairy": reading the Python source, you see all sorts of complexity that looks like it is the result of patching problems in previous versions. There are all sorts of comments of the form "we have to do this weird thing here because some obscure things breaks if we don't." The most extreme example of this I've seen is this monster comment, written in the style of a FAQ because it's so complicated:

https://hg.python.org/cpython/file/9beeb4c1375d/Objects/type...

I've had so many experiences of wondering "wtf is Python doing, I don't get it," only to find some strange behavior in the interpreter that I never could have anticipated. Things like this won't affect most people writing straight Python, but once you are writing C extensions the inner complexity of CPython becomes far more apparent.

Here is an example of a problem I ran into. I had a C extension that was working fine in Python 2.x and 3.3, but then I got bug reports from my users that it was broken in 3.4. It turned out Python 3.4 had added a restriction: it was no longer allowed to make a statically-allocated type derive from a dynamically-allocated type. Why the new restriction? It's not entirely clear: this rule isn't documented anywhere, and the bug/change that added the restriction has no explanation (http://bugs.python.org/issue22079). They probably discovered some scenario where allowing this could lead to problems, so they added some code to disallow it.

This is my experience working with CPython. Lots of subtle rules and semantics, many of which are not documented except as comments in the source.

P Guo is a Guy Steele level thinker and lecturer. This is good stuff, pay attention.

100% agree. He writes very incisive articles. He also created the online Python Tutor!

Awesome! I have been trying to find something like this for Lua or Perl

Like this[1]?

1: http://perldoc.perl.org/perlguts.html

No, that is a good guide is you are doing XS stuff

Ah, well, maybe one of the pages here is useful: http://perldoc.perl.org/index-internals.html

Saw this link on Stack Overflow the other day and have been watching attentively. Great videos - I found the presentation of the PyObject particularly helpful (lecture 4). I'd be overjoyed if there was an extension to the course in future!

Is there something like this similar for Javascript? V8 maybe?

Reply on news.ycombinator.com