Faster CPython at PyCon, part two | Hacker News Reader

Faster CPython at PyCon, part two

lwn.net

81 pointsjwilk37 comments

Part one: https://news.ycombinator.com/item?id=35902506 (> 140 comments)

It is unclear to me whether Python 3.12 will receive significant improvements. Based on the information from https://github.com/faster-cpython/benchmarking-public, it appears that there may be a 2% performance enhancement. Is this the anticipated result, or are there additional developments awaiting merger?

Initially, the "Shannon Plan" (https://github.com/markshannon/faster-cpython/blob/master/pl...) aimed for a 50% improvement with each release. Has this goal been deemed unattainable, or are there adjustments being made to the plan?

lots of big ideas are still remaining to be done. One example is the register based interpreter, see https://github.com/faster-cpython/ideas/issues/485

A previous plan called for the beginning of a JIT in 3.12, seen as "Trace optimized interpreter" here: https://github.com/faster-cpython/ideas/wiki/Workflow-for-3....

but no longer scheduled for 3.12: https://github.com/faster-cpython/ideas/wiki/Python-3.12-Goa...

Isn't the feature freeze in a couple days though, or does that not apply to CPython?

CPython is the official Python implementation, as I understand it.

They measure different 3.12 forks, maybe these forks implement different ideas and combined version will be faster.

I wonder why much speedup could be gained by having a control over Python’s dynamism. Dynamic attributes, etc. are not needed in most of normal application code that is not meta heavy. Could there be any gains via making backwards incompatible changes e.g. by sealing some modules or functions to be “static” only?

It would not matter if a subset of an application is partially incompatible, as often one wants to optimise smaller hot code paths only.

Isn't that the whole idea behind mypyc? I don't know what performance gains are realized though, as I haven't tested it. We're in the process of typing a 300 KLOC codebase mostly for the correctness benefits, not so much performance.

Mojo is taking this approach. I think it's a good idea.

Mojo focuses on numeric highly parallel workloads only as far as I know. Does it offer anything for normal SaaS applications?

I assume one could just use 'fn' and 'var' and 'let' and get all the nice benefits of the static-ness of those features in any kind of code.

One thing you can do as a user already today to reduce Python dynamism of classes is using __slots__: https://wiki.python.org/moin/UsingSlots. I enjoy this a lot, not only for the memory and access performance boost, but because it also makes the code more rigid (and thus more resilient to monkey code attacks).

Yes, already using __slots__ heavily.

Since Python 3.10 you can also do

     @dataclass(slots=True)
     Class MyClass:
         pass

Congratulations to the Faster CPython, it's nice to see all the improvements the implementation is getting lately. That makes me wonder if ESR could have pulled the GCC repository migration without resorting to a Go port of Reposurgeon, had he access to a VM with current improvements, back then. See http://esr.ibiblio.org/?p=8161

There's a popular error that people make about speed when you see a steady diet of this microbenchmark improving by this many hundred percent and wow, this other effort to speed up is paying off, and so one assumes that the thing in question must be fast now. This is not the case. Something can get many hundreds of these little tiny improvements, but if the underlying architecture is slow and the little microbenchmarks were really just pecking around the edges, the net improvement can end up very small.

This is where CPython lives. CPython has been getting sped up for the last 15 years. The end result is not a blazing fast language knocking the socks of the competition; the end result is that as they've added features, they're been successfully able to more-or-less tread water or make a bit of progress, and they have remained merely one of the slowest languages in common use, instead of unusably slow.

Picking off a few improvements in instruction dispatching in CPython wouldn't have changed Python reposurgeon's capabilities much at all.

The gulf between Python and Go isn't a matter of a few dozen percent, it's a matter of a few dozen times in terms of speed and generally substantial integer multiples on memory use too.

As I said yesterday, many people interpret this as an attack on Python. It isn't. It is just something you need to know to make good engineering decisions. I've used Python many times in the past several years, and I've never been bitten by its performance issues... because I think about it before I use it. It is definitely something you need to think about before you reach for it. It actually was OK for reposurgeon for the most part... it just couldn't handle the largest repos, but those are the exceptions, after all.

Considering there's been data and benchmarks provided to support the argument that the performance improvements have been substantial, can you provide any evidence to support your argument against it?

Otherwise, your argument isn't very compelling, especially after having admitted that you haven't even used Python in ways that would warrant any assessment of performance for those use cases.

The largest number I have seen for microbenchmarks is around 50% speedup in comparison to the last version before this project started.

So, if that version was 40x slower than C (and that is very optimistic), the latest version is still 30x slower. That is the point that the GP made.

I would not call that substantial at all. I'd prefer to keep the CPython interpreter simple, people who want faster Python can use pypy or other things.

The real work in scientific Python will always be done in C or CUDA extensions. No 50% speedup will change anything.

Immaterial to the substance of your point but a 50% speed-up is an improvement from 40x to 20x, not to 30x.

        X’  X’/Y   X’/Y
  0.5 = – = –––– = –––– 
        X   X/Y     40

        X’
    =>  – = 20
        Y

Depends on how you define speed-up. This is the most natural definition for me:

As a baseline, my code can do 100 operations / second.

With 50% speedup (+50%), my code can do 150 operations / second.

If C was 40x faster than the baseline, it could do 4000 operations / second. After the speedup C can still do 27x as many operations per second. (4000/150 = 26.66..)

Well, that’s true. The good old miles per gallon vs. liters per 100 kilometers conundrum.

Stop interpreting sane evaluations of the speed of a language's implementation as an attack.

The evidence that Python is still slow is abundant and overwhelming. It's not like it's a subtle thing. It's a huge gulf. If you don't understand this, the problem isn't that jerf on Hacker News didn't spell it out to your satisfaction; the problem is that you are not evaluating things calmly and sensibly from an engineering perspective, but are using your political brain to evaluate programming languages. Until you stop doing that, you will never understand what is going on; once you do stop doing that, the truth will be so obvious you won't need me to spell it out for you.

> The gulf between Python and Go isn't a matter of a few dozen percent, it's a matter of a few dozen times in terms of speed and generally substantial integer multiples on memory use too.

I think here is your problem. You're expecting dynamically typed, interpreted language to be comparable to statically typed compiled language.

Python is best used as a glue language, or when performance is not critical.

That doesn't mean you can't have fast code with python. I for example was able to encode a live video into using x264 then steam it over the Internet in python, then on the other side receive the data, decode and display it.

Though the encoding/decoding was done by a C code that was interacted with through python package.

I suppose I could have the whole thing done in C, but then my code would be more complex and longer. It will also be harder to iterate on it.

I don't see the GP expecting anything. GP talks about good engineering decisions, using C-extensions when appropriate would be one of those.

But C-extensions don't require the small speedups and additional interpreter complexity that this project brings.

Reducing the memory overhead per Python object, as described in this LWN article, certainly would have helped. Whether it would have helped enough, though, is a hard question.

ESR mentioned in a comment [1] that he hadn't profiled the Python code in years, which would have been a sensible first step if he's concerned with performance. I suspect there were parts that could have been improved.

Maybe all those debug logging statements in the last Python version [2] that, unlike the built-in logging framework, use the string formatting operator to construct the log message regardless of whether debug logging is enabled.

I can't imagine that encoding everything to latin-1 as a workaround for Unicode support is very good for performance either, unless he was only using the Python 2 version.

And maybe, for example, the empty_comment function doesn't need to call the str.strip method twice on the same string? Maybe that could be more efficient?

[1] http://esr.ibiblio.org/?p=8161&cpage=1#comment-2065946

[2] https://gitlab.com/esr/reposurgeon/-/blob/911d5c1168f7839855...

IIRC, late in the port he reported they quashed some previously unknown O(n^2) bottlenecks. Backporting those fixes would surely help.

Anyone know why Guido seems to have reversed course in terms of keeping the CPython interpreter as simple as possible & is now supporting the addition of various performance improvements to the interpreter?

Not a python programmer, but I would guess that a) the language's semantics are pretty well-understood now and I wouldn't expect too many new control structures etc to come in (changes that would be much harder to make to an aggressively-optimised interpreter) and b) python is now so widely deployed that even small performance improvements are going to make large aggregate improvements to total time spent evaluating, total energy usage, etc.

I have a hard time understanding what "pretty well-understood now" means. Could you elaborate?

The match statement was added in 3.10. That's a control structure with a pretty big change in semantics, right? And done only two years ago - well after this performance push began.

What sort of changes do you think are precluded now, that might not have been precluded 10 years ago when things were less optimized?

Since you aren't a Python programmer, you can't really answer these questions, which make me feel like your response muddied the waters rather than providing useful insight.

If you think that I do not have the right to participate in this discussion, why are you asking me to continue the conversation?

I do not think that. And I could be wrong.

It seems to me that he has tried to keep Python for the web relevant at Dropbox by adding typing and asyncio, but Dropbox switched to Golang.

Now he is trying to keep it relevant for the web by adding the small speedups.

(I say "he", actually most of the work is done by other people.)

I guess the goal is to make python to do all the logic glue _and_ the computation? otherwise, just using it as a logic glue language, and offload anything memory intensive or computation intensive to a C library(e.g. numpy), that shall work pretty well to me.

to make Python as efficient as C will be great of course, but is it really that critical? it's a scripting language after all, and all script language all have similar "problems" by design: they're designed to glue, not to compute to me.

a different approach to me is to improve the CFFI, e.g. how to avoid memory copies and data passing, that might be easier, I'm not sure.

in the end, any efforts to make python (along with its modules) faster is nice to have.

Python as efficient as C is not the goal, the goal is to make python faster.

Just read this and part one. Came up with an optimization idea: use page write guards to skip some of the adaptation checks. E.g. instead of checking every time an optimized block is called, that a type was not modified, the check could be removed completely, and instead the page that has the type can be marked read only. When that page gets a fault optimizations in all the associated adapted blocks of code are reevaluated.

Does that work with Python's reference counting? Aren't all objects everywhere updated all of the time to track the reference count?

Good question, I suppose if GC head (another set of fields) and reference counter are placed just before the locked page, it would.

Reply on news.ycombinator.com