You Should Compile Your Python and Here’s Why

Python, while slow, is "fast enough" because it's super easy to add native objects for anything that's too slow. It's kind of an inversion for the usual scripting language layout with fast native code loading slow snippets of script to provide customization, and tbh that's probably the better approach for most situations. Write the 99% of your application that's not performance critical in something quick and easy and flexible, and then optimize the heck out of the remaining 1% that matters.

(Not a card carrying Python fan or anything, haven't used it in a decade, I just like this architecture.)

> IOW, the question is not whether Python is "fast" or "slow", but whether it is "fast enough" or "too slow" for a given user.

100% this. There isn't REALLY any such thing as good/bad/fast/slow/cheap/expensive, there's just "meets spec better" / "doesn't meet spec as well". Everything is relative to your requirements.

> the Python intepreter has a slow startup time.

This really depends on your perspective. Sure, it's a lot slower than running a native binary. But it's still fast enough for interactive tools that you run often. If you compare that to java tools, for example, which take seconds if not dozens of seconds to start (gradle, I'm looking at you!), python is far better.

I don’t always know how fast will be fast enough when I start a project and the risk of needing a rewrite really puts me off Python.

(Thinking into the wild here…) Wonder if python could be sped up by not being loaded as usual per call. Ie something more like a service with careful namespace management. I’d think something that copies what’s already in ram to new registers would be much faster.

Function calls sadly are also very slow due to heavy overhead

Especially reading and understanding the "moderately" optimized Python version is a lot harder than reading the C version.

Also I really dislike python optimization tricks like `output = stdout.buffer.write`.

I really hate these kinds of articles because they don't DO anything useful in the sample programs.

Grab a web page and parse it for some data. Suddenly, C looks like trash. The network latency negates any speed advantage from C and the fact that C has to manually manage memory everywhere makes your C code look like a dumpster fire relative to the Python code. You can always pick a task that favors one language or the other.

What people forget is that fast/slow ALSO includes "time to develop the code". If I can write the Python code in 1 hour and the C code in 10 hours, the code has to be a lot slower or be run a lot more times before the time wasted writing the C code pays off.

And if he'd applied the same optimizations to the C version, it would have trounced the optimized Python version. I really don't like it when authors unfairly penalize their "not preferred" solution -- it makes me feel like the entire time spent reading the article was nothing more than listening to pointless chest thumping by the author. The whole article was barely about the actual compiling of Python code.

mypyc has escape hatches, e.g. casting to Any is a common trick.

What's AoC?

Cython is good for little numpy shims.

I think Python is simple to use but difficult to scale. It can be useful to think of programming languages as tools. Assembly is the workbench where tools are made and raw materials are handled. C++ is like your standard set of machining tools. Java is like a bag of tools, hammers, handsaws. Python is like an electric nail gun, or perhaps a gas chainsaw. It does many of the things the other languages say that you can do directly out of the box and often in a manner that’s more sensible to people initially using a tool. But when it comes to building a house and all you have is a chainsaw? You’re gonna have some very cool log cabins, and if we go outside the analogy for a moment: code that uses tricks to accomplish what others would do with better building, architecture, or execution. Now that said, how many people do you know who might go off and build a log cabin vs. a modern home on their own?

It's because Python is both simple and easy to use, and complicated and hard to understand.

It's a Janus of a programming language. There are lots of complexities that alternately tie software engineers up in knots and let them go into raptures of blissful hackery. But they generally aren't all that visible in the face it presents to data scientists and devops.

And it's hard to make sweeping statements about "big or performance critical". A couple times in the past few years years I've seen or been involved in projects that successfully replaced big Java applications with Python implementations that were 1% the SLOC and had better performance. They were special situations, to be sure, though more so, I think, in terms of explaining the performance improvement than the cost savings.

Reminds me of that Tao of Programming koan (if I remember it rightly) that went something like:

The novice, in his frustration, struck the side of his computer. The master walked over and asked what he was doing. The novice exclaimed, "my computer is not working and I do not know why!" The master admonished him, saying "you cannot solve the problem by striking the computer without knowing what is wrong." Then the master struck the side of the computer, and it worked flawlessly.

(I love this one because one time I was doing a gig and my computer refused to boot. Suspecting that the vibration from being lugged into the car and taken for a long drive had unseated something slightly, I gave it a sharp smack and power cycled it, and it started up fine.)

> I built up a string of ascii 0's and 1's, sliced the string, and then converted back to integers.

How about a list of integers:

  $ txr
  This is the TXR Lisp interactive listener of TXR 274.
  Quit with :quit or Ctrl-D on an empty line. Ctrl-X ? for cheatsheet.
  This could be the year of the TXR desktop; I can feel it!
  1> (digits 37)
  (3 7)
  2> (digits 37 2)
  (1 0 0 1 0 1)
  3> (reverse *2)
  (1 0 1 0 0 1)
  4> (mapcar (op - 1) *3)
  (0 1 0 1 1 0)
  5> (poly 2 *4)
  22

:)

> Today I did something terrible. I like to believe it was out of wisdom rather than ignorance. I needed to do some nuanced bit manipulations, and so I built up a string of ascii 0's and 1's, sliced the string, and then converted back to integers.

Similar, in awk:

https://gist.github.com/jaysoffian/e41ca479d70e60efe59fded93...

glyph is ridiculously good at the intersection of smart, funny, empathetic.

Probably for the same reasons people use Java and Go instead of C. It's just more convenient sometimes. But sometimes not.

In terms of building a team at a company? In my experience it's been arbitrary. Manager / Lead has experience in language X, decides to hire people who also know it. Or all the other teams are already using language X. Or Manager / Lead has heard "X language is good at Y" and decides to go with that. Or there's simply 10x more engineers (and cheaper) available for language X than Y.

The times I've seen a language picked for a particular purpose:

- Perl/Python used for web apps. It's interpreted so you can just upload your source code and refresh your browser. Faster and simpler than having to compile/package it, lots of useful frameworks and modules, lots of developers.

- Erlang/OTP for telecom. Kind of on the nose, but there you are.

- C and C++ for embedded applications.

I use mostly Python, with a little Golang in my job and my reasoning is:

1. Most of what I am doing is calling other programs, often over ssh. The speed difference between the two is going to be tiny (where it is not, I use Golang)

2. Python is a more ergonomic language to work in a lot of the time. Wrapping a series of steps in a try block and then handling the errors in one go (very common in the things I am doing) is just easier to write, easier to maintain, and easier to read than what I have to do in Golang. And there are a ton of modules in Python that are just easier to work with than in Golang. And the resistance to the 'while... else' and 'for... else' patterns just saddens me. And the 'for' implementation in Golang often makes me want to tear my hair out... why make it hard to do this by reference?

3. Even in places where Golang should be much better, sometimes it is a challenge. For example when I am trying to parallelize something, but want to only have N number of workers going at a time. In Python I just use the worker pattern and I am done, I can even feed from one set of workers to another. In Golang I have to use a limited channel, and be careful that I block on that channel before I do anything that is going to eat a lot of memory (otherwise I have protected the CPU, but not memory resources). It feels like I am fighting the language there.

Consider that a lot of ML programs sit at 100% CPU use (on a single core), and 50-60% GPU use (if you're lucky).

The common use of Python is a huge issue.

For what it's worth, go is still "new" in my book. I'm seeing 2012 for the first 1.0 public release. In contrast, we've had python since 1991. There's a large number of programmers who haven't had a chance to be exposed to go. I have used it, and enjoyed it. I'd expect, but haven't confirmed, that python has a larger collection of libs (especially long tail).

Although, looking now I'm seeing a short support window for go releaI. That can hurt uptake.

Like go 1.18 was released 2022-03-15 and will EOL Q1 2023.

Lots of projects want long term support so that churn isn't desired. I'm curious how teams handle that.

> Not to be shit-eating, but I never understood why people use python instead of Go

I've got a few sysadmin type scripts I wrote in perl that I'm currently rewriting in python because other people will use and need to update those things and they will be more likely to know python than perl. I can't imagine they'll be more likely to know Go either.

Python is incredibly easy to learn, so more people learn it as their first language these days.

I feel like most modern languages are so uninspiring. I wish there was some popular Lisp or something (I guess Clojure is pretty close, but JVM).

Most projects were started before Go was a thing.

You can use numpy just fine. Unlike Cython, mypyc doesn’t give you a way of accessing numpy’s C API (because you’re basically just writing Python code).

I would advise against optimizing your Python programs for speed. Fighting against a language's nature never leads to good results, it's much easier and better to use a faster language if (or where) you need the speed.

This is even more relevant if you are still learning the language. Focus on learning it, and leave arcane always changing implementation details for after you know it well.

There is currently a "Faster CPython"[1] project going on that seek to improve the speed of Python. With Python 3.11 later this year there will be several speed improvements targeting common uses of Python, which might make some of these existing speedup tricks irrelevant. Or at least less relevant.

[1] https://docs.python.org/3.11/whatsnew/3.11.html#faster-cpyth...

My 5-minute version:

- Get the overall system structure in Python. Get the architectural design right, and the big-O stuff right.

- If there are bottlenecks, re-code those directly in C, cython, or similar. Or better yet, find libraries.

Python is great for expressing high-level operations and system design. It is also very easy to integrate with native code. I've never had much happiness in optimizing Python itself beyond that. Broadly speaking, code falls into three categories:

- Most code: Instant. Performance doesn't matter. Python

- Some code: Big-O(lifespan of the universe). Not worth building.

- Narrow slice of stuff in between: Don't do in Python, but use Python as the glue.

Fluent Python; go for it!

There are plenty of things for which python is not slow, but people using python don't know a lot about programming, and end up with a slow result.

A couple of advises:

- the right algo will go a long, long way.

- know your data structures. E.G: assigning to a slice is ridiculously fast (even while unpacking), memory views may save a lot on byte heavy workloads, heapq and deque are underrated, etc. Also check out https://wiki.python.org/moin/TimeComplexity for big O notations on python builtin types common operations to understand what you pay for.

- know the stdlib. collections, itertools and functools all contain incredible gems.

- delegate. Python is a fantastic glue language, use it for what it's good at. Your database, your numpy arrays, your cache are all amazing are what they do, no matter the language. Let them do the heavy work.

- don't kill good perfs by ignorance. I regularly see people casting a generator, iterating on a dataframe, calling readlines() on a file or doing something else that is destroying the otherwise excellent perfs of their program.

- know the ecosystem. There are some very good fast libs out there: diskcache, sortedcontainer, scipy, uvloop...

- use threads to avoid blocking a GUI, multiprocess to share work between CPU and asyncio to speed up network operations. Each tool has a sweet spot. But threads are underrated, they work well for a couple of hundred parallel network operations, and most C libs will actually release the GIL, so they can use several CPU more often than you'd thin. Also, use pools if you can, shared_memory in 3.8 or mmap.

- sometime the dirty solution is just faster, like subprocessing to ffmpeg.

- The more recent, the slower. Sure, I love statistics, pathlib and dataclasses. In a regular code they are great. On a bottleneck however, they are very slow.

- the array module is not supposed to speed up the code, only save memory. But sometimes it does.

- printing to the terminal is limiting. Sometime your program is doing fine, the display is preventing it to go faster. At least check the flush.

- comprehensions are faster than alternatives.

- pre-allocating lists and dicts can help.

- measure. The austin profiler is your friend.

- rewriting the hot path in a faster language is likely more interesting than writing the whole program in it. A bit of nim or rust is easy to call from python.

- some python distribution are faster than others. I don't mean just pypy, but also regular cpyhon that have been built specially for some architectures. E.G: https://www.intel.com/content/www/us/en/developer/tools/onea...

- nuitka code compilation can speed up the code by a factor of 4, it's nice, especially for startup.

After reading the article last night. I spent the whole evening about 5 hours getting one of my DOM libraries to compile with mypyc. It's a hacky codebase with little to no type hinting. I had to rewrite a fair abit to appease mypy and in most cases just used 'Any'. It's still got runtime issues and is buggy but I got at least a 2x speed increase on rendering a single node... https://github.com/byteface/htmlx/blob/mypyc/benchmarks.md

However I was unable to compile from my mac, could only compile using linux. but that could be as I'm using older version of dev tools.

mypy compiles itself with mypyc, for about a 3.5-4x speed up. black very recently started shipping mypyc compiled binaries, for about a 2x speed up (iirc).

glyph's article goes into this case and finds a 25% speed improvement in that case.

That part is just about the naïve C

"Mypyc is currently alpha software. It’s only recommended for production use cases with careful testing, and if you are willing to contribute fixes or to work around issues you will encounter."

At least for now, I wouldn't use it in production.

The obvious catch is that the compiled binary is not portable. You need to compile and deploy for each platform separately. You don't need to worry about platforms at all when deploying/sharing pure Python modules.

Try looking into polars as a much faster pandas alternative.