How we made Python's packaging library 3x faster

As a pip maintainer I primarily think about resolution and resolver performance. I got involved when pip introduced it's current resolver around late 2020, and got my first PR landed in mid 2021.

I recently became a packaging maintainer, from working on fixing edge case behavior around specifiers and prerelease versions.

When I did some recent profiling I noticed that A LOT of time was being spent in packaging, largely parsing version strings. I found a few places in pip and packaging that reduced the number of Version objects being created, Henry really ran with the idea of improving performance and made big improvements. I'm excited for this to be vendored in pip 26.0 coming out at the end of January.

If anyone is interested the next big improvement for pip is likely to implement a real CDCL (Conflict-Driven Clause Learning) resolver algorithm, like uv's use of pubgrub-rs. That said, I do this in my spare time so it may be a year or more before I make any real traction on implementing that.

I've been following this development on the PyPA Discord.

"Python's packaging library" in the title means the library actually called `packaging`. This is an important vendored component of pip and also gets used by other tools, such as the various build backends and the reference `build` frontend. So quite a few people will benefit, non-obviously.

This was a fun read, and my first time seeing 'asv' used to track performance over time. I was surprised to see the regexes used to capture version info, I thought it would be a custom parser