With a little bit of tinkering (multiprocessing, choosing the right libraries written strictly in python, PyPy plus a lot of memory) I was able to optimize some workflows going from 24h to just 17 minutes :) Good times...
It felt like magic.
If you've written it off on that basis, I'd suggest it's worth giving it another shot at some stage. It might surprise you.
Last I saw there was still a little bit more overhead around the C interface, so hot loops that just call out to a C module in the loop can be just a smidgen slower, but I haven't seen it be appreciably slower in a fair while.
> We have support for c-extension modules (modules written using the C-API), so they run without modifications. This has been a part of PyPy since the 1.4 release, and support is almost complete. CPython extension modules in PyPy are often much slower than in CPython due to the need to emulate refcounting. It is often faster to take out your c-extension and replace it with a pure python or CFFI version that the JIT can optimize.
https://doc.pypy.org/en/latest/faq.html#do-c-extension-modul...
I have seen great success with cffi though.
https://doc.pypy.org/en/latest/cpython_differences.html#exte...
""" The extension modules (i.e. modules written in C, in the standard CPython) that are neither mentioned above nor in lib_pypy/ are not available in PyPy. """
The lifecycle of generators makes pypy code very verbose without refcounting. I've already been bitten with generator lifecycles and shared resources. PEP533 to fix this was deferred. Probably for the best as it seems a bit heavy-handed.