Cython 0.23 released

Cython is amazingly productive for me. I am so happy to write typed pythonic code.

If there's one thing missing it's more people showing off their Cython style and tricks, like how they do macros (since it's not supported in the language), when to turn off safeties, and balancing pure python with typed.

For instance, I like C macros in header files, but I've played around with using a jinja2 pipeline in my setup.py to write cleaner looking pythonic code, and have done generics this way.

more posts like the one using cymem would be welcome.

To answer the question, that came up:

Yes, the 0.x number also put me down at first. But I am using this tool now several years, since 0.13 or maybe earlier and did never find a bug or problem.

Cython is great, in-spite of its version number!

I also inspected the generated code several times and it looks rock-solid. Also one big advantage (for me): It can produce code that runs equally on 2.x and 3.x versions (there are some limitations, though, like several really outdated versions are not supported any more).

People should really think about the message a 0.x version number is sending. This project is 8 years old. Why is this still 0.x?

If you use Python for anything remotely performance sensitive, I highly recommend giving Cython a shot. The trick is to find your computation-intensive inner loop stuff, move it off to its own module, add types, and go. It makes your build a little more complicated (i.e. most Python projects have no build process at all) but if you're in the niche where it's useful, it's damned useful.

Here what it does. Take a look at this Python code:

    import math
    def distance(x1, y1, x2, y2):
        return math.sqrt((y2-y1)**2+(x2-x1)**2)

It looks simple, but it's doing a lot behind the scenes. It respects __sub__/__add__ metamethods, respects and promotes between floats/longs/ints and even Bignums, it has to do a __getattr__ on the `math` object to find the sqrt function in the first place, etc (and can't cache that because this is Python so you may have changed it in between calls). After every one of these calls, a Python exception is checked for, and many reference counts are incremented and decremented. Every now and then, the interpreter checks to see if it's time to release/reacquire the GIL and if any OS signals have come in. And it does all of this by compiling Python to bytecode and running that through its interpreter.

Here's the same code in Cython:

    from libc.math cimport sqrt
    cdef double distance(double x1, double y1, double x2, double y2):
        return sqrt((y2-y1)**2+(x2-x1)**2)

Cython takes that code and compiles it to this C code (a bit simplified):

    #include <math.h>
    double distance(double x1, double y1, double x2, double y2) {
        double a = y2-y1;
        double b = x2-x1;
        return sqrt(a*a+b*b)
    }

So now instead of running through dozens of intermediate internal Python functions, you're just loading the numbers into your FPU and doing the computation there.

Of course, you've given up all of your Python dynamicness. Now this won't work with Bignums, it can't be monkey patched, and now you have a compiling step to running your Python code. You can do this to your existing code one function at a time, so you don't have to give those things up for your entire project, just where it matters. And where performance matters, it's a godsend

I ported a small number of pieces of reddit to Cython. Here are some simple examples:

* https://github.com/reddit/reddit/blob/master/r2/r2/lib/db/_s...

* https://github.com/reddit/reddit/blob/master/r2/r2/lib/utils...

* https://github.com/reddit/reddit/blob/master/r2/r2/lib/mr_to...

* https://github.com/reddit/reddit/blob/master/r2/r2/lib/wrapp...

As the underscore naming implies, these are only selected performance or compute sensitive functions factored out of their normal non-underscore Python modules which generally wrap them. These functions were coming up disproportionately in profiling because they're called very often and/or do a lot of arithmetic, and afterwards they are entirely nonissues. Often something like a _sorts.pyx:_confidence call got over 25 times faster.

I've also talked about the downsides in a thread here[1]. Go in knowing that stuff too, but after my excellent experience with Cython I'm a True Believer.

[1] https://www.reddit.com/r/Python/comments/2xlya2/anyone_willi...

Does it support/is-compatible with the new python type hints?

I'm glad to see that this import project continues to get updates.