Hacker News Reader

Comment by j_baker | Hacker News Reader

Does anybody write C extensions like this anymore? Most of the time it's easier to use swig or cython. If it's really simple, you can also use ctypes.

View on HN

Actually, a lot of people do write them this way. When you take something you can write quickly and with no dependencies other than python, and instead pull in swig or cython - you now have two problems instead of one.

As the author shows, writing a python c extension is fundamentally easy which is why Python has such a huge ecosystem of them to begin with.

Writing C extensions is only easy if you do trivial things. As soon as you do something significant, writing all this C code by hand becomes increasingly complex, especially because of reference counting and error management (http://docs.python.org/release/2.5.2/ext/thinIce.html).

If dependency for distribution is an issue, you can just include the generated C code (as we do in numpy and scipy, as a matter of fact). Also, cython can generate both python 2 and 3 compatible code.

It depends on what kind of "trivial". I have a lot (> 30000 SLOC; hundreds of functions) of C code that does intricate math/stats/optimization, but from a control flow and memory allocation point of view is simple. This kind of code is perfect for the simple c-extension API of Python as described in the OP.

In fact, if you set up the gateway interface to your C code cleanly, you can have the same C code callable directly from (e.g.) Matlab via the "mex" mechanism.

Yep, in that case, that's relatively easy - but cython is even better. Cython is used a lot in the scipy community - if you don't know it, you may want to look at it. Since I am using cython, I am trying to avoid writing raw C API as much as possible, there is just no point.

I wrote a few things in cython to give it a try and it's amazing what static typing can do to improve the efficiency of the code. Just don't look at the auto-generated .c file though, it looks scary, but this seems to be a common occurrence (swig comes to mind).

SWIG has its own problems. Dependencies have already been mentioned, trusting obfuscated code is another issue:

http://www.artima.com/weblogs/viewpost.jsp?thread=95863

"But even if I find the cause of the leak, I may not be able to plug it. M2Crypto is written using SWIG, which contains lots of obfuscated code and doesn't exactly give me a warm fuzzy feeling that it's always doing the reference counting correctly."

I prefer a clean handwritten C extension any day. There is a lot of repetitive work involved, but most of that is really just copy&paste, so it tends to go quickly.

Reply on news.ycombinator.com