Hacker News Reader

Comment by jnbiche | Hacker News Reader

Please expand on your assertion. How are you storing your "two ints"? Two int tuple? Dict? List? I don't even have enough information to respond.

I do agree that ctypes is primarily meant for compatibility, but it can be used for speed optimizations in certain cases, particularly when there is a speed penalty for non-continguous storage of data structures (like a list of dicts, or list of tuples).

Edit: Thanks for the additional information. I totally agree that Python build-ins are often faster, as in this case. However, there are times when no build-in is available, and you have to use a custom function (usually from C). This gets back to the compatibility question, and I totally agree that ctypes is primarily meant for compatibility.

View on HN

Apologies for the confusion. The comparison was against a new-style class as described in the original post. All of the types have two ints. The code is now linked so you can inspect and respond.

Edit to respond to edit: Built-ins and slots are even faster but the key point is that the ctypes-based class is actually slower than simply using a class with ordinary Python variables.

The reason ctypes is slower is that addition isn't defined on c_ints which requires casting them back to Python ints for each operation. This can be avoided by using ctypes to call a C function to perform addition.

Response to edit: you may be right, I don't have time to do comprehensive benchmarking now. I never use ctypes on its own, but rather always with an external library. I assumed that the ctypes Structures in a ctypes array would be faster because its stored contiguously, but it's possible that is incorrect. I'll have to come back to this later when I have time.

I stand by my assertion that ctypes along with an external C library is a great way to do Python speed-ups. It's very simple to do, see here:

http://www.jiaaro.com/python-performance-the-easyish-way/

This is the kind of optimizations I usually use ctypes for, or for interfacing with a third-party shared library.

The ctypes structures are stored contiguously with a C-compatible memory layout, but every time an element is accessed, a new Python object is constructed to hold that return value. It's like it defines getattr to call int(<C value>). That's why they end up slower than regular classes - every access needs to create a new object.

ctypes + C code can be quite efficient, but you have to write the entire fast-path in C, not flip-flop between C and Python. It's best when you have a certain operation that needs to be fast (say, serving your most common path on a webserver, or running a complex matrix operation in Numpy), and then a bunch of additional features that are only called once in a while.

Reply on news.ycombinator.com