Hacker News Reader

Comment by bionade24 | Hacker News Reader

If they support python packages exposing the C ABI and expose a C ABI for embedding, I can't see why not.

They're doing threading without the GIL, which means it's not compatible with existing C extensions.

CPython is also building a -no-gil compile-time option. If that gets traction, some of the big performance-oriented C extensions like Numpy etc would likely build a non-GIL extension version too.

Then, if RustPython's non-GIL implementation is compatible with CPython's, and their C ABI is compatible, those C extensions might work.

gil or no gil is mostly irrelevant for numpy, an extension that does most work with gil released.

Relying on the cpython gil and releasing the gil are two different thing:

  1. releasing the gil means multithreading is opt-in for a given code section in NumPy. Only very specific parts of the code need to be threadsafe.
  2.  not relying on a gil in cpython runtime means multithreading becomes opt out. Now all the code by default needs to be threadsafe, including the libs you depend on.

A lot of C/C++/Fortran scientific code is not thread safe, and the whole scientific python ecosystem depends heavily on those codebases.

I understand what you are saying, but I don't see how it's relevant? Numpy doesn't rely on the gil, which is why it releases it.

It releases the GIL only in very specific sections. Most of the Numpy C code runs under the gil.

A quick check on master shows only 10-20 calls using NPY_BEGIN_ALLOW_THREADS (which is an alias to Py_BEGIN_ALLOW_THREADS).

A lot of the NumPy code manipulates python runtime objects, and doing so without thread safety would likely break everywhere. A lot of efforts would be needed to gradually make large C extension thread safe.

Reply on news.ycombinator.com