Hacker News Reader

Comment by mikece | Hacker News Reader

Is there any chance that a project like RustPython could supplant CPython?

> For your big question about a CPython replacement I think that this is our current goal. I do not think people will be using RustPython in their production any time soon but I do think we can be competitive in WASM. Having said that I do not think we should be 100% compatible with CPython. For example we have chosen to implement threading without the GIL. Being compatible with CPython is very helpful as we can use the documentation and tests of CPython.

https://github.com/RustPython/RustPython/issues/1940#issueco...

Not having a GIL makes this very interesting for all production uses. I hope this works out.

It also rules out using most C extensions which rely on it. These libraries are most of the draw of the Python ecosystem

TLDR: No

The hardest part of supplanting CPython is the fact that the FFI is already validated on a huge amount of implementations. I believe Cinder and Pyston work out of the box but they are on older versions of python but Pyston wants to be merged back into CPython and so does Cinder (or at least what is relevant). On the other hand JIT in CPython can be achieved by other python packages. Pyston has also extracted the JIT and can be added to 3.7 - 3.10 by installing it. See https://pypi.org/project/pyston/

Pypy is a separate reference implementations that has tried to achieve something different but hasn't supplanted python for years. It has had a JIT and the ability to change garbage collectors but the system stuffers when interacting with C FFIs

Pyodide is inthe browser so its something completely different but has no support for CFFIs.

IronPython (.net), Jython are inactive. Also GIL removal from CPython has been attempted multiple times but now that CPython is being funded by Microsoft to become faster makes it more unlikely that CPython will be supplanted.

GraalVM also has a nascent Python runtime, which could be extremely performant.

But I think OS devs dont really want to put optimization work into GraalVM, as their hard worked could get sucked up and locked behind the Enterprise Edition.

Mainly resource exhaustion from developers makes the other runtimes fade into the background while resources being put into web deployment (pyodide) and microcontrollers (micropython and circuitpython) are a much better idea.

What's graalvm and where does it fit in with the larger Python ecosystem

Its a "universal" VM from Oracle that supports many languages and other bells and whistles like AOT compilation of interpreted languages: https://www.graalvm.org/

But its Java support is most prominant, and the its runtime is significantly faster than OpenJDK.

Which is why its so promising for Python. I'd argue that OpenJDK is way ahead of the Pyton runtime, and GraalVM is way ahead of OpenJDK.

But its also kinda iffy because:

- Python support isn't very good now

- Oracle is Oracle. Hence they have the better optimizations walled off behind a "Enteprise Edition" registration and license.

I wouldn't touch it with a ten foot pole because other than Oracle suing Google over API reimplementation they can eventually abandon the runtime.

Depends...

There are already other implementations. But CPython is the reference implementation. That's part of why we're only now seeing optimizations that reduce code clarity for reading. (i think, dont quote me on this)

No need to quote you, but GvR himself [0]. That's the Faster CPython initiative, which has been ongoing for about 2 or 3 years I think. 3.11 got some nice speedups from it, and 3.12 is on its way to get more. [1]

[0] https://github.com/faster-cpython/ideas/blob/main/FasterCPyt...

[1] https://github.com/faster-cpython/ideas

To solve what problem exactly?

To solve the problem that is C.

Any constructive comments or we’re /r/ProgrammerHumor now?

it wasn't suppose to be funny. C has specific problems. Rust is a solution to those specific problems.

I think the question is how those problems are applicable specifically to CPython's use of C.

I agree that it would be better, all else being equal, if CPython were implemented in a memory-safe language (though it would almost certainly need to use unsafe escape hatches in some places), but I think this'd be more viable if it were done by incrementally migrating the existing codebase with the involvement of the current maintainers, rather than as a third-party rewrite from scratch.

Then there is python written in python called pypy

If they support python packages exposing the C ABI and expose a C ABI for embedding, I can't see why not.

They're doing threading without the GIL, which means it's not compatible with existing C extensions.

CPython is also building a -no-gil compile-time option. If that gets traction, some of the big performance-oriented C extensions like Numpy etc would likely build a non-GIL extension version too.

Then, if RustPython's non-GIL implementation is compatible with CPython's, and their C ABI is compatible, those C extensions might work.

gil or no gil is mostly irrelevant for numpy, an extension that does most work with gil released.

Relying on the cpython gil and releasing the gil are two different thing:

  1. releasing the gil means multithreading is opt-in for a given code section in NumPy. Only very specific parts of the code need to be threadsafe.
  2.  not relying on a gil in cpython runtime means multithreading becomes opt out. Now all the code by default needs to be threadsafe, including the libs you depend on.

A lot of C/C++/Fortran scientific code is not thread safe, and the whole scientific python ecosystem depends heavily on those codebases.

I understand what you are saying, but I don't see how it's relevant? Numpy doesn't rely on the gil, which is why it releases it.

It releases the GIL only in very specific sections. Most of the Numpy C code runs under the gil.

A quick check on master shows only 10-20 calls using NPY_BEGIN_ALLOW_THREADS (which is an alias to Py_BEGIN_ALLOW_THREADS).

A lot of the NumPy code manipulates python runtime objects, and doing so without thread safety would likely break everywhere. A lot of efforts would be needed to gradually make large C extension thread safe.

Only if the CPython devs will it so.

With HPy yes

Reply on news.ycombinator.com