Hacker News Reader

Comment by wyldfire | Hacker News Reader

First we get awesome CPy C-extension emulation and now this? Thanks, team!

Completely serious: when can PyPy obsolete CPython? Why don't more CPython core devs work on PyPy? Besides perhaps some remaining gap for C-extensions support, what else is missing?

View on HN

C-Extension support is still not complete and comes at a massive overhead. That's better than no support at all but the performance impact might be a problem in some cases.

PyPy is also still several versions behind implementing the latest Python 3.

So while PyPy is definitely a great project, groundbreaking in many ways and a very useful for a lot of applications, it's not a replacement for all applications, for now at least.

Also while the actual PyPy Python interpreter is reasonably simple overall the project is incredibly complex. It at least appears as if there are parts of the PyPy project not even all PyPy developers understand. Making PyPy the reference implementation is probably questionable for that reason alone.

Perhaps that not everyone is writing long-running services using Python?

PyPy 5.3.0 is twice as slow as Python 3.5.0 for some work I regularly deal with (1.1s for PyPy, 0.6s for CPython). Another project it is ~40% slower (7.7s vs 5.6s).

Additionally, PyPy just released an alpha containing compatibility with Python 3.3 (which was released four years ago). There are some nice additions to the language in 3.4 and 3.5.

> PyPy 5.3.0 is twice as slow as Python 3.5.0 for some work I regularly deal with

Is this because you are using ctypes or a C extension written against the CPython API? If not, have you filed a bug report?

I do write some code using FFI, but I have a cffi layer in there for PyPy. I do not interact with the CPython API.

Currently I don't have an interest in taking time to try and make PyPy less slow for my use case, for many reasons that I'm not going to get into here. (But I will note I'm primarily writing code that focuses on being easy to distribute. Most users aren't going to have PyPy installed anyway.)

Anyway, the main thrust of my post is just that Python isn't a monoculture in which PyPy is always a better solution.

Thanks for providing context.

I have some data generation scripts in a bash pipeline where pypy is significantly slower due to startup overhead. When you run anything in a loop ten thousand times, the startup time matters a lot for some people.

That said, PyPy is awesome. :-)

Why not write the loop in Python itself?

I remember hearing Guido comment that CPython is much easier to hack on compared to PyPy. I think this reason alone is why other Python implementations will remain separate from the 'reference' CPython.

The majority of PyPy development is done in RPython which probably makes hacking on the runtime available to something like 100x as many developers. The only limiting thing is knowledge and exposure, but mostly exposure.

Sorry, CPython is much easier to hack on than PyPy. To start with, for every change you do recompilation takes 4GB of memory and half an hour to complete.

This is a rather slow feedback cycle.

Secondly, while the syntax of RPython may be clean the necessary incantations to actually achieve something aren't at all obvious.

I don't know much about pypy, but do you need to compile it each time? Just use pypy to live interpret the interpreter.

Lots of RPython development can be done in the repl, code can be debugged with pdb. Not nearly as difficult as it is often made out to be.

Platforms I guess. Plain C code is much more portable than JIT and PyPy is reinventing JIT implementation as opposed to using something like LLVM, so they're much more limited in terms of supported target platforms.

Oh, my mistake -- I had assumed that it did use LLVM. Good point about portability, CPython has tons of supported targets and PyPy just a handful. Also CPython is really easy to build IMO.

LLVM isn't as widely portable as generic C, either. ARM and x86 are well supported in LLVM, but CPython runs fine on MIPS, Sparc, PPC, etc (via GCC or other C compilers).

You can compile PyPy without its JIT.

Yes, although without JIT it's slower than CPython. So, what's the point?

If you are happy using CPython, you can't be terribly concerned with performance. The point is, pypy will run on more platforms than its JIT targets.

Sure. But if you're already using CPython (as most users are), and you know Pypy will be 3x slower on your platform, why switch?

You have no reason to. We're talking about a hypothetical future where PyPy has become the reference implementation.

In addition to the other stuff, I believe CPython also fits in much less memory, since it uses reference counting for its first layer of garbage collection, and doesn't have to store JIT output anywhere.

Reply on news.ycombinator.com