Hacker News Reader

Comment by pjmlp | Hacker News Reader

It is a bit sad that PyPy great achievements are mostly ignored, and it took Microsoft and Facebook stepping in, Julia and Mojo arriving into the scene, AI GPU frameworks JITs, for finally CPython to take having a JIT seriously.

View on HN

I never understood why it didn't completely take over. It worked flawlessly for every use case I tried.

It always seemed to me like it a "free" way to make it faster. Why would it matter whether it was endorsed by the BDFL or PSF?

     If you want your code to run faster, you should probably just use PyPy.

Guido Van Rossum (once CPython's BDFL) said that.

At PyCon 2015:

* https://www.youtube.com/watch?v=2wDvzy6Hgxg&t=16m52s

pypy has never kept up with cPython features and releases. it was always well behind the curve and did not provide essential behaviors in a timely fashion. plus it can't run any C extensions or cython code (or has to run them with emulations that erase the speed benefits), which takes a big chunk out of the available ecosystem. Example: the situation with numpy: https://doc.pypy.org/en/latest/faq.html#what-about-numpy-num...

It keeps up pretty well, I think. But given what's in a version of Python is "what's implemented in CPython at that version", I don't see how it's possible to keep up very easily at all.

this is nonsense, cPython's development is very public and works on a multi-year schedule. It's not like Python 3.5.0 is released and that's when everyone knows what features it will have, each major version goes through at least a year of alphas and betas. Pypy spends a few years behind that and on top of it sometimes introduces new bugs that have to be reported and fixed. this is not anyone's fault, it's just for what pypy is doing to be successful, they would need a much larger maintenance ecosystem which they just dont have. The contributor ecosystem for cpython is literally ten times larger than that of pypy.

It doesn’t work with psycopg or many other libraries that use c extensions. That’s the main reason.

Only because they were compiled against cpython and not pypy right? If pypy had taken off/over, wouldn't it have been the other way around, where you need to do a special compile any time you want to use a C project with cpython rather than using the prebuilt pypy-compatible version?

Edit: things seem to have changed since I last looked into compiling a dependency for pypy. https://doc.pypy.org/en/latest/faq.html#do-c-extension-modul... now says they support c-extension modules without modifications

The package you mentioned, some postgres thing apparently, mentions they support pypy 3.9 and 3.10: https://www.psycopg.org/psycopg3/docs/basic/install.html

"Only": https://github.com/psycopg/psycopg/issues/154

By “some postgres thing”, you mean…pretty much the only way that one would connect to Postgres in Python!

Right, I just wasn't familiar with it (never heard this name before) and the only association I could think of for this abbreviated name was psyops and cops. Turns out it's actually for postgres which, of course, is a name I very well know. Didn't realize it was a well-known library!

There's also https://github.com/MagicStack/asyncpg

Oh, I'd thought some significant effort had gone into emulating the CPython C extension interface.

But maybe even with that, it's unable to support some extensions?

If someone suggested switching to a new C++ compiler with 99% of the features of the C++ standard with the selling point that it ran template metaprograms 100x faster, could you convince your company to switch? In the setting where python shines and performance matters, python is serving as the metalanguage executed at compile time for a strictly typed compiled language such as SQL, torch graph traces, jax, or z3

that only wins when it will lower costs significantly (percentage-wise) at my company :) . We don't care how much more green it would be, how fast it is, our main thing is always "how much does it cost" with everything converted to $$ (dev time, server cost, etc) and it can't change uptime, we would lose customers quickly without uptime, we have a fickle bunch.

I had always assumed it was because they had exposed far too many internals over the years, making a JIT that was backwards compatible with the majority of existing code, an extremely difficult undertaking.

I assumed it was because they explicitly did not have performance as a goal, probably because performance is so bad they wanted to have a good answer and "performance is a non-goal" is the best they could do. Psychologically that is much easier to digest than "yes it's awful, sorry".

I don't think backwards compatibility can really be the reason. Python does not care that much about backwards compatibility, as seen from the extensive list of features they remove in every release:

https://docs.python.org/3/whatsnew/3.12.html#removed

I’m confused now. What’s the point of PyPy if performance isn’t a goal?

It’s a completely unrelated project

Oh they’re referencing the fork and not base PyPy, which is still about performance?

The CPython project didn’t have performance as a goal.

PyPy is a different, non-PSF implementation of Python

Oh okay. I completely misunderstood the comment. Yes, CPython isn’t designed specifically for performance. I love how legible the code is.

The fact that PyPy has a JIT is kind of a side-effect. PyPy's goal is not to make a JIT compiler for Python. To be honest, the goals are broad and ambiguous, but it's something along the lines of "refining the core Python language, a subset of Python, and then see if it can be made to run Python in various other ways".

I think, that the first goal of PyPy was to compile Python to C or something like that. JIT came later.

PyPy was a successor to Psyco, by the same author. JIT was always a core part of the project.

You may want to read this: https://www.pypy.org/posts/2018/09/the-first-15-years-of-pyp... and further down. You'll see that JIT wasn't always a core part of the project.

You just for some reason decided to use the fact that the project was renamed at some point to fight a stupid battle on the Internetes.

“One of the not so secret goals of Armin Rigo, one of the PyPy founders, was to use PyPy together with some advanced partial evaluation magic sauce to somehow automatically generate a JIT compiler from the interpreter. The goal was something like, "you write your interpreter in RPython, add a few annotations and then we give you a JIT for free for the language that that interpreter implements."”

And?

Are you sure you understand the comment you are replying to? Your replies makes less and less sense. They just look like an attempt to pick a stupid fight.

If this needs repeating: I didn't deny that the project has a JIT compiler. I denied it was it's main goal. What you quoted confirms what I claim, but you seem to be misunderstanding this quote too?

I actually think you're the one misunderstanding the quote:

"One of the not so secret goals of Armin Rigo, one of the PyPy founders, was to use PyPy together with some advanced partial evaluation magic sauce to somehow automatically generate a JIT compiler from the interpreter."

He is saying one of the founder's original goals was to automatically generate a JIT. It may not be the "main" goal but it is certainly one of the original goals per this quote.

It always seemed to me like they didn't have a very good relationship personal with Guido for some reason.

Guido actually said

     If you want your code to run faster, you should probably just use PyPy.

I don't think these statements are in conflict.

I don't think I said the opposite.

I don't see any reason for Python to want to have JIT.

There are already languages that are more suited for that kind of model, that have a long history of incremental improvement of JIT compiler design and implementation. Why is Python trying to compete in the field where it's almost certainly guaranteed to lose, and is almost certainly guaranteed to win nothing?

Having an interpreter is great for some things. Especially, if you want to have bindings to third-party libraries, which is something Python does a lot.

----

My interpretation of the events is very different from yours. I believe that Python became swarmed by people who really, really want Java, but they feel that Java isn't cool for some superficial reason. And guided by the fear of crowd opinion, they won't just use Java and be content with it. Instead, they slowly make what used to be cool into a very bad clone of Java.

And the reason why the big names you mentioned have anything to do with JIT development is because they want to ride this wave of insecure and self-doubting developers to score popularity points, and, eventually, to control the popular technology. Esp. in the case of Microsoft, it's a multi-prong offensive. They've installed their people in Python Foundation, various SIGs related to Python. They've made CPython builds and CI run on their infrastructure, preventing it from choosing free tools for their development process... Microsoft has a bad rep with people who used to like Python, and that's why they don't rebrand it into MS-Python, but really, if they wanted to, they could certainly do it.

Because that is how we end up with AI tooling, and Web sites written in Python, or desktop applications in Electron.

No matter how things are supposed to be, that is not how bootcamp, self called "engineers", do their programming.

Also as proven by other ecosystems, with JIT, AOT, REPLs, in the box, having an interpreter like CPython isn't a requirement for anything.

As for your MS jab, it was called IronPython.

Reply on news.ycombinator.com