PyPy 7.3.2 triple release: Python 2.7, 3.6, and 3.7 | Hacker News Reader

PyPy 7.3.2 triple release: Python 2.7, 3.6, and 3.7

morepypy.blogspot.com

133 pointsogogmad53 comments

Love seeing the triple release. PyPy has come a long way on Python 3 since Py3k [1] and Fulcrum [2] and great to see a non-EOL Python 2 out in the wild.

[1]: https://morepypy.blogspot.com/2012/01/py3k-and-numpy-first-s...

[2]: https://morepypy.blogspot.com/2014/06/pypy3-231-fulcrum.html

Long live Python2 :)

    docker run pypy:2.7-7.3.2 python -c "print 'no quotes, integer division %s / %s => %s' % (5, 3, 5 / 3)"
    no quotes, integer division 5 / 3 => 1

Edit: i mean no parenthesis

Why do they hang on to Python2.7? I suppose it doesn't have so much maintenance cost for them.

I have to say that their official benchmarks point out a performance comparison to python 2.7, and at this time that sounds irrelevant - comparison to Python 3 would be relevant.

The main reason to move to py3 is the libraries, but they're not using any libraries.

It must rather be about what their users need - their users use libraries for sure. But maybe their users are still on Py 2.7.

> Why do they hang on to Python2.7?

https://doc.pypy.org/en/latest/faq.html#how-long-will-pypy-s...

PyPy is written in RPython, which is a subset of Python 2 without most of the dynamic features, so it can be compiled. Moving it to a subset of Python 3 would be a major effort, as they would have to rewrite both the PyPy code and the RPython compiler.

And RPython, being a subset of Python, can also be interpreted, so they probably want to keep the Python 2 interpreter so they can interpret it.

Thanks, that's understandable.

I think that they should switch to displaying and talking about Py 3 comparison benchmarks to give the impression that they are a non-legacy project.

Given how limited RPython is, are you sure RPython is not Python 2/3 compatible as is? What Python 2 only features it uses?

The faq that you've linked says that RPython will continue to run on Python 2 as long as Pypy exists, it says nothing about running RPython on Python 3 interpreter.

Cool now do UnicodeDecodeError :)

And print a traceback of a re-raised exception :)

PyPy is awesome. I wrote some scapy packet-processing code, got a 10x speedup from PyPy

+1 scapy really helped me before and I'm glad pypy is making packet dumps better

To predict COVID treatment side effects, we built a whole-transcriptome Trie with Scylla and PyPy and it sped up a bajillion times over raw python. Good times

Did you benchmark PyPy against numba?

Doesn't numpy require changing the code? When it works, my understanding is that pypy is a drop-in replacement.

It is, I ran python-neat with it. Almost doubled the speed.

OP said "numba", not "numpy" - numba is a JIT that's compatible with CPython!

https://numba.pydata.org/

Ah, I'd assumed a typo; thanks! However, I can't tell if/how much my concerns hold:

> You don't need to replace the Python interpreter, run a separate compilation step, or even have a C/C++ compiler installed. Just apply one of the Numba decorators to your Python function, and Numba does the rest.

(Emphasis mine)

Also, I can't tell whether https://numba.readthedocs.io/en/stable/user/5minguide.html#w... implies that some code won't work, or won't get faster?

Yes, some code won't work -- numba has a nice compiler that will show you errors if it cannot infer the type of even a single variable at compile time (which usually happens the first time you call your function at run-time).

The argument of "some code won't get faster" is null, since you typically only want to use `@njit`, which ensures that you're in `nopython` mode.

I guess that's a double edged sword, in that when it says `nopython`, it really does mean no python.

This means you can only use features from the python interpreter that the numba team has re-implemented in LLVM IR.

---

IIRC `@njit` does involve an overhead in `lowering` the types from python -> LLVM when the first njit function in the call graph is invoked, but not after that.

All this means that if you use `for-loops` in nopython mode, they are guaranteed to run faster, at least in my experience.

Who said anything about numpy?

Ah, I'd assumed that was a typo

Great progress, one day this will be the main reference implementation.

Is there an up to date comparison of PyPy and CPython with areas where the former must catch up?

Yes,https://speed.pypy.org/

coming from you, I might just have faith that it might :-)

Is it still true that PyPy is both faster and slower than CPython - faster when restricted to pure Python but slower than CPython using C extensions?

I've never heard of that.

What I have heard, though, is that string operations are more expensive in pypy than cpython.

Regardless, performance gains and regressions are unlikely to generalize; what's true for someone else's project may not be true for yours. If you're concerned about performance, I recommend benchmarking your own codebase.

The PyPy website suggests this is the case.

https://www.pypy.org/features.html

> Our main executable comes with a Just-in-Time compiler. It is really fast in running most benchmarks—including very large and complicated Python applications, not just 10-liners.

> There are two cases that you should be aware where PyPy will not be able to speed up your code:

> Short-running processes: if it doesn't run for at least a few seconds, then the JIT compiler won't have enough time to warm up.

> If all the time is spent in run-time libraries (i.e. in C functions), and not actually running Python code, the JIT compiler will not help.

Does it? It looks to me like it indicates performance will be the same when calling c functions.

It depends on how the C functions are called. CPython extension modules either don't work or have high overhead.

CFFI-based modules should work without a big penalty, but these are generally slower than extension modules.

It's true. Source: I benchmarked some PyGame pixel pushing. HPy might alleviate that in the future, but C extensions need to switch to HPy for that.

It depends on the code. You don't get speed for free by using C extensions with CPython. You have to write code specifically for that extension. To get significant speed benefits from numpy, for example, you need specific knowledge of numpy and the code produced will be completely different from regular Python. Pypy, on the other hand, is essentially a free speed boost when you have standard python code.

This innocent (albeit convoluted) piece of code used to segfault on pypy 7.3.1 . It is now fixed in this release! It was a bug in their JIT.

  def main(n):
    # Note: exact value of n and prints are significant, don't change
    a = [0] * n
    for _ in range(n):
        pos = -1
        for i in range(n):
            if i > 0:
                print(a[i - 1], end="")
                pos = i
        print(a[pos], _, n)
  main(191)

I really wish some python company invested money into making python 3.8+ compatible pypy, and running more tests on libraries for compatibility fixes. We could all use faster code.

Agreed, otherwise there is always Julia or a Common Lisp to choose from, and they can easily have bindings for the same C++ and Fortran libraries.

Hm, why do i think it is funny you recommend Julia and CL in a thread about PyPy, while preferring Java to Clojure, Kotlin and Scala?

Because 30+ years of experience have taught me to only use platform languages for production code.

Clojure, Kotlin and Scala are guests with temporary permit until the landlord acquires all features that matter to the masses, then they will join Beanshell and friends.

Julia and Common Lisp are the landlords of their own stacks.

As for the feature of the language themselves, I always considered myself polyglot, using the platform languages for each kind of deployment scenario.

And typescript? Is it better to stay away also?

I only use Typescript where it is the platform, e.g. Angular.

With other Web frontend frameworks, I use straight JavaScript, and direct HTML tags without any build tool.

Also if I am responsible for the whole stack, I use the JavaScript minifiers from the SSR frameworks like JEE and ASP.NET.

Is that something PyPy would even accept patches for, or they haven't done it because it doesn't interest them?

I would hate for someone to do all that work only to end up with a fork

Great news! Congrats (and thank you) to the team!

Is there any chance for Pypy to work with Numba, especially with HPy coming? Yes, they're both JIT compilers, but I feel like the kind of optimizations they do are complementary rather than redundant. I want fast business logic with fast algorithmic code in the hot parts :)

It became less relevant in the age of Julia language.

Do you have numbers to back this up?

See https://lh3.github.io/2020/05/17/fast-high-level-programming...

Also I should note that Julia performance greatly improved since. Next week there should be the 1.6.0 release, which will have even better numbers.

> I don’t see Julia a good replacement of Python. Julia has a long startup time. When you use a large package like Bio.jl, Julia may take 30 seconds to compile the code, longer than the actual running time of your scripts. You may not feel it is fast in practice. Actually in my benchmark, Julia is not really as fast as other languages, either. Probably my Julia implementations here will get most slaps. I have seen quite a few you-are-holding-the-phone-wrong type of responses from Julia supporters. Also importantly, the Julia developers do not value backward compatibility. There may be a python2-to-3 like transition in several years if they still hold their views by then. I wouldn’t take the risk.

I doubt that Julia 1.6.0 will be released next week. There were no rc (release candidates) not even an alpha or beta release.

I probably messed up something, but I meant alpha, yes.

It always depends on what applications you're looking for, but yes there's tons.

https://benchmarks.sciml.ai/html/MultiLanguage/wrapper_packa...

https://gist.github.com/ChrisRackauckas/cc6ac746e2dfd285c28e...

https://gist.github.com/ChrisRackauckas/4a4d526c15cc4170ce37...

are the ones from the top of my head.

That's a weird claim without specifying the context. Julia doesn't run python code, so that's not it. It doesn't use python's programmers... etc.

Is it "for new code which is maths/vector heavy and not relying on any existing environment, Julia is a valid competitor to pypy"? Or is there a better qualifier?

On the contrary, Julia might be the kick on the Python world to finally start discussing about having something like PyPy as standard.

Eh, there are still huge legacy codebaes around that benefit from pypy, not everything is worth a rewrite

Currently my greatest concern is the inability of Julia to provide stability even after they've reached 1.0. On paper 1.0 is a LTS release and still supported, but how many packages actually support it anymore? There seems to be no guidelines either, the official stance is that "we move on and drop LTS when the package developers move on."

>but how many packages actually support it anymore?

99%? You won't get the most recent versions because a lot of packages started requiring v1.3 when the new AD and multhreading mechanisms came out, but you can still boot up the LTS and run any standard analysis that existed back when 1.0 was around, which is something like 3000+ packages. Where is your FUD coming from?

Reply on news.ycombinator.com