PyPy 5.0 Released | Hacker News Reader

PyPy 5.0 Released

morepypy.blogspot.com

284 pointsmattip62 comments

Because everyone asked, I gonna clear a few things about PyPy3 support, please keep the comments civil.

* We are not against working on Python 3 - it just happens that there is a lot of interest these days in things like numerics, warmup improvements and C extensions that we want to focus on.

* We essentially exhausted Py3k pot. I personally think it delivered what it promised, despite being short on the funding goals. It's crazy what level of expectations people have with crowdfunding - it's really difficult to find someone to deliver a big, multi-year project for 60k, even outside the states.

* We're closely watching py3k adoption - since we're always a few releases behind, we'll probably do a 3.5 after CPython 3.6 is out, but it all depends on good will of volunteers, who I have no control over.

* Money can easily change focus, but it would need to be a significant enough amount to actually commit to delivering a fast and compliant PyPy 3.5, not 5 or 10k

I hope this clear some things up, those opinions are my own and not necesarilly represent everybody in the pypy project

EDIT: there is just over 8k USD left in the py3k pot. At $60 USD/h (official SFC rate) it's 146h. That's not enough to even fix the inefficiencies in the current version. We hope to use it to get to version 3.3

Is the plan to go straight to 3.5 after 3.3?

Why not start a fund for 3.5 (or whatever is next)?

What exactly are the inefficiencies in PyPy3?

Thank you so much for taking the time to comment on this. I have very little vested in (any) python (distribution) - but would love to see "real" python3-support in pypy.

As the sibling comment mentions: Maybe now is the time to try for another funding run for py3 support? A lot of frameworks and libraries have been ported, and a lot of new code is being written for py3. Perhaps there'd also be real interest from people that now look at pypy as "just" "performance enhancement for legacy py2" to see py3 on the pypy platform?

In my personal opinion the speed gains of pypy aren't really all that interesting (other than perhaps, in the sense that it enables more code in python rather than c etc). But rpython+pypy as a framework for language development in general, and as a host for python in particular (along with the work on cffi) are really great aspects of pypy.

And while anyone can get rolling with rpython and implement their own toy prolog/brainfuck/whatever -- other benefits from running python on pypy will remain out of reach for those that for various reasons are targeting py3.

Then, maybe we could do another funding run to get mercurial ported to py3, later ;-)

At any rate, I love the work that is being done on pypy, and I certainly think it doesn't make sense to devout resources to py3 "just because new"; I believe free software development works best when driven by stakeholders. But I think the fact that "pypy doesn't really work for py3 also drives some people away from pypy that might otherwise contribute in various ways to help push py3 support further.

If I want to help on PyPy3, is there any sort of mentored bugs or good first bug (similar to Mozilla's) that a beginner could get started on? I would love to contribute, but never really got to do so since I don't know where to start.

Come by the PyPy IRC channel and ask

I think its a real shame that pypy3 is not updated with the new releases of pypy. It is still on 2.4.0.

I understand that the major sponsors of PyPy are interested in the python 2.7, but not updating it for 1.5 years seems like they have abandoned it.

I know it does sound a bit bad, but Python 3 is only now catching up. The old PyPy3 that we have is a bit terrible, because it's both old (3.2 only) and slow. Serious push would be necessary in order to get it up to 3.5 and fix the performance problems that we're just not willing to do right now. Once the commercial world catches up to python 3, we will probably too, but people using PyPy are not people who are getting excited to move to python 3, for the most part

> I know it does sound a bit bad, but Python 3 is only now catching up.

Catching up as in PyPy's support for it or as in adoption of the language?

catching up in adoption

Given that Python 3 is nearing a decade old, it is always astonishing that people are still on 2.

Well, I'm sorry that it does not look like you would want it to look like, but as a pypy dev this is not my battle - we're there to provide a fast version of the mainstream python and mainstream for now means 2.7. If it changes, we'll adapt

Isn't that change coming now? Most of the most popular libraries have python 3 support now, fedora will soon move to python 3 as default, and there are soon more developer support behind 3 than 2.[0]

[0] https://blogs.msdn.microsoft.com/pythonengineering/2016/03/0...

If you're not part of the solution, you're part of the precipitate.

It really is your battle. Major Python projects not supporting Python 3 in 2016 is shameful and holds back the broader adoption of Python 3.

You can't blame someone you aren't paying for not implementing a feature you want.

They are being paid. $63K, including by me. It's their most funded donation goal.

https://sr.ht/qXYC.png

> "They are being paid. $63K, including by me. It's their most funded donation goal."

From a later comment by a dev...

"We spent the py3k budget (almost all of it) on delivering pypy 3.2."

Even the link you shared shows the pot is down to $8222. Hopefully that will increase, but I don't know what would be required to get support for later Python versions.

We spent the py3k budget (almost all of it) on delivering pypy 3.2. If someone is very unhappy that it was either too expensive or we didn't deliver enough for the money we received, I can apologize.

Would be nice if the numbers on the home page were more representative of the current situation, then.

They say we have 8800 left, right?

Well, yes, but it also has this big meter that looks 60% full and gives the viewer the impression that monetary support for this feature is well underway and likely reduces the amount of donations you'll receive for it.

There isn't a huge amount of win for going to Python 3, but there's plenty of code to change. It's not like Python 3 is 10x faster or has multi line lambdas or something.

Plus Python 2 has been updated many times in the past decade so it's not a maintainance based motivation.

Personally I code against Python 2. There just hasn't been enough motivation to learn 3 and make sure all my third party libraries work with it.

I think the last sentence of you really also is in part its own cause. People excited about Python 3, are not using PyPy since it seems like it is not maintained at all.

At least adding the same improvements that were added to the 2.7 interpreter would put a bit of confidence in that. Most libraries also support up until CPython3.3, which as far as I know is mostly caused by the fact that the "u" string prefix is allowed again. So just adding that and keeping the core up to date with the PyPy2.7 releases would make it usable.

This could very well inspire other devs to add support for the other features that they are missing from later releases CPython3 releases.

No, it runs deeper than that. You're mostly reaching out for pypy in case you're running into the wall with performance, one way or another. In that situation the LAST thing you want to do is to move to another language that does not give you answers to that performance problems. People are moving to go, yes, but not to python 3 in that case.

But then there are the people who've only started their codebases within the last five years, and so have been Python 3 all along.

Right. And those people are slowly getting to the problems that pypy potentially solves. We are focusing so far on widening the base of what we do, but if there is enough interest, we'll work on pypy3.

Python3 is going to be the default Python in Ubuntu next month. Do you anticipate that having any effect on your perceived user base?

FWIW, almost all of the applications I've written in Python over the past 4 years have been Py3. I'd like to use PyPy on them.

He doesn't care because that isn't where the money is coming from

It doesn't sound like just money. It sounded like he was saying that he doesn't care because there is comparatively little (historic and current) demand for PYPY3.

From a certain perspective, money == demand.

I don't blame him, pypy has always had a money problem (hence crowdfunding etc etc).

There's a similiar project to PyPy called ZipPy which is based on Graal/Truffle, except that it targets Python 3. It isn't complete, but it's probably easier to flesh out than porting PyPy to 3 itself. Check it out:

https://bitbucket.org/ssllab/zippy

pypy3 has always supported the u'' prefix despite it otherwise being limited to 3.2 support.

"Once the commercial world catches up to python 3..."

fwiw, lack of PyPy support is one of the most common reasons I hear for people being leery of Python 3. Of course the people using PyPy aren't the people excited to move to Python 3, if they're effectively mutually exclusive.

"If you build it, they will come" might apply here. Just to play devils advocate.

Feel free to make a donation to that project. You can donate specifically to the py3k pypy project.

Why, when it's been made clear here that they're not interested in working on it?

That's not completely true. It would be trivial to find people to work on Python 3 support if they were paid. It's just finding the volunteers that's hard.

They say pretty clearly that they're not working on it because the donations for that goal are exhausted and that the money they have was donated to be focused on other parts; that they would be interested to make Python 3 support better if the donations were there.

Myself as an hpc user of python (this pretty much sounds like an oxymoron at this point) in deep learning, I would be much more interested in having solid numpy and c extensions support (tensorflow / theano maybe?) than python 3, and I feel that many people feel the same way. The only reason to use PyPy is performance; making it very compatible with the other main source of performance gains in python (c/c++/fortran extensions) feels like a strong priority/ a no brainer

I'd like to use PyPy but all of my new projects are Python3.

Well, great job, team!

I get a free speedup for my non-numpy/scipy projects and that's flipping awesome. The 2.7 and 3.2 support is just fine for my needs. Your focus on the C API emulation seems totally appropriate to me.

If numpy and friends worked well on pypy, IMO there'd be little reason left to use CPython.

Pypy is really cool, and I hope it becomes the default interpreter down the road. However, little py3 support makes me edge away from it for now. Hopefully, all pythons will merge in the near future

The versions cant really merge, there's runtime differences and you don't want to bloat with two of everything

I think the GP could be referring to PyPy becoming a replacement for CPython (or incorporated into CPython) as the canonical Python interpreter. It could theoretically happen after for a major release, and it could work out to be beneficial, though I think it's quite some time away if it happens at all.

It will never happen, the pypy codebase does not lend itself to the sort of quick evolution and extensibility that CPython delivers. Remember, the original goal of pypy was not "speed", that was a side effect.

What was the original goal of PyPy, and have they delivered on it?

The original goal, if there was one, was Python implementation implemented in Python. This goal is achieved.

The original goal was to implement a runtime that could run multiple languages, similar to Perl's Parrot project, but targeting Python and written in what was basically a subset of python. It so happened that the resulting implementation ended up being very fast in some situations, the commercial world adopted it, and the project basically pivoted to being "the fast python runtime".

Good to see PyPy progressing.

I mainly use Scikit Learn, theano, numpy and pandas; is PyPy able to work with the above, and likely to give any speedups at this stage?

Numpy support is improving but incomplete, see http://morepypy.blogspot.com/2015/02/linalg-support-in-pypyn... - and the performance is not going to be any better than stock numpy with CPython. If your code spends all its time in C extensions, PyPy doesn't buy you anything since its API compatibility layer is slow and incomplete. You should probably look into Julia.

Why major version change, why 5.0 after 4.0.1?

The C-FFI is significantly changed. Which means lxml can now be supported.

We (PyPy release manager here) adopted a release policy where we update the major version about every three months. Bug fixes, if needed, will be 5.0.1, and 5.1 should be released in a month or so

Why not use a calendar based scheme then? This differs so much from CPython versioning it's confusing.

Oh no, not Chrome versioning again... And this time not even on an app. :(

What you're saying is that major number will change with time, not based on some groundbreaking updates. How am I supposed to know if I should be careful when upgrading from version 12 to version 14? Were there massive changes in between? All I know now is that half a year has gone by since last time I updated. Why would I care about that? Yeah, sure, I can read changelog for _every_ _single_ _library_ _and_ _tool_... Really? </rant>

EDIT: other than that, congrats on a new release. ;)

http://semver.org/

Summary

Given a version number MAJOR.MINOR.PATCH, increment the:

MAJOR version when you make incompatible API changes, MINOR version when you add functionality in a backwards-compatible manner, and PATCH version when you make backwards-compatible bug fixes.

That's what amenod was referring to. The current versioning model is different than semantic versioning model that everyone is used to.

Personally I really hate how everyone (especially web browsers) blindly copies what Google comes up with, without putting much thought into it, whether it is UI changes, versioning or behavior changes.

I don't know what's the issue. PyPy shouldn't have breaking changes. They're targeting Python 2.7.x which is a frozen specification.

In the unlikely case that a new release causes code that worked in a previous release to fail, it's simply exposing a bug in either the new or old release.

What version of CPython does this match? The announcement doesn't say.

It's 2.7. They had an announcement about the new versioning last fall:

http://morepypy.blogspot.com/2015/10/pypy-400-released-jit-w...

Also see fijal (one of the core pypy devs) commenting in this thread about the pypy devs not putting much work into supporting Python 3 right now.

Wow, a ton of crying here on Python3 support. Short sweet answer: patches welcome. Get to work. Or, donate your money to the authors so they'll do it for you.

no numpy, not interested ... will check again next version.

How is numpy doing? How about the sandbox?

Reply on news.ycombinator.com