PEP 3146: Merge Unladen Swallow into CPython | Hacker News Reader

PEP 3146: Merge Unladen Swallow into CPython

svn.python.org

117 pointsdchest30 comments

I don't really understand the politics surrounding Python, but I don't understand why they believe this could be "blessed" by inclusion into the repository.

Performance goals have not been met so far; it increases memory usage; PyPy is an alternative with more community support (if only because the typical Python programmer knows more Python than C++); and it makes the code much more complex, introducing lots of hard-to-debug problems, at a time when the developers already have trouble getting people to migrate to 3.x.

The 3.x branch does not need a reputation for instability.

(None of this is to say Unladen Swallow is a bad idea, or should be killed; but it should prove its worth first.)

I agree. Why are they in such a rush to get into trunk? The work they're doing is great, but their sense of entitlement about is is a bit misplaced.

This is speculation, but: Google is paying some people actual money to work on Unladen Swallow. People who have made bold claims about the performance improvements, which, apparently, were not entirely justified.

I imagine that it would be politically desirable to show success by getting officially blessed, and it would also mean that the project wouldn't just stagnate if the people now working on it were told to do something else.

More optimistically, they acknowledge that more coders would be welcome, and hope that it's easier to find those after being included in the main repository. It would also give them an edge over PyPy, their sort-of-competitor.

> but I don't understand why they believe this could be "blessed" by inclusion into the repository.

Google is, of course, a big company with lots of very smart people, and they rely on Python (and probably want to rely on it more in the future). Given that they employ Guido and also the primary developers of Unladen, the chances are very good that they will continue to vigorously improve upon it until their performance goals are met.

As far as introducing C++ goes, it's going to stay roped off in LLVM land (see [the PEP](http://python.org/dev/peps/pep-3146/) for details).

As far as community support, LLVM has plenty of that.

Hmm, yes, I understand why the Unladen Swallow developers, and to a certain extent Google, would want to do that. It just seems to me that the community would benefit more by waiting a little.

By the developers' own admission, LLVM's JIT infrastructure is immature and integrating it with Python uncovered lots of issues. Sure, clang has pretty good community support, but that doesn't mean that the Python integration - or even the JIT stuff - is equally well-supported. And the Python community doesn't contain many LLVM experts, I'd wager.

Very simple: this is a PEP to improve the state of "Cpython" (this is the main python implementation right now, and has been since the invention of python).

Pypy, on the other hand, is a new implementation of Python (just like ironpython or jython), and it could eventually replace cpython in the future if it proves to be succesfull (although it would be a long way off).

In few words: Python, the language, have many implementations. Cpython is the current reference one. Unladen Swallow aims to improve it, not to replace it. Actually, even the US developers acknowledge that Pypy (or any other succesfull implementation that may come up) is the future, because it doesn't have to bear the burden imposed by the c platform.

A pretty, rendered version of the markup is available here: http://www.python.org/dev/peps/pep-3146/

I don't think it's enough to just add JIT. For a runtime system the garbage collector is important, it's important to do optimizations such as V8's hidden classes, function inlining, to remove the GIL etc. This is probably the reason why their perfomance gain is so small - - they have added JIT, while applying only a few smart optimizations.

I don't think Unladen Swallow should be merged into CPython until the benefits are proven. Currently it will just add an extra layer of complexity, worsen Python's greedy management of memory and add little benefits in perfomance.

I want a pony from Guido too

Surprisingly on-topic :-) For the curious, the reference is within this article: "We seek the following from the BDFL:

Approval for the overall concept of adding a just-in-time compiler to CPython, following the design laid out below.

Permission to continue working on the just-in-time compiler in the CPython source tree.

Permission to eventually merge the just-in-time compiler into the py3k branch once all blocking issues have been addressed.

A pony."

Guido doesn't want to share the pony: http://farm4.static.flickr.com/3569/3401670115_54e1d32200.jp...

It obviously helps if you're paying his salary.

It's unfortunate that they never hit their original 5x performance boost. Hopefully acceptance into CPython will help.

The performance increase is still notable though - I would like to see that static LLVM link changed; I like python for its size, speed, and expressiveness.

So is the goal to gain resources by becoming part of CPython? It seems like the small performance increases they got are completely overshadowed by the additional memory requirements. I don't pretend to understand the mainline Python development cycle, but this surely doesn't seem mature enough to justify declaring it as "the way forward".

Ya I noticed that too. I'd love to have some performance benefits from JIT too but not at those costs. 2.43x - 2.76x increase in memory to Django[1]. Ugh. That's a large enough change that I'd have to upgrade hardware. Come on now.

[1] http://www.python.org/dev/peps/pep-3146/#memory-usage

From TFA "We view reducing memory usage as a blocking issue for final merger into the py3k branch. "

So, you'd accept unladen as the official JIT based on the vague promise of improving its glaring issues sometime in the future? Does that give it the right to be in a "blessed" repo?

I propose my own python implementation for acceptance into the source tree. Right now, it's a single empty file (but I've got lots of ideas!). I'll even consider it a blocking issue if it achieves less than 10x speedup using half as much memory.

Down further they say something about how they'll ask the community what the acceptable memory usage increase will be. In other words, they might reduce it by say 15%, but not anywhere near the JIT-less levels, and call it a day. I suppose you always have the option to compile python as --without-jit...

Extra memory means a longer start-up time (for small apps) ala JVM, and makes hosting much more expensive. Lots of web hosts are memory / IO bound, not CPU bound. And you are sacrificing memcache size for python speed if you increase python's footprint - not always a winner.

Well, they've still got at least a year or so to work on it, since it's unlikely Django will be running on Python 3 before then...

> It's unfortunate that they never hit their original 5x performance boost.

Give 'em a chance. They're only just out of the gate. :)

See http://www.python.org/dev/peps/pep-3146/#performance-retrosp... for a discussion of why.

I think that a JIT compiler will eventually be added to python, so the sooner this happens the better. The new generation of javascript engines (e.g. V8) have shown us how fast dynamic code can be; there is no reason python can't achieve this either. I also think the swallow team has gone about this in the right way: presenting what they've done, and asking for what needs to happen before it can become mainline. They obviously value this merge, so kudos to google.

On the other hand, performance so far is quite underwhelming. Certainly, as a PEP, this is by no means intended to be merged now, but to provide a roadmap for the future. Here's to hoping they can remove their remaining roadblocks.

Until they get more performance out of it, it's way too marginal for the memory used. If they can get it 2x the speed of CPython I'm all for it.

Of course one day pypy will emerge too, so unladen swallow should hurry up!

(TBH I'd prefer pypy to become the canonical python in the end)

Sorry but I need to know this: What does "Pony" mean in a nerdy-geek humor context?

"... and a pony" is just an admission that you've already asked a lot and it might not happen. A pony is the kind of thing a child requests on a whim, who is then crushed because they don't understand how ridiculously impractical it would be.

Thanks. That's a very good answer :-)

If it was at terminal velocity it should be considered, but it's yet to get off the ground. Until then, psyco is a great JIT compiler that really works.

...African or European?

Reply on news.ycombinator.com