Python 3 Wall of Shame | Hacker News Reader

Python 3 Wall of Shame

python3wos.appspot.com

122 pointsiamelgringo78 comments

"Wall of Shame," sounds harsh. However I don't think it's entirely inappropriate. The premise is that it has been two years since py3k has been release and the community has little to show for it. One would think two whole years would be enough time to port a library. Especially if most of the fundamental challenges of doing so are matters of syntax (not in all cases I'm sure) and changing some names.

OTOH, maybe the community still refuses to switch? Maybe py3k is viewed as "that bad," that no one wants to bother with it?

FWIW, py3k is quite fast now and the language "enhancements" do make a significant difference. I actually quite like it now that my primary OS, Arch Linux, gave me the boot and made py3 the default interpreter. Many of my projects are still in py2.7, but I have been working on converting them. Sometimes I get lucky and all I need to do is run 2to3!

Guido should have stuck with his original plan - keep updating python 2.X till it's converged to python 3.

Deprecate print statements, warn if you don't import division from __future__, and so on. Maybe even leave a 2.X compatibility mode: `from __past__ import print_statement`.

I guess unicode is a sticky point though. I really don't know what can be done about except rant about how hard unicode is.

I am sure Guido realized one thing. Where would he get the resources to keep updating and do P3. I think he chose the right path. 5 year plan (half way there) to move everything to P3. That gives them a good time-frame to get things in P3 solid and stable (however you want to apply those terms).

"Maybe py3k is viewed as "that bad," that no one wants to bother with it?"

I don't think anyone's viewing it that way. A lot of people want to switch and many are, but there's the non-zero cost of porting and also the generally low demand. No one wants to spend 3 days of initial work on a port that no one currently wants, then tack on the continued maintenance of two branches (or a single code base that works with both and requires twice the testing).

Part of the reason we have the PSF Sprints funding is to solve that problem. A group of 6-7 developers in Cape Town is taking advantage of the funding in early March and plans to complete more of the py3k port of matplotlib. matplotlib is a great one to work on in terms of community impact since it usually ranks highly in any poll of 3.x blockers.

Actually, my LUG is also considering doing some sprints. I'm more inclined to join them after using py3 and wishing there was more support for it (and feeling bad about my own laziness in porting my own stuff). The sprints are a great idea and we're quite lucky to have such support!

The equivalent for Ruby 1.9 is here:

http://isitruby19.com

And there's a version that shows rails3 compatibility amongst other things here:

http://railsplugins.org

Let's admit it.

Once Django is Py3K-ready, all other projects will follow suit.

Is Django really that big a deal in the python world? I know it is a big deal in the python web world, but the web seems a pretty small part of the python world as a whole. If I where to ask all the python developers I knew to prioritize the projects they want ported to py3, I'd imagine the results being numpy/scipy/matplotlib first followed wxpython or pyqt second followed by PIL and a whole bunch a smaller libraries, with Django getting a "sure I guess" down at the bottom of the list.

I don't do webdev, so I fully agree with that :) FWIW, NumPy supports python 3 (for quite a few months now, IIRC), and I imagine that SciPy does too.

I would expect NLTK to have a fairly broad user base too, although possibly the community is less likely to be involved in the broader python landscape.

The thing with Python is it has very long tail of libraries, like NLTK, OpenCV, openopt, Mayavi etc. who's individual communities might not make up significant portion of the python community, but taken together is probably more significant than that of any single high profile project.

There is a poll about p3k compatibility on python.org: http://python.org/3kpoll

Django on top, followed by wxpython, scipy and mathplotlib.

Anyone else find it amusing that there aren't quite 3000 votes on the Python 3000 wishlist?

Package Number of Votes Django 760 wxPython 423 scipy 366 matplotlib 328 PIL 295 py2exe 243 Twisted 169 PyGTK 157 Pygame 117 web2py 91 ---- 2949

OK, probably just me.

If this is true, then there's a bit of a circular dependency Django is a pretty high-level project, though, with a lot of other dependencies of its own. I believe that the stated position of the Django core team is that there's no point in porting Django until some (all?) of the core dependencies are functional on Py3k. This, at least means wsgi and the database connectors, and possibly more.

So I hope that the rest of the Python community isn't sitting around waiting for Django to take the lead here.

Right, Django wouldn't do much for me without MySQL-python.

I would say, practical implementation of PEP-3333 is more important than Django.

Most Python frameworks are stuck because of WSGI vs Python 3 issue.

Funny to see zope.interface on that list. I remember back in 2005, when Zope 3 had just been launched, about how everyone who mattered in the Zope world would say things like "only stupid, retrograde people won't switch to Zope3". I guess the stupid people won, because 6 years on Zope is like the Cobol of Python web frameworks. I just hope the same thing won't happen to Python itself.

Let's just call Zope 2 the Common Lisp of Python frameworks... I guess it survives mostly because Plone needs it.

Zope 3 and Grok are pretty cool.

twisted and a few others rely on zope.interface too. I never saw the draw personally

I've been doing a ton of Python work lately. Frankly, in my experience, Python 3 is more of a pain to develop with, primarily due to two factors:

* Encoding specifications required.

* maps/reduces returning iterators objects instead of a list. Having to call list() on anything I map is just a pain.

I don't plan to switch to Python 3 until a compelling reason shows up. 2.7 works great for what I do.

The real story here is how Python 3 appears to be a largely unwanted improvement, IMO.

  maps/reduces returning iterators objects instead of a list. Having to call list() on anything I map is just a pain.

Why do you require it to be a list?

I like that map and reduce return iterators; it lets me map over a huge collection even if I just want the first few.

To me py3 has far too few "killer features" to switch. I just keep using 2.6 for my stuff because a) it does the job and b) it's just a major inconvenience without any benefits to port everything to 3.

This is a really good list for people who want to get involved in a Python project. I only wish it were bigger and it would contain smaller projects as well.

DecoratorTools shouldn't really be on that list though, since it doesn't make any sense for py3.

The list is horribly wrong and misleading. The author and others are working to rectify it. There's a lot of packages on there that simply don't belong there.

"Wall of Shame"? Really?

As someone who's project appears on this list (and in the WRONG color) all I can say is that I don't think anyone is trying to dis Python 3.x. Support will come when a critical mass of developers are using 3.x. I know it's kind of a chicken and egg problem but this seems to be saying that it's the package developer's fault and I don't really think that's fair or true.

I think it's not really a chicken and egg problem though.

From a user point of view, using Python 3 looks like giving up on a lot of libraries, which in turn means a lot more work to accomplish the same things. It's a big effort that does not benefit anyone. If switching to a new version of a language makes my life harder, then I will not, especially if working on a startup or a project which is time critical.

On the other hand, this is a work that the developers of the libraries will have to do anyway, sooner or later. Doing it now would benefit a lot of people that would like to use the latest version of Python and now simply cannot afford. So why do not do it? In this case the effort would benefit a lot of people at once, which is the purpose of a library in the first place.

"On the other hand, this is a work that the developers of the libraries will have to do anyway, sooner or later. "

But... will they "have to"? If a library author is waiting for 'critical adoption', they may never upgrade to Python 3 themselves, and have no need to upgrade the library, and the circle continues.

It's a chicken/egg problem in that developers are saying I'm not porting because no one is using Python 3.x and users are saying I'm not moving to Python 3.x because none of my packages are available.

I can't just move to Python 3.x and abandon 2.x. And I have not been able to find a way to have boto support both with the same code base. So, then it becomes a matter of maintaining multiple versions of boto. Just shoot me now. If the barriers weren't so high, more packages would be running in Python3.x.

Yep, everyone has limited time and resources. Another problem is that many libraries have dependencies, and they can't start porting until the dependencies have been ported (or a replacement is found). Finally, library developers only want to develop for languages they like using, and who likes a language with no good libraries?

It might mean they'll have to support two versions for a longer while though. Which also means making sure all community patches are provided for both branches, or manually integrating them.

Which package is in the wrong color?

Pretty much inaccurate data. For example many project release under a new "3k" name, and leave the old ones. One of those is a backport of a stdlib module (multiprocessing), etc. So, it's bad data, and misleading, but I admire the idea.

Thanks :) You'll notice I removed multiprocessing. I'm considering removing setuptools as well. Too bad I have to hardcode these things. I wish there was some metadata about a package having a py3k counterpart.

Is this a wall of shame for the packages in red, or for Python 3 itself?

I think my inspiration was a little of both.

I think a large part of the problem is that there aren't strong incentives to switch to Python 3, or at least that was the case when it first came out, and first impressions tend to matter. To me, as a user, it looked like "slower and none of my libraries will work."

A better way to sell python 3 should be to highlight python 2's pain points, and show why 3 is better. If they were to show a significant performance gain over Python 2.x, or some sort of killer new feature (ie, get rid of "self" everywhere) I suspect python 3 would get a lot more traction. As it is, Python 2 as a language works just fine for me, and most of what I want (a better interpreter) are being addressed by the PyPy project.

At first I was going to say that we need some sort of dependency graph to focus on what needs to be ported first. As a Plone developer, I see lots of stuff down the stack that needs to be done before Plone could move to Python 3. However, simply ranking based on number of downloads seems to do a pretty good prioritization.

Out of the first 10 packages that haven't been ported to Python 3, I think nearly all are needed for Plone. (Although things like virtualenv are not a requirement for Plone, they are widely used in Plone development)

I am currently doing a similar sort of website, but trying to limit it to popular projects only. Also trying to group together a lot of the python 3 related articles and porting tips.

My current research can be found here: http://goo.gl/SCImr

Any corrections, or ideas for what can be on the site would be appreciated.

You're welcome to use content from https://bitbucket.org/pypy/compatibility. There is a growing list of dependencies at https://bitbucket.org/pypy/compatibility/wiki/depends.yaml.

That just saved me a hell of a lot of work. Thank you!

You're welcome. You can even clone that and make a wiki for py3k from it if you want. There's a script (in the wiki repo) to parse the wiki into YAML, so reuse of the current content should be easy no matter what format you want.

So this is a small shout out to Pythonistas. Get involved with a project and help port it.

It will be interesting to see how Guido et al will judge the py3k transition-strategy in hindsight and whether they will repeat it or move back to a more traditional, incremental development model.

If you 'incrementally' include changes which are incompatible, you end up with many more compatibility barriers to keep track of.

"Oh wait, this machine has foo 1.5.2, not 1.5.6!!!"

Though a solution to this scenario is to avoid including too many libraries in the standard distribution in the first place, so that incompatible changes in those libraries don't affect the base.

This in turn means that big packages with lots of dependencies would need to say "you need fooliba-1.5.6, foolibb-4.3.2, etc" instead of just "you need foo-5.2", but other languages do this and they seem to manage it alright.

Well, most other languages use the incremental approach, python is the outlier here.

I'm not saying one or the other is definitely better, just that I'm looking forward to the final judgement after this multi-year effort.

Arch Linux uses Python3 as the default system Python, which I found a bit daft as almost every Python library I use relies on Python 2.x.

Just picked one from the red list (pytz) and it looks like Py3.1 is supported...

And beaker too. Most projects i've clicked through only offer some source tarball or easy_install instructions but no mentioning of supported Python versions..

I am wondering how the list would really look...

Too many authors don't update their classifiers before uploading to PyPI. Plenty of things on that list are incorrect, but it's the package author's fault.

There's a lot of improvements in 3, but I'm sad to say I'm really hung up on print (and that's why I'm still using 2.x).

Eventually, I guess. If I have to.

Yeah, there's no good reason to change print. Sure, making it a function can be handy; but they should recognize the "print foo" syntax and compile it to a function call.

The real issue is that for both developers and library maintainers, moving to Python3 entails a lot of aggravation for a comparatively small net benefit.

Porting is actually fairly trivial in many cases. If you go and look at most of the porting stories for Python 3, you'll find the authors saying "I was worried it would be hard... but it was really easy".

I have the impression that translating the Python is easy (it was for the one project I did), but the C bindings are more of a pain.

The wall of shame itself is written in Python 2.

Well of course - there's no libraries for Py3k.

Interesting to see pymongo on that list.

What about other databases?

Edit: Mysql appears to be represented too

SQLAlchemy is ported at least, don't know about all the low-level drivers though.

You don't need to name it as Wall of Shame.

No, that is a sensational title. It got everyone to look though didn't it.

I just realized: Python is supposed to move slowly. Maybe they should have come up with a better name?

Why isn't reportlab in the list?

well, it's the same with php4 and php5. and php5 has been released in 2004 ...

Is it? I was under the impression that there's very few hosts/libraries that don't have PHP5 support now.

PHP 5 was still, as much as I can remember, compatible with PHP 4, wasn't it? Anything that worked in PHP 4 worked in PHP 5.

That's not true. If one used OOP in PHP4, running that code in PHP5 will produce lots of bugs related to the new object model.

What place is still running php4? The 4-series has been abandoned even for critical security patches some time ago, the move to 5 solidified a long time ago.

Maybe you're confusing PHP 5 and PHP 6; PHP 6 has been "just around the corner" now for a few years.

does this highlight how hard it is to move from python 2 to 3?

sidenote: love the pink select must be html5 boilerplate :)

funny... everyone needs to run out and convert so that the 3 people using py3k can have new packages... doh

And the reason only 3 people are using py3k is that there are no packages for it...

So what is the point in using py3 then? They either should keep using the version with the largest library base, in order to profit from other's momentum, or invest their own ressources (money/time/skill) into making a py3-only library attractive enough to pull everybody else against their will.

What they're doing right now, is not investing anything at all to make py3 more attractive, but simply being loud and vocal and trying to coerce the majority through social pressure to do work to support their version of choice, even if there is no actually noticeable benefit in for them to do so.

Well...eventually 2.7.x is going to be it except for security patches. The future of Python is the 3 branch and I am confident that everyone will port. You have to keep in mind though that P2->P3 was laid out as a 5 year process. We are only half way through it at this point.

"Get the source for this GAE app at google code."

Google App Engine should be in the list as well. GAE supports Python 2.5.

GAE is not distributed through PyPI. I think the list is generated from that.

Google only supports 2.5 for anything python related. Take a look at the python protobuf client:

http://code.google.com/p/protobuf/issues/detail?id=66&q=...

Don't expect google to upgrade anytime soon.

Python 2.7 is now on the roadmap.

http://code.google.com/appengine/docs/roadmap.html

Oh now that'll be nice. Wonder if they'll let let me future import ;) I seem to recall some restrictions in how they implemented the VM for AppEngine.

Reply on news.ycombinator.com