Python 2.x vs. 3.x survey results | Hacker News Reader

Python 2.x vs. 3.x survey results

wiki.python.org

154 pointsjanzer79 comments

I find these responses promising. I see some negativity here that after 5 years, over half of the devs are still on Python 2. But reading some of the threads in the last few weeks, you would assume that fewer than 5% of all people attempting Python 3 stuck with it. And this survey, if it is representative of the community at large, obviously doesn't show that.

Sure, more people use Python 2 than 3, but is that really surprising with the legacy code out there? The point is that Python 3 is also being used quite a bit.

Don't you think there's the tiniest chance that 3.X users will over-represent themselves in a nonrandom survey just as they do amongst the developers?

This is laughable: "we asked people with a vested interest in seeming more numerous than they are to take our survey. And they did. Reassuring!"

We ran a user survey for IPython, where there's little reason to expect a bias towards Python 3 users. 22% said they use IPython with Python 3.3 (they could select more than one answer). I guess early adopters may be more likely to answer surveys, but from this and bug reports, it does feel like there's a growing minority using Python 3.

I don't think py3 devs "over-represent" themselves at all. If anything, it's very fashionable to proudly announce that py3 is a waste of time and you're never going to use it.

The problem is more that this is a biased representation: if people on the Python development mailing list (which I can't imagine spends much time talking about Python 2 ;P), people who care enough about Python to still be on Usenet using comp.lang.python (although they seem to actually mean the official Python user mailing list, given the hyperlink...), and the programming hipster crowd from Hacker News (which is one of the few communities I am a part of that seems to remotely like Python 3) are at this low of a representation, the average developer base is going to be much lower ;P. Given this prior expected distribution, these results are thereby quite damning evidence against Python 3... (which matches the less intrinsically flawed, although still problematic in the other direction, data from a few days ago that less than 2% of downloads from the Python Package Index are from Python 3 (this is problematic as this is likely biased somewhat by production websites that are spawned and download packages over and over and over again; that said, that expectation itself is demonstration of the relative strength of Python 2, and could even be considered an ignorable bias if the question is not to measure users but instead to measure importance).

Fewer than ~22% are sticking with python 3. Admittedly that's more than 5%, but I'd hardly call it a blazing success story, when 60% of people have tried it.

So, 22% of 60% of people all up are sticking with python 3. 13%. Woo...

Where do you come up with 22% of 60% as sticking with it? The 22% I assume came from the question of which do you program in most. I program in Python 2 most currently too because of legacy projects. But I also do a lot in Python 3 as well, just not "most".

Less than 4% of pypi download stats are Python 3, so i doubt the results of this survey are representative.

I'm wondering where that number comes from? I've seen this stat a few times recently but don't know where it came from.

Just doing a quick perusal of the top 5 projects listed at http://pypi-ranking.info/alltime

1 (boto) does not support 3.x

3 (distribute, zc.buildout and pip) only distribute non-version dependent files

leaving just 1 (lxml) with version dependent installers listed at all and it is only for the windows installers.

Drilling down into lxml, roughly 95% of its downloads are the version agnostic .tar.gz file. Also the version dependent installers do not include a 3.3 version but only 3.2 for python3. But even with only an installer for a old version of python3 available it still is >8% of version dependent downloads.

I suppose it was derived somehow from https://pypi.python.org/pypi/pypstats

As a number of people have pointed out though, there's no new data since May 2013, no idea why (https://bitbucket.org/pypa/pypi/issue/33/package-stats-no-lo...), so the the '4%' number is probably out of date anyway.

> Do you have dependencies keeping you on 2.x? > Yes: 61.46% > No: 38.54%

I know that's why I returned to 2 after trying 3. For example BeautifulSoup (HTML parsing/scraping) has only supported Python 3 since October '13. There was nothing wrong with 3, it was just dependency issues.

Plus, when 2.7 was released it was done with a promise of ~6 years of support[0] to aid in the transition to 3.x. I think the definition of success here needs to take that into account.

[0] - https://mail.python.org/pipermail/python-dev/2010-May/099971...

Python 3 meets my needs just fine.

However, I do tend to live in the future.

What's also missing here is differentiating between "dependencies" as other python libs or host apps with built-in python interpreters stuck to a particular version.

For example, VFX studios are generally stuck on 2.6.4 as that's what Maya 2012, Nuke 7 and other commercial software is using.

Lots of VFX studios have hundreds of thousands of lines of Python code integrating stuff across their pipeline. Until the host apps support Python 3 (some like Houdini support both 2/3 via different plugin proxies), it's a Chicken and Egg situation, as VFX studios won't port their python code to 3 until they need to, and the commercial companies making the software above aren't going to waste time adding Python 3 support until their customers ask for it.

So I'm assuming 2.6/7 is going to be around for a while...

One solution would be to provide the two in parallel. But then again this area is slowly moving. I just recently (early last year) wrote my last MEL code and moved fully onto python.

Yeah, but that's quite likely (without some sort of façade layer) going to be double the work adding new functionality, and it's definitely going to be twice the testing of the entire Python functionality of the apps.

What we _really_ need is a way to manage python2 and python3 codebases, infact we need a story to manage pythonN+ codebases interoperating in the same process. How is this a one time event? It isn't, nor should it be. Everything is future legacy.

Versioned imports down to the language level. Each file should say what version of the language it uses. Code once written that works should not fail to work in the future.

Some are stuck on the 3.x series, like Blender.

Speaking from the perspective of someone who doesn't code in Python: I recently considered doing so for a scientific computing task at work but quickly ditched the notion. Python 3 seemed the recommended choice for a green-field project, but good luck getting anywhere with it due to poor library support. Want to access a MySQL database, generate 3-D graphics, or use someone else's code? Good luck with that! Python 3 may have been released five years ago, but uptake among developers appears negligible. So why not use v2.7? Several features in v3 looked appealing, and documentation makes it clear that v3 is the future. So I decided not to invest in a language that can't make up it's mind and stuck with R.

I wish more people in the dev community would ponder your experience and decision-making process.

I don't think Python 3 was a mistake. I understand the reasons for doing it, and the difficult decision making process there. I also have tremendous confidence in core dev team and Guido to make the right decisions even if they are difficult.

However, I think calling it Python-next or -NG or something non-numerical would have been better, and CLEARLY signalling that Python 2.x IS NOT DEAD. The big problem is that 3 is bigger than 2, and by advertising that 3 is not yet ready but 2 will no longer be supported, there is sort of an Osborne effect (http://en.wikipedia.org/wiki/Osborne_effect) that is hurting adoption of Python 2 AND Python 3.

"Python 3 is the future" but "Python 2 is also the immediate future", like for the next 3-4 years AT LEAST. My company is undoubtedly going to need to support it for well beyond that. There is no problem here. Python 2 evolved so much over its point revisions that the idioms in 2.7 for instance are vastly more concise and improved from 2.4. Would it have made sense for someone to not learn Python 2.4 just because eventually 2.7 would improve or deprecate some of the things they'd learned?

Looking through the main python.org page through the eyes of a beginner, it's hard to find clear guidance on this.

My company is squarely in the space of promoting the use of Python for scientific computing and data analysis, and it's difficult sometimes to get folks in the community to recognize the difficult job of ambassadorship to other languages, like R, Java, C#, etc. That being said, there is still tremendous adoption of Python for data and scientific purposes, so I'm not really worried. I'm just sad to hear that a potential newcomer was driven away by unclear messaging.

(Especially to stick with R... <shudder> :-)

> So I decided not to invest in a language that can't make up it's mind

It's a bit unfair to try to group the actions of vast swaths of people under the same banner, and penalize them for not acting in concert.

Why isn't the entire Python community using 100% Python3 right now? Because they are a disparate group of people with differing motivations and goals. It's only been in the last year that some major projects have started to support Python3 (e.g. Django). Not everyone has the luxury of spending all their time porting over existing code-bases to Python3 the moment that it's possible.

When choosing a language, the community is (almost) everything. If the community is divided and unwilling to support v3, that is more than enough reason to "penalize" them and not use the language.

I've little doubt that the folks using Python are making the decisions that are right for them. The underlying problem that I saw in my survey was that Python 3 broke compatibility while not offering developers a compelling reason to update their code beyond it being the right thing to do. Hence, folks are taking their time doing so.

Developing in Python also seems an either/or proposition: either you target v2 or v3. This puts library authors in the unenviable position of supporting two code bases.

This is unnecessarily pessimistic. Most of the major packages support Py3k and if they don't it's usually very simple to port over (it usually means combining 2to3 with some bytestring to string conversion debugging).

I'm not in the scientific field so forgive me if I'm being parochial but I've been writing software in Py3k exclusively for a year and half now. Using scientific computing, distributed queue processing, webapp, CLI utilities, etc...

[EDIT] Better words.

> This is unnecessarily pessimistic

Remember, he's looking at using Python for the first time. I work in Perl-heavy environments, and I can assure you that the 2/3 split in Python is a reason that many will give for not using Python at all. It's very different among those who are already comfortable with Python.

> Most of the major packages support Py3k and if they don't it's usually very simple to port over

For you, maybe. He's just starting with Python.

With regards to scientific Python, I took a peek at the install link on the Ipython site. It says that Ipython works with Python 2.6, 2.7, and >= 3.2. That was only mention I saw (in an admittedly casual look) for Python3. All the examples were in Python2.

> the 2/3 split in Python is a reason that many will give for not using Python at all

I don't know Python, but I've gone through a few phases where I'll play with it for a bit and then put it back down. (Incidentally, I appear to be entering a "play" phase right now.)

I can't explain it exactly, but the 2/3 thing makes me really uneasy about learning the language.

On one side, there is Python 2 which is supposed to die in the non-immediate future. I don't need to learn it, particularly since development isn't my primary career at the moment and I'm unlikely to "need" it before I inevitably switch careers back to dev. If I don't need it now, and I don't expect to be in a situation where I need it in the near future, why not go for Python 3 right away?

On the other side, Python 3 is the new hotness, and it's the future of the language. It's the heir apparent. It's going to be "the" Python in the nearer-than-you-think future.

But it doesn't seem ready. You get the impression that you'll start a greenfield project and then get to a point where you want to drop in a well-known component and you just won't be able to.

You also look at the fact that it's been around for over half a decade yet 80% of of respondents to this survey say they use 2 more than 3, 40% have never touched it, and 20% say the whole thing was a bad idea. If you were walking into that environment, would you really be so quick to disregard it?

I suppose the answer is to just learn it all and try to keep the differences in mind as you go, but it's understandable the a lot of people don't want to half to learn and semi-unlearn the same language all at once.

It's definitely a shame to hear that has put you off. In any case, I'd recommend you do have a proper play with it - it's a lovely language.

If you wanted to be safe you'd just start with 2 as it's easier to get help while learning the ecosystem. There's actually not much to learn / relearn when switching - the difficulty has been moving the dependencies to 3 (which is well underway now).

The shocker for me was seeing that MySQL-python didn't support v3. And I recall all sorts of problems getting IPython 3 running in Emacs alongside the system version of Python 2. Porting a library to v3 was more than I was willing to bite off and a brand-new user when I had a customer deadline approaching.

I haven't used MySQL-python in years. It's long bordered on being utterly unmaintained. Use oursql or pymysql.

I was shocked by MySQL-python not supporting Python 3 as well, but then I discovered that there are several better alternatives to MySQL-python that do support it, such as pymysql. AFAIK it's 99% compatible with MySQL-python.

There are actually several MySQL drivers and at least 2 worth using on Python2 and Python3:

MySQL driver 1: http://pythonhosted.org/oursql/ MySQL driver 2: https://github.com/PyMySQL/PyMySQL

Both are well supported drivers. Much nicer to use than the old Python2 only MySQL-python driver, imo.

I use pyodbc[1] to connect to SQL Server from my Python 3 code and it works just fine. Does it not work well with MySQL? I couldn't find any indication of that in an admittedly cursory search. Are there advanced features you need that are outside ODBC's scope?

I agree completely on the graphics side, though. Terrible.

[1] https://code.google.com/p/pyodbc/

I'm surprised, if not saddened, by the low popularity of `3to2`: less than 2% of the people surveyed have used it, even though it is a convenient way to have your code in python 3 when stuck with libraries that use python 2.

edit: direct link to the pdf: https://wiki.python.org/moin/2.x-vs-3.x-survey?action=Attach...

By the time 3to2 came out, the community had already figured out that it is a better path to right code that runs under 2 and 3 rather than using conversion tools.

Conversion tools are mostly a relic of when everyone still had to support 2.5. Once you are 2.6+, most people settled on single source as the best approach.

I would presume most people who are writing 3.x code have written 2.x code before, or don't care about 2.x. I personally have gotten used to the syntax that works in both, and tend to write my code in the most compatible manner possible.

That's something for which I should strive. I've never written "polyglot" scripts longer than 1 file with 100 lines. Writing compatible syntax is feasible without too much hassle, but most problems I encountered were when I wanted to use a library whose API evolved between 2 and 3...

Do you have experience with the `six` library[0]? I've read a few times about it recently and it seems a convenient way to write portable code. I still wait for an opportunity to use it!

0: http://pythonhosted.org/six/

You can see some usage here[1]. Mostly just using it to check for string types, but it's the only usage example that I could pull off the top of my head.

[1] https://github.com/twilio/flask-restful/blob/master/flask_re...

I tried for a while to use 2to3, but found it only worked about 95% of the time, which was enough to be really annoying. Personally I'm now just using python 3 where possible, but don't fancy the pain I imagine I'd get with 3to2.

Of course, I could very well be unfairly judging 3to2, but I don't have unlimited time and spent longer than I would like trying to get 2to3 to do what I wanted.

As per results, if a lot of people are stuck with 2.x because of dependencies why would they bother with these tools?

Same reason people use Coffeescript: even if you have to run on python2, it's still nicer to develop in python3.

But nicer by enough to make it worth the bother of invoking a translator before you can run your code? Projects using Coffeescript often go out of the way to hide that step, one way or another (Rails asset pipeline, etc.). And Coffeescript arguably improves on raw JS a whole lot more than Python3 improves on Python2.

Not enough nicer to deal with any of the not-nice things that come with straddling the major version.

Biggest dependency still keeping me back is Twisted, progress: http://twistedmatrix.com/trac/wiki/Plan/Python3

It's either that or writing some blocking code or something. No thanks.

Would you consider Tornado? It's compatible with 3.x and even uses the new asyncio stuff.

I moved away from Tornado recently; I was quickly getting lost in callback hell. This was at least partially due to using libraries which did not support Tornado's method of async io natively, and a lot more fear that even putting these outputs in threads would still cause delays we could not afford.

I moved this project to Go, though I've quickly run into quirks there as well.

I'm not familiar with Tornado but I wouldn't rule it out.

I'm in the same boat, but there do seem to be alternatives which are popping up. The problem is that there is a big difference between the Twisted way of doing things (deferreds), and everything else. Once you start writing code in a twisted way it's hard to port it to anything else.

Have you considered contributing to Twisted's porting process?

Twisted is its own island.

I was pleasantly surprised by these results. Language adaption is not linear, it follows an S curve. So while 22% are on Python 3 after 5 years, we can expect to reach 80% much faster.

So the bigger half of developers stick to Python 2 and the smaller other half actually tried migration to Python 3. After 5 years of promoting Python 3 as the future.

From this short survey we can't really know if it's "developers stick to Python 2", or "have tried Python 3, but migrated because of X", or "haven't tried Python 3 yet", or "work daily on a project that will not be ported to Python 3 as long as possible".

I would expect that the last group is really big - imagine a company with a huge python 2 codebase. You're not going to just say "let's port everything" without a good reason for it. Even after you do, you have to port and prove all your dependencies are working correctly, only then going for the big projects. So while most people will still describe themselves as people working primarily with python 2, there may be a person or two in their teams, who already work on porting...

The original plan was for ~5 years as a transition period. Now that most of the popular libraries work with 3, there's a lot less holding people back (note dependency problems affect most of the responders). Also some of the cool features have been back-ported to 2.x so even projects that haven't migrated are benefiting.

The reality is that after 5 years of development, i.e. around about now is when the promoting is just starting. A significant number of developers are early adopters who started using 3.x even before it is finished.

These are the results from the survey that was on HN a few days ago.

https://news.ycombinator.com/item?id=6990481

The only thing really "keeping" me on Python 2.x is how easy it is to host it. Although Flask/Django/Bottle all support Python 3, most (if not all) PaaS providers I have used only support Python 2.7.

Heroku supports 3.x fine, IIRC.

Looks like you are correct [1] - this was not the case last time I checked. Thanks!

Any others?

[1] https://devcenter.heroku.com/articles/python-runtimes

A problem with hosting Django on Python 3 on Heroku though is that you need to use something else (like s3) for staticfiles. Boto doesn't support python 3 yet, so you can't use the django-storages implementation. The Heroku documentation recommends using dj-static as an alternative to s3, but dj-static relies on a library that was last updated in 2009.

New Relic seems to support Python 3 now though, which is nice.

A different view of the data.

https://docs.google.com/spreadsheet/pub?key=0Aj15Prf9CEepdGx...

Python 3.x has to be the future, there are lots of really well articulated reasons for that. The problem here is that the transition was not, in my opinion, handled intelligently.

What I mean by this is that in five years there are a number of things that could have been introduced incrementally and with minor impact to existing libraries. For example, making range() behave like xrange() seems trivial from a user's point of view. The same with print vs. print().

Yes, some of the other important differences would require breaking code in more serious ways. This, too, over five years, could have been introduced with incremental transition tools that would make the upgrading of existing libraries as automatic as possible.

If a language doesn't have massive and useful libraries you can play with it and break it. Nobody will care and adoption will be swift. That's not the case with Python. Aside from language benefits I'd say the main differentiating reason for which Python is really valuable is that there are libraries out there to do just about anything you want. Breaking (or un-breaking, depending on your perspective) the language breaks the libraries, therefore, no matter how wonderful the language might be it is a non-starter.

The transition from one version to the next has to be automatic. Nobody has the clock cycles to engage in months' long research projects to comb through code and fix things. No, tools like 2to3 are not what I am talking about.

One non-coding example I can offer is what happens when you open an old Solidworks file with a newer version of Solidworks. The program immediately fires-up a format converter and tells you it is converting to the new file format. Once done, you save and you are done. All you have to do is open the file and re-save it. That's how it should be. Surely in this day and age there's enough CS intelligence out there to make a fully automatic version-to-version converter. Now that would be something. It would certainly bring the Python community forward in unimaginable ways.

I'll go further and suggest that this could actually be an idea that could set Python apart: No new version is released without a matching fully-automatic old-to-new converter.

Hey, ideas are easy.

For those interested in python 2/3 that haven't (recently) read the (unofficial) faq, I highly recommend taking a look:

http://python-notes.curiousefficiency.org/en/latest/python3/...

One thing I don't understand is if there are third party libraries holding people back then why isn't there something in the python 3 standard library that you can substitute?

I would love to move to Python 3 as I have no technical dependencies. The only reason I'm still writing my code and open-source library in Python 2 is the ecosystem of my clients. Even if that was technically possible to migrate to Python 3, it might not viable business-wise. It would be considered as unnecessary expense for something that "just works".

I guess we just need to reach some adoption tipping point or end of support of Python 2. Whichever comes first.

I kind of get the sense that a lot of the recent belly aching is because people can smell the adoption tipping point coming.

I thought about this, as someone with reasons to prefer Python2. But then I quit worrying. Python2 isn't going anywhere, for starters. But more to the point, if lots of people move to Python3, some of them will share my issues. They'll make noise, push for resolution, offer solutions, etc. Widespread adoption makes it more likely that my concerns will be addressed.

Nowadays most libraries have python 3 versions, but not every machine has python 3 (or even 2.7) installed. Most of times it is very difficult to install softwares on those machines. The availability of python 3 is my major concern.

What are the conclusions from this survey?

A PDF? What is this?

PDF stands for "Portable Document Format". It was invented by Adobe; you can find more information about it at http://www.adobe.com/products/acrobat/adobepdf.html .

It's an excellent format for sharing documents, because, unlike, for example, HTML, it preserves formatting exactly and it can be self contained: you can embed all the fonts for a document within the PDF. It has some disadvantages compared with HTML: it doesn't reflow and is slower to search through, for example. But when you need or want a document's formatting to be preserved in detail, PDF and Postscript are the only open specifications you can turn to, aside from just using images.

I meant it in the "What is this shit?" or "What is this, 2004?" sense. There's no reason to use PDFs, especially when we're talking about developers and development. I'm not going to install a program just to view your document.

What are you running that can't view a pdf?

It's not as much "can't view a pdf" as it is "not willing to pull in 20 packages and dependencies to view a pdf." You might as well send it around as a spreadsheet: who doesn't have Excel installed???

I appreciate the text-based version that was posted in this thread, thank you!

If you're working solely in an environment so minimal and obscure that it can't view a PDF, I think you've lost the right to be catered to (and complain) when it comes to document format decisions.

The comparison to Excel is absurd; PDF is an open format and has been for more than five years.

what system are you running such that viewing a pdf takes 20 packages and dependencies? How are you on hacker news if you can't get your system to view pdfs?

I don't have Excel installed, but can PDFs just fine, thank you.

He means, "wtf, why is there a wiki page with a download link to a pdf that contains maybe 2k of data tops" we both know he knows what a pdf is. He is complaining about a shittier version of sharepoint, something we can all get behind.

There we go...

Q1 Have you ever written code in Python 2.x?

Yes - 97.51% 4,660

No - 2.49% 119

Q2 Have you ever written code in Python 3.x?

Yes - 60% 2,862

No - 40% 1,908

Q3 Do you currently write more code in Python 2.x or Python 3.x?

Python 2.x - 78.09% 3,692

Python 3.x - 21.91% 1,036

Q4 Do you think Python 3.x was a mistake?

Yes - 22.91% 1,071

No - 77.09% 3,603

Q5 Do you have dependencies keeping you on Python 2.x?

Yes - 61.46% 2,883

No - 38.54% 1,808

Q6 Have you ever ported code from Python 2.x to Python 3.x?

Yes - 32.83% 1,554

No - 67.17% 3,179

Q7 Have you ever written/ported code using 2to3?

Yes - 16.27% 768

No - 83.73% 3,953

Q8 Have you ever written/ported code using 3to2?

Yes - 1.93% 91

No - 98.07% 4,627

Q9 Have you ever written/ported code to run on Python 2.x and Python 3.x unmodified?

Yes - 31.12% 1,473

No - 68.88% 3,261

I know the link says .pdf at the end, but it's actually a webpage with the option to download a pdf. https://wiki.python.org/moin/2.x-vs-3.x-survey?action=Attach...

I was also expecting a long article or something when it turned out to be just a few bar graphs and tables. No complex formatting or anything else that would possibly justify it. Why can't they just put the results on the webpage directly so we don't have to take an extra step? That's what bothers me the most.

Reply on news.ycombinator.com