Making Python 3 more attractive | Hacker News Reader

Making Python 3 more attractive

lwn.net

141 pointsexplosion164 comments

I think the obsession with the GIL is sort of missing the point - people on python2 live with the GIL and don't really miss it, I don't think that's really the killer feature to drive python3 adoption. Especially considering its almost impossible to do without breaking something (most likely C extensions) and when you see the wailing and gnashing of teeth that came from python3 forcing people to fix their text encodings it doesn't look seem like more breaking changes are going to drive python3 further. What's more, python already has quite convenient multiprocessing, and python3 concurrent.futures makes it even easier - frankly I think too many people complain about the GIL without having tried multiprocessing (doesn't help that it seems every tutorial starts with multithreading, then tells you it doesn't really work, and only then tells you about multiprocessing). You only get to complain if that doesn't work for you!

Personally I think the driver is going to turn out to be type annotations. When you see the enthusiasm for adding type annotations to JS (typescript, ES6, etc) its easy to see that translating to python. Static analyzers can be a huge help (you can already get a taste of it with PyCharm) and I for one would like to move away from "traceback driven development" where you just have to keep re-running the code until all the preventable glitches are worked out...

Anecdotal data to agree with type annotations:

We have a fairly large Python codebase here. I've been wanting to move away from Python for a whole host of reasons, a top one being that it's dynamically typed. However, when mypy was released, that was enough of a reason to port from Python2 to Python3. Now we have a mostly-annotated codebase, and I'm a lot happier.

We'll probably move away from Python eventually (I still don't think it's an especially good language, and type checking with it will never be as useful as with a language where it's built in and required), but it's not as pressing of a concern.

I think type checking turns Python into a bearable language.

To which languages could you possibly move, as a replacement for Python?

Making python 3 more attractive is not the solution, it's part of the problem.

I can't use just python 3 because python 2 is still widely used. I can write code that works on both but now I'm using the worst of both worlds, and even worse I now have to test on both. And that'll last until python 2 goes away completely which, - when has a language ever gone away quickly?

These aren't fun problems. Improving python 3, making it more attractive, that's fun. But that's problem 2. Making migration less painful for me should be problem 1. Who's working actively on that?

What particularly grinds my gears is the apparent disregard for migration in the design. Take the changes to the print statement. Often this is the only thing that prevents my existing code from working in python 3. And it's in my muscle memory so I always get snagged when debugging on python 3. For what? Take a read through the rationale: https://www.python.org/dev/peps/pep-3105/#rationale. Most of the benefits you could have had if you'd named the new print function something else. I can understand that you regret adding it in the first place. What's even worse though? Adding it and then removing it in an incompatible way.

If you want to make python 3 more attractive maybe make migration easier before going on to the fun improvements? There are low-hanging fruits for migration too. How about adding back the print statement?

There has actually been a lot of focus on making migration less painful, including restoring some removed features, e.g. Python 3.3 has added back the unicode literals. This is limited to things that are actually painful to migrate though not trivial non-issues like the print statement.

It's not really migration though, it's supporting both python 2 and 3. If you write modules that people depend on you're in a really, really painful spot with the 2 and 3 transition. You can have two completely separate code bases and deal with all the duplication and problems that come with it, or you can try to write code that works with both python 2 and 3 and deal with the huge amount of warts and pain that come with it. Both are painful so it's easy to see why a lot of people just don't even bother trying to support both and ignore 3.

I agree migrating is painful, and supporting both even more so. But not because of print. Migrating print statement usages and even making them work on python 2 and 3 is trivial, even fully automated and statically verifiable.

Compare that to unicode changes, where you need to manually inspect and potentially deeply analyze every single string literal.

Why waste resources making the trivial easier just to please people that will not migrate anyway?

I'm a library consumer not a maintainer, but with info like this- https://twitter.com/mitsuhiko/status/586294700430254080 If you look at the date on the calender, the fact this is still being discussed with virtually no progress like it's still 2008 when 3.0 got released tells me that this is over.

As an end user, I say drop 3.x support in your libraries. The gig is up. If 3 ever takes over, do a 1 time wholesale port and drop 2. Tell people who want 3.x support before then to port and maintain it themselves. Enough complaining.

> I say drop 3.x support in your libraries. The gig is up. If 3 ever takes over

People dropping 3.x support until 3.x becomes the most used version basically guarantees that it won't take over.

Is this a code equivalent to the bystander effect?

I don't think so. 3 will take over eventually if they don't give up on it. But the issue is that the onus of the labor involved is on those enthusiastic for Python3, and the core dev team.

Few to no one in userspace asked for this problem. For me, it's not a problem. I suspect a lot of people will eventually see it my way. Instead of learning/porting from 2 to 3, they'll stick with 2 and pickup something else like Go. I believe adding Go to my toolbelt was a much better use of my time. While there's other, maybe better, choices than Go, my argument is that it's so easy to learn that it's just about as difficult as the transition/porting of existing projects that Python3 would've been.

I personally gain little to nothing by spending any time on Python3. No amount of features, moving distros to 3 by default, or anything else can convince me. All job prospects 2, and all the libraries support 2. No one cares if my resume has Python3 on it, but adding Golang alongside Python2 opened up a lot of opportunities. If a Python(2) user wanted to spend time on Rust, that would also be a great use of time.

For each person I see demanding Python3 support, I want to see that person spending hours, weeks, months, porting everything they can find to 3 and then maintaining it. Because the rest of us don't care.

For me, Python3 has 1 main feature. It enables you to beg library maintainers to port to 3.

Removing the print statement is, to me, an extremely frustrating breaking change to make.

If nothing else, I feel that backward compatibility should be given toward's a language's canonical "hello world" example.

Yeah, this was a very unwise thing to make, at least this could be postponed for later.

What? And make another language breaking version down the line? Better to rip off the plaster in one fell swoop.

It's better to make multiple breakages one by one than not doing them at all. On a plus side, maybe at some point you'd realize this print statement isn't that important anyway to make thousands of edits all over the world.

It can't be that difficult to do a one-off `find . -type f -name '*.py' -print0 | xargs -0 grep 'print '`, change the print statements to functions and add `from __future__ import print_function` can it?

from __future__ import print_function on python2 solves the print function incompat.

Exactly this

Why is changing 'print stuff' to 'print (stuff)' such a big deal?

I think 2To3 solves a lot of print cases https://docs.python.org/2/library/2to3.html

The print function is clearly a better, more precise way to handle it, to be sure...

But the print statement was one prominent, attractive way that Python was essentially pseudocode-- its removal seems an improvement from a pedantic sense, but one that seems to run contrary to convenience and (perhaps) the expectations of a beginner.

I disagree about this being worse for beginners - if we're teaching a new programmer about functions, surely a basic print function is a great introduction? Whereas before it was a one-off special case.

As a beginner, I ~hated~ the print statement. The print function had analogous syntax to all other functions, whereas the print statement always seemed to use some kludgy tricks.

To what end though? It's a small thing sure but it's busywork. Why am I being asked to do busywork?

Yeah, I don't like the busywork that Pep8 makes me do it, and it's a lot, and some of it is stupid, but it has to be done

You certainly don't have to do it if you don't want Python 3 and 2to3 makes it a lot easier

>Yeah, I don't like the busywork that Pep8 makes me do it, and it's a lot, and some of it is stupid, but it has to be done

Actually no, it hasn't. PEP8 is just a formatting standard, your code will work with your own standard too.

True, I meant this https://pypi.python.org/pypi/pep8

Guido doesn't like that tool; they're guidelines, not rules.

> Guido doesn't like that tool; they're guidelines, not rules.

Yes, a lot of people need to get that memo

I don't like it as well, but it keeps a group of developers in check. And it can be configured, so there's that as well

Too bad autopep8 produces bad results out of the box

Migration is a critical issue for python 3 adoption. I think python 3 should have done something similar to Protocol Buffers v3 (proto3) by introducing syntax = "python3" statement and let runtime support both v2 and v3. This allows people migrate one file at a time. In any large scale production environment, this is pretty much the only practical way for migration.

Another critical mistake python 3 made was unicode support. For people who work deeply in i18n support, UTF-8 encoding is the only practical solution. Python 3 string should use UTF-8 much like Go. Just for reference, Google stores almost all data as protobuf, which only support UTF-8. That proves the story.

A runtime that can support both v2 and v3 files is a nice idea and has been proposed before, but is virtually impossible because of the unicode changes.

I don't understand where you see the mistake in Python 3 with unicode. What encoding Python internally uses to store strings doesn't really matter. What's important is that it is always known what encoding is used. This was unclear in Python 2 and Python 3 fixed this.

Nowadays, most of text data are stored in UTF-8 format. If the language uses UTF-8 as native string format, it is much easier for text processing, which is why Go chose it. There are a lot of subtle technical details you will only realize after working with many text processing components.

Can't agree more.

`print` statement is not a big deal for me. But one thing that especially annoys me is the functions in the module `itertools`. In Python 2, I use `itertools.izip` or `itertools.imap` a lot, because I'd like to avoid creating a large list only to iterate over once. When I tried migrating to Python 3, I found out that these functions are gone, and I was supposed to use plain `zip` and `map` instead. I was furious. How easy it is to let `itertools` contain these functions as aliases to the builtin? And I felt that I was punished for writing efficient code, while people who code carelessly get rewarded. I switched back instantly.

Doesn't 2to3 handle all of these problems?

2to3 will write you a patch so you can port the code to py3 if that's what you want, but if you then also want to maintain py2 compatibility you've got more work to do.

I can't speak to the wisdom of making map and zip builtins, but that's what they are in py3, and the six module will make using them that way automatically backward and forward compatible:

    from six import map, zip

Map and zip were builtins in 2.x as well, just not the itertools.imap/itertools.izip variants.

It is not the difficulty of handling these situations. It is the lack of regard for Python users like me. Again, as I have said, I feel punished for milking the most from the language, whereas those who code carelessly are rewarded with performance improvement and no pains in transition. Why should I switch to a language that punishes me for good behavior?

It is the same as the `u` prefix. Those who actually cared to get Unicode right got punished when Python 3.0 came out because now none of their code compiled, whereas those who just assume everything is ASCII went on without problems.

Why couldn't they have had a solution like with Fortran, where there are two incompatible versions with different file extensions? Then let projects be written in both .py3 files and .py files.

>I'm using the worst of both worlds

To me this is the main pain point and I'm surprised that people are not talking about this.

2to3 and 3to2 are too kludgy and writing native py2-and-py3 compatible code is quite painful and requires a number of workarounds (e.g. unicode/str/bytes type, different methods on dictionaries...)

Does the `six` library not help with this?

Making print a statement again could break a lot of Python 3 code. In Python 2 it's a statement, and statements can't appear in lambdas. In Python 3 it's a function call, and function calls are expressions, which can appear in lambdas.

I don't think it could (easily) be resolved by treating it like a function when followed by parens and a statement otherwise. For illustration, consider what happens when you use parens after print in Python 2: the parens are treated like they would be in a mathematical expression — only there for the sake of order of operations. The parens effectively disappear when parsing.

I don't mind print being a function so much, but rather the required parens. If they could somehow be optional, I'd bet 90% of the problems associated with the break would dissolve.

GIL seems to me to be only a niche problem in reality. But it is a bit of a storm in a powerpoint (i.e. people see it on a list of features of python and panic. Even though, as the article says, Javascript has much more constrained single-threadedness).

I've been deploying python for almost 20 years, and I haven't had a single performance issue that was caused by GIL and couldn't easily be worked around. In my experience, building big multithreaded applications with shared memory access isn't great design anyway. I prefer systems that share as little as possible, and can therefore scale beyond a single machine when needed.

So I think the focus on the GIL is a false quest. It isn't a bad compromise to a thorny implementation issue (it allows certain performance optimisations, without forcing you to worry about re-entrancy and atomicity when writing simple code). Removing the GIL will be a big thing for pundits, I think, but won't make much of a difference to big python deployments. It certainly isn't the killer app for Py3 adoption.

Disclaimer: I used both Python and Node.js at job, and use Node.js regularly for side projects.

I think the difference is that Node embraces the single-threadedness as part of its architecture. I don't use Python that much now so I'm not sure if it's changed, but I found it clumsier by default.

I know there's Twisted and other nice evented/multiprocess libraries that make Python closer to Node, but it surprises me that a batteries-included language with a GIL has to be, as you say, worked around. I don't work around single-threadedness in Node, I work towards it! And the platform encourages me to do so.

EDIT: I'm not sure why you seem to have been downvoted (man, I hate HN sometimes). Here's an upvote to counter it.

GIL is a bogeyman, particularly from folks who've never deployed large python codebases, so I was fully expecting downvotes. Unpopular opinions attract them. Though it would be more gratifying to see downvotes along with people's use cases for when GIL was a genuine issue for them.

I agree that Node has its single-threadedness baked in, and Python doesn't clearly have one way to do it. Still, I think that multithreaded programming isn't very scalable in the long-run. Learning Erlang and figuring out that message passing buys you much more headroom was important to me. I like that you can do simple things in Python simply, but it is important to have some understanding of why different languages solve the problem in different ways, and what approach is best for a problem domain. My sense is that Node's default approach is actually very good for a large range of situations, a larger range than multithreading, anyway. But that could be because my use-cases are often either UI or CPU bound.

In your example, is your domain primarily internet servers (what I assume is Node's sweet spot for commercial deployment).

Sharing nothing between reqeusts is (IMHO) a much more resilient model in web applications, mainly because you are forced to use the database as state and can scale out horizontally. On a single node, something like mod_wsgi will also let you run in multiple forks automagically, and even you can set things like MaxRequestsPerChild to kill forks every 5000 requests to defend against memory leaks.

So yeah, the GIL is not an issue there. You'll just want something like celery for backend jobs, which is nice because it also maintains job state/info that survives process death - which globals in twisted won't do for you.

So, basically, it's a much more resilient way to do thngs, particularly if the front end of your application is request driven. Doesn't fit all models though.

(Also, Django + Django REST Framework is pretty awesome).

I agree, it's not a problem in most spaces.

I'm one of those not so lucky souls that could benefit from having the GIL removed. I do a load of concurrent read only access from multiple threads of execution on a large object in memory. At the moment I'm forced to fork using multiprocessing. It works, but it sucks to have the overhead.

Though, as you say, it's really a non-issue for most workloads.

> The Unicode support that comes with Python 3 is "kind of like eating your vegetables", he said. It is good for you, but it doesn't really excite developers

Saddly I agree with that. There needs to be either a big stick (Python 2 being really bad, but it is actually pretty good) or a large carrot ("Oh look 3x performance improvement!").

Something like a carrot was presented during Pycon and that was gradual types (optional types). These reduce some cases covered by unit tests, make code more readable for new developers, help with IDE support, and of course assist with general static checkers. According to Guido 3.5 should start having a partial support for it.

But in general I would have liked either one of these instead (some are contradictory, arbitrary hard, or downright impossible):

* At least 2-3x performance improvement

* No GIL

* Merge greenlet library in the core (to make eventlet or gevent work)

* Some kind of an ahead of time compiler that bundles just the needed interpreter library parts into an executable

* Firefox and Chrome agree to add browser suport for it

* Mobile support (native Android support or Apple ditches Swift and uses Python instead).

Firefox added Python support in the past and there was no uptake. Brendan Eich has a hnews comment on it.

I think 2x-3x perf improvement /is/ possible. I mean we have the example of javascript that went from terrible perf to almost native parity. Of course Mozilla & Google each dedicated an entire team to get there.

We already have the example of PyPy too which /today/ averages a >5x (http://speed.pypy.org/) speed up with CPython!

Looks like this https://news.ycombinator.com/item?id=3096459 is the Brendan Eich comment.

And again, recently:

https://news.ycombinator.com/item?id=9188011

> I mean we have the example of javascript that went from terrible perf to almost native parity.

It didn't have the ecosystem of C extensibility and mature extensions Python has, which drive much of its adoption in e.g. scientific computing (a big factor for Pypy having trouble making inroads).

> There needs to be either a big stick (Python 2 being really bad, but it is actually pretty good)

Doing any non-ASCII string-processing in Python 2 with any regularity is a more than regular-enough beating for me.

> Doing any non-ASCII string-processing in Python 2 with any regularity is a more than regular-enough beating for me.

Can you elaborate on this? Unicode processing is the same on 2.x and 3.x for the most part. There are some differences in interpreter internals, how string literals are represented and the internal representation was changed (and obviously the literal defaults and bytestrings were removed), but other than that the unicode support is more or less equivalent.

The big issue with Python2 is that it's easy to accidentally mix unicode and byte strings during development, and it works fine until some user has a non-ascii home directory or similar. You get bugtracker conversations like "it crashes on XP but works on Vista", "I can't find this traceback file you mentioned", "what is a file system encoding?", etc.

Or you write a logfile parser and it works great for half a year, until in March where you get an UnicodeEncodeError because March is "März" in German, the only month with an Umlaut.

I strongly disagree, in regards to Unicode. Unicode in Python 3 alone makes it worth switching from Python 2.

Reasons I am excited about Python 3:

* "yield from"

* Unicode support (I'm German and the clear distinction between bytes and unicode really makes my life easier)

* function annotations (PyCharm interprets them and uses them for static type checking)

* cleaned up stdlib (not only names, but also features)

* asyncio

Library support is very good nowadays, pretty much all of the important libraries are either ported to Python 3 or have an active fork. Even OpenStack is working on Py3 support.

Most of this stuff is great and I'm actually looking forward to moving to py3k. But I've been looking forward for a few years now, though, and it seems like that will continue to for a while longer.

The main problem for me remains dependencies. Python makes it so easy to integrate stuff via pip/easy_install, but that's a double edged sword in this case: Since there's such a huge abundance of great libraries for anything out there, and they're so easy to install - people use them. And now I'm stuck with non py3k compliant libs in a big codebase (some of them not trivial), and suddenly there is a much bigger cost to switching - migrating away from, or porting of all those deps.

So now all those up-sides which are nice to have but are not real game changers (I'm handling Unicode fine - albeit in an clunky and ugly way, but it works), are not worth the pain. And that's sort of a chicken-and-egg thing.

If there was a much bigger gain from switching, that will outweigh the cost - e.g. better concurrency, better overall performance - it would be worth the pain. Otherwise, I guess I might use py3k for new stacks I'll build from scratch, but not for the current stack I work with.

Do you have a list of dependencies that are causing you the issues? When I did my migration (about 9 months ago) I found that there were a few dependencies that had been superseded by much better libraries. At the time there were a couple where I had to use a py3 branch, but no longer.

One of the most annoying ones is Thrift. But I just googled it and it seems like they are very close to supporting Python 3. A couple others - mysql-python, fabric, oauth2. These are the ones that are on the wall-of-shame. But I didn't test the entire dependency set.

I think on the mysql front there are a few alternatives (I use postgres now - and if there was any way of going that route, I'd do that instead). I believe the pure python version is the popular choice these days [0].

I mentioned it because when I went through the process, I was looking for py3 support from my existing libraries and didn't consider swapping them out (as in, initially it didn't even occur to me).

In some cases I found far better libs that were more actively developed (eg I ended up using xlsxwriter for my excel stuff and it was way better than my previous solution).

[0] https://pypi.python.org/pypi/PyMySQL

ps I believe they like to call it "Wall of Superpowers" these days :)

Depends on your requirements on code/performance, but in one of my projects I've been gradually switching over to python 3 (actually, moving old stuff over from 2) by using this module:

https://pypi.python.org/pypi/python-bond

It spawns a second python process that you can call/execute code from almost invisibly, first-class exceptions included. The main difference with other similar solutions is that it support call backs: a remote function can call back new code, and can do it so recursively.

You can intertwine old code and new one.

The main drawback is that it's not efficient for small/lightweight functions.

Cleaning up Exception chaining, and removing unnecessary boilerplate from super() were two more nice improvements I personally really enjoy.

exceptions are actually the most important thing for me in practice, even though almost no one ever mentions that. guess only my code ever throws exceptions. /s

So far I've focused on making new projects both Python 2 and 3 compatible, but "new stuff" in Python 3 is not always available when:

- Laptop running Ubuntu Trusty: Python 3.4

- Production servers with Debian Wheezy: Python 3.2

- Laptop running Fedora 20: 3.3.2

You don't usually install Python in Linux using upstream, you install the version provided by your distribution and unfortunately Python 3 has differences between these versions (eg, "yield from" was introduced in 3.3, IIRC), whilst 2.7.x it's been without changes for a long time.

Even Apple includes 2.7 now (handy when I distribute a game made with Pyglet); so I'm excited too, but Python 2 still makes my life easier.

You can install the binaries with a Virtualenv (at least on 2.X, virtualenv on 3.x is a pain until 3.4)

Python is actually much better than almost all equivalent languages in this. Consider that custom package managers like rvm/bundler, nvm/npm, etc., are considered must-haves for any kind of serious development on Ruby or Node, whereas use of virtualenv is still kind of seen as a "bonus" feature. The fact that Python is good enough to allow that demonstrates its robustness.

Stop using Python 2. All of the justifications I'm seeing in this thread are complete non-starters. If you want to use a 3.3 feature, install Python 3.3. Are you seriously suggesting that we revert to the problems of 2.x, discarding all the effort that's been dumped into Py3 compat over the last 7 years, because you don't want to install a software package? It's really common to need to add extra sources to a package manager for new versions of things, and it's also common to need to build your own packages. It shouldn't be that hard. Be grateful that at least there's a possibility you can use the system Python, since Ruby communities don't really have that luxury.

After we migrated to Python 3, we came to regret the decision with considerable regularity. We mostly use MacOS, and working with OpenGL and scientific libraries has been painful. We never found that a library we needed was not available for 2.7, but lots of stuff has not been ported to 3.3. The syntactic advantages of 3 have been too minor to warrant the troubles we earned. Your mileage may vary, if you have different applications, of course.

Debian Jessie and Fedora 21 both ship Python 3.4, so that problem is going away.

In the meantime, it's always possible to install a self-contained Python 3.4 interpreter and a virtualenv. It's pretty straight-forward, actually. The only thing you'll miss is distribution-provided binary packages (like lxml), but pip/setuptools compiles them for you.

Refer to this comment for an alternative way to install Python 3.4 without compiling it yourself: https://news.ycombinator.com/item?id=9389742

I don't think any of the things proposed will really drive conversion from Py2 to Py3. I don't think developers need to be excited about it, it just needs to be plausible. Python 3 has some significant momentum now (these talks seem overly negative about it). Developers are waiting for the signal from the enterprise Linux distributions that Py3 is "truly stable", i.e., they're waiting for it to be made the default Python. Once this happens, any remaining holdouts, which, again, are not that many of the actual libraries that people regularly use (cf. https://python3wos.appspot.com ), will stop their moaning and finally catch up.

I think if someone hasn't moved to Python 3 yet, no iterative change is really going to get them to do it. It's OK if old software is resistant to breaking changes; this is about building a good ecosystem for the software to come. If it was about making things easy for people who've already written their software, Python 3 would not have been released in the first place.

Honestly I don't really see why anyone cares about whether Mercurial is Py2 or Py3, since it's not a library and isn't holding up new development. Mercurial can use the Py2 interpreter to its heart's content and it shouldn't have any effect on the prosperity of Py3.

The Python community needs to get serious about pushing adoption of Py3 by the distros, and then we can put this navel gazing to rest and move on with Py3 finally realized as the standard.

Mercurial is a canary - if they see so much pain involved in moving from Py2 to Py3, and little reward, then that's representative of the overall ecosystem.

Having people on older versions of the interpreter is a problem, because it divides the community in half when it comes to knowledge, skillset, capability, etc.. It also confuses outsiders who are looking at the situation, and don't understand which version they should move forward with, because it's not clear which one will by supported by the people they want to hire.

There is a real cost to having the community split - which is why so much effort is put into trying to pull it back together.

More of that effort needs to go into compromises back to Python 2.x as opposed to figuring out how to beat people into going up to Python 3.x; it should not have taken until Python 3.3, for example, for the u'' syntax to return. They need to sprinkle a few well-placed "this is a way to get compatibility with Python 2 and 3 at the same time, without crazy tools, at least as an interim state". When I shifted from Ruby 1.8 to 1.9, there was nowhere near as much of a flag day of pain, even though they were attempting to solve the same overall problem (Unicode), and even though Ruby 1.8 was actually abysmal at Unicode (unlike Python 2, which contrary to many of the people who like to try to carrot people to Python 3, does not have any issues with Unicode; in fact, if anything, Python 2.x is better than Python 3.0 was, as Python 3 decided to "fix" a bunch of things, like filename encodings, which actually should not be assumed to have an encoding :/).

my experience with python3 unicode has been just the other way round - there are some minor troubles with python3 in regard to unicode too, but when doing anything user facing (starting with i18n/l10n) I'd take python3 without thinking twice. its 2015 and bugs like the unicode usernames problem in GTA V lately are just embarrassing for developers

I'm not saying you can't do this with python2 but you have to pay attention to unicode all the time. this gets worse A LOT when trying to serve python 2 and 3 from the same codebase, something I wouldn't recommend at all if you want to do eg. application development. of course this is a bit frustrating if you learned to properly do unicode in python2 the hard way, but for new developers not having to worry about unicode issues really is a blessing

Mercurial is a large, complicated project, so it makes sense that they'd delay a backend upgrade like Py2->Py3 as long as they can get away with it. It's not representative of the overall ecosystem. That's like arguing that as long as "canaries" like Facebook don't see any reason to move to Ruby or Python for web dev, the rest of us might as well stay on PHP too. There's a large investment in the existing infrastructure that often can't be ignored no matter how compelling the features of the new platform are.

The differences between Py2 and Py3 are not substantial enough that it "divides the community in half" in any tangible fashion. Yes, it's possible new software may not take Py2 compatibility into account and you'll need to switch to Py3 to get new stuff, but that's been the plan all along, right?

183 of the 200 most-used libraries are Py3 compatible now, including major systems like SciPy and Django. It's not 2009 anymore, and we should stop talking like it is. Py3 is the way forward and the Python Foundation needs to quickly and firmly deconstruct any sign of flapping on that front if they hope to convert the remaining Py2 holdouts.

I believe it would be very damaging for the community if Python backpedaled and said "OK, we know you just spent 7 years investing in the Py3 platform and converting your apps and libs to work on it, but visible projects like Mercurial still don't like it so we decided it wasn't worth it anymore." People don't take time out of their day to make their voices heard unless they're already discontented about something. We shouldn't assume that everyone hates Py3 just because Py3 users are going about their business quietly.

Don't get me wrong - I'm totally on board with Py3, I'm just saying for newcomers to the language, they hear, "Python 2, if you want to be compatible" and "Python 3, it's the future" - and sometimes, they just throw up their hands and say - "I have no idea what version I should use."

I think things will get better when operating systems start to default to Python3.

Python developers would almost all upgrade in a single minute for 30%+ better performance.

It's interesting that performance wasn't a topic at this rump session as reported; I moved over to Go about a year ago, and while I miss Python's expressivity at least once a week, I'm just not willing to slow down all my programs by 5x.

On the other hand, if Python could double in speed, I'd likely try to rework it into our workflows. Well, maybe. I really dig Go.

Have you tried Nim[rod]? Seems to be pythonesque in almost every way, except speed, in which it is goesque.

Aside from having whitespace-sensitive syntax, Nim isn't Python-esque at all. Nim very much encourages TIMTOWTDI over Python's there-should-be-preferably-one-way-to-do-it. Nim also has extremely flexible syntax where Python is rigid. See Nim's pervasive use of metaprogramming via macros, or its inclusion of user-defined symbolic operators (as per Haskell and Scala), or the fact that it considers `foo` and `Foo` and `FOO` and `f_O__o` to be equivalent identifiers, or its inclusion of UFCS.

Nim is a neat language, but comparisons to Python solely because it has semantic indentation are completely shallow (especially so when you consider that Guido doesn't even think that whitespace sensitivity is an important feature).

I agree that idiomatic Nim (if there is such a thing) is very different than idiomatic Python.

What I meant (and did not express clearly) is that you can write mostly pythonesque code in Nim, and it will perform as well as Go. You don't have to modify the syntax, use macros, define symbolic operators, operational transforms, etc any more than you have to use metaclasses in Python.

At it's core, if you write "typed python" code in Nim, it will work, be readable, and perform well.

I haven't, but I have tracked Nim since announcement. It might be a great language, but I don't think it will ever be appropriate for my use case, which is reasonably performant mid-size codebases that junior/journeyman developers can be productive in almost immediately.

If you read the "Why Nim" posts, they are highly salient points for a 'journeyman' or better developer that wishes to be highly productive on a small project; I read the list of 'better than Go' stuff with that hat on, and it's very appealing.

But, there's too much rope to hang oneself in Nim for my use case. Getting a language stripped down just enough that team productivity over years is maximized is a very, very hard thing to do. I think the go folks have the best take on it right now, and it's run by courteous and responsive grownups who do what they say they'll do. That's a total win in my book.

Add in go fmt, a very good (not without warts) module import system, reasonable testing and a multi processor programming model that's easy to reason about, and it is a very, very good solution for my needs.

If you need better performance you'd be looking at PyPy instead of CPython. If Python 3 runs faster on PyPy as well that could make a difference.

>Python developers would almost all upgrade in a single minute for 30%+ better performance.

But strangely they don't - PyPy has hardly gained traction (albeit the python2->python3 switch didn't help) and looking at the benchmarks that's more like 5-7x performance.

I suspect that most often, in places where performance matters enough in a way that would warranted refactoring a code-base from cpython to PyPy, they already rewrote the botte-neck parts in C anyways.

pypy is great (we use it at work for a few services), but it has slow startup/jit warm up, and uses crazy lots of ram. Once it warms up though, it is pretty darn fast (for python).

A lot of the problem is that PyPy isn't completely compatible with C extensions.

I want to use PyPy, and I have used it for a biggish server-based project I've been a part of.

The issue I've usually had with it is just small things – maybe an module we depend on doesn't work with PyPy yet, maybe PyPy3 isn't 3.3-compatible yet and can't use "yield from", which we use in our code or which our libraries use.

Dealing with those sort of small issues (and/or waiting for PyPy to fix them), worrying about whether the project I'm currently working on can use PyPy or if I need to use CPython instead, etc makes me stop worrying about trying to use it after a while. In my dev environment at least.

I do really love what the PyPy guys are doing though, and when it works it works damn well. Building it from source is also super pretty.

I like PyPy, it's great work. But, it doesn't always work. Our last project that used python seriously used tornado and occasionaly numpy. Getting it all glued together was lots and lots of frustration, confusion about versions, and so on. PyPy just isn't mainline python. Which is really what the 2 vs 3 debate is about, in the end. If you have a 'batteries included' language, making breaking changes could take a long time.

Nothing in that article matters to me and I use Python every single day. I guess if you know too much about a thing it's easy to lose sight of what "normal" users care about. So here's my list:

1. Speed

2. Language warts (e.g. del, __init__, import *, while: else:)

3. Lack of a modern UI toolkit

4. No native support in Android, iOS, or Browsers worth mentioning.

Right now Python is the perfect prototyping, glue, and modest workload language.

You can make it better for heavy loads by fixing 1. You can make it even more attractive to novice programmers by fixing 2.

But you really have to get to 3 or 4 before it becomes truly attractive and you get a mass adoption.

PyQt I think satisfies "3"? Though it's not builtin.

I always found it weird that python functions quite happily tell you that you're missing a colon, but can't 'just run' without it.

Excluding one-liner syntax, why does Python actually need a colon to define a function, given it's goal of being free of unnecessary syntactic elements? I'm only an intermediate pythonista, but I'd be interested to know if there was a particular point to the colon.

It doesn't, it just is easier to scan. There isn't a goal of being completely free of syntactic elements, semicolons went because they were excessive and noisey, colons seem, at least to me (comparing with say, coffeescript) to make it less noisey.

Thanks for the info, all.

It's because there's a rule that every time you are going to start indenting more, there's a colon first. Since whitespace is rather invisible, it's a concession they make to make scoping more clear.

A few "unnecessary" syntactic elements help find errors at "compile time". In Javascript for example, automatic semicolon insertion can lead to very subtle errors.

Don't count out Kivy for a modern UI toolkit, but speed and no requirement to include a runtime for distribution will not come for Python without a different approach to execution.

regarding the speed issue: i looked at python 2.7 - here the interpreter is creating a dictionary for each function frame, now local variables are looked up by name of variable in this dictionary ! (globals have their separate namespace dictionary)

(i made a tracing tool that traces a python program and prints out all accessed variables - it actually makes use of this by-name-lookup-feature http://mosermichael.github.io/cstuff/all/projects/2015/02/24... )

i think that's quite wasteful, fixing variable access so that it is by some internal index and not by name would probably be a big improvement, even without changing the global interpreter lock.

Closures might make this difficult, since a variable might be referenced by name before it's defined in any scope.

Variable access already works by index, not by name. Normally, LOAD_FAST and STORE_FAST are used.

what's wrong with qt as a modern ui toolkit? python seems to be one the best supported languages as far as qt bindings go.

Nothing per se. However the situation with two competing python bindings and no official out of the box support or blessing of either makes it non-obvious as to what the 'correct' way to write and ship GUI apps in python is.

If I just want to wrap a simple GUI around some functions then going down the qt route is currently far to 'heavy'.

Have you ever tried to deploy a native looking Python3 qt app on both Mac and Windows? I am not aware of anyone who lived to tell the tale. If you can do without a native look (menus, dock icons, app packaging), i.e. you just want to have some academic lab tool, then qt is good enough, IMHO. But no comparison with native UI development on Win or Mac.

no, linux only. i'll admit that i wasn't taking native look into account, but then again, no default gui toolkit would be likely to have a native l&f anyway - you'd want to use the platform-native bindings for each platform separately, if that was a strong concern.

> Windows has the CreateFiber() API that creates "fibers", which act like threads, but use "cooperative multitasking". For POSIX, using a combination of setjmp(), longjmp(), sigaltstack(), and some signal (e.g. SIGUSR2) will provide coroutine support though it is "pretty awful". While it is "horrible", it does actually work.

I do this, and it works perfectly well. Here's a full implementation demonstrating this approach: https://gitlab.com/higan/higan/blob/master/libco/sjlj.c

It's been successfully used on x86, amd64, ppc32, ppc64, mips, arm and sparc in several projects.

However, it still has a good bit of overhead. But you can implement this concept absolutely trivially on any platform for maximum speed. All you need to do is save the non-volatile registers, swap the stack pointer, restore the non-volatile registers from the swapped-in stack, and return from the function. If you haven't realized, one function can reciprocally save and restore these contexts. Here's an x86 implementation, for example:

    co_swap: ;ecx = new thread, edx = old thread
    mov [edx],esp
    mov esp,[ecx]
    pop eax  ;faster than ret (CPU begins caching new opcodes here)
    mov [edx+4],ebp  ;much faster than push/pop on AMD CPUs
    mov [edx+8],esi
    mov [edx+12],edi
    mov [edx+16],ebx
    mov ebp,[ecx+4]
    mov esi,[ecx+8]
    mov edi,[ecx+12]
    mov ebx,[ecx+16]
    jmp eax

This turns out to be several times faster than abusing setjmp/longjmp.

I turned this into the simplest possible library called libco (public domain or ISC, whichever you prefer.) The entire API is four functions, taking 0-2 arguments each: create, delete, active, switch.

The work's already been done for several processors. Plus there's backends for the setjmp trick, Windows Fibers and even the slow-as-snails makecontext.

If Python does decide to go this route, I'd certainly appreciate if the devs could be directed at libco for consideration. It'd save them a lot of trouble making these, and it'd get us some much-needed notoriety so that we could produce more backends and finally have a definitive cothreading library.

libco seems pretty interesting. Do you alswo have some code to show which uses it? Just to get an idea of what you use it for typically. Have you used it with Python alreday?

The code that uses it is pretty complex (go up two folders in my gitlab link for that), so here's a minimal example of the full API:

  cothread_t a, b;  //cothread_t is a typedef to void*
  int main() {
    a = co_active();  //get a handle to the main thread
    b = co_create(entrypoint, stacksize_in_bytes);
    printf("a");
    co_switch(b);
    printf("c");
    co_delete(b);  //no need to delete a
  }
  void entrypoint() {
    printf("b");
    co_switch(a);
  }
  //output: "abc"

We've only been using it in emulators so far. It's in MESS, higan, twoMbit and a few others.

FYI, greenlet already includes stack switching code, so it would make more sense to use that.

https://github.com/python-greenlet/greenlet/tree/master/plat...

Ah, neat. Looks like they have even more backends than me. I've never even heard of S390 before. Okay, nevermind then. Thanks for the link.

That's the downside of these libraries, there's as many libraries as there are people wanting to do this. I've been hoping for a standard to emerge, even if it's not mine. This looks a bit too tied into Python to be general purpose, though.

Honestly I thought that we where done with 2 vs. 3. Every new project we start is Python 3 and I don't hear anyone in the office preferring Python 2.

Dealing with Scandinavian languages having the Python 3 Unicode support is the killer feature in Python 3, it just make everything so much easier. In terms of performance it's fine and library support is no longer an issue (for us at least), everything we use just works.

Yeah, perhaps your experience in your office is not generally representative?

In PIP, Python 3 has like 5% uptake...

What do you mean with "In PIP"? Are only 5% of the pip downloads from Python 3? That would make sense seeing as pip is included in Python 3.4, so there no need to download it.

If you mean that only 5% of the packages on pypi.python.org is Python 3, I would say that sound a bit low. Also unimportant for most, if it's just the correct 5%. There's also a ton of old cruft on pypi.python.org that would count against Python 3, but not really be important to anyone.

Quick check, there are 7633 Python 2 in the package index (https://pypi.python.org/pypi?:action=browse&c=527) and 8949 for Python 3 (https://pypi.python.org/pypi?:action=browse&c=533)

Neither downloads of PIP, nor packages in PyPI.

I was talking about versions of Python used to download packages off of PyPI (with pip et al).

As of 2014/1, those accounted for < 5% of the downloads, with 2.X being the rest:

https://alexgaynor.net/2014/jan/03/pypi-download-statistics/

Aaah okay, makes sense. I would like to see new stats though. It wasn't until Python 3.3 we started switching, before to many 3rd party libraries where missing.

Still I don't see much reason to not switch at this point. I don't believe that most developers would be missing anything.

Small note: We run our own pypi server for a large number of packages, so our Python 3 installations aren't counted for everything, but the same should be true for many Python 2 setups.

This post seems to conflate language and implementation. IMO, Python 3 the language has tons of improvements and no regressions. The grief on the internet about Python 3 makes me seriously wonder how many people who don't like Python 3 have actually tried it yet. (There are legitimate critiques of Python 3, but they're few on the ground and none are presented here.)

The suggestions in this post are mostly changes to the implementation (i.e., make it go faster), not the language itself. While CPython 2.7 and CPython 3.4 (implementations) surely have interesting implementational differences that don't boil down to just language changes, I'm not aware of them.

The idea that language and its implementation can be separated is only theoretical. In practice, the adoption of a certain language is highly influenced by its implementation (and ecosystem).

The language improvements are nice. I know this because I started with Python 3 then switched to Python 2 and missed some of the goodies now and then. But the language improvements are not enough to overcome the breakage of backwards compatibility. Only a vastly improved implementation (that is not backported to Python 2.7) will.

Python is notable among its peers in that it has a well-defined spec that is not just a list of bugs in the reference interpreter. It is thus important to note that some of the non-reference interpreters do already have the properties requested in TFA, but for whatever reason they haven't supplanted the reference implementation. (I suspect that the ability to drop down into C/C++, link against preexisting libs, or use tools like Cython and taking advantage of the compiler's optimizer, is actually more important than, say, the average PyPy speed boost. Certainly it is for me.)

I find it somewhat unfortunate that a LWN subscriber link is being abused like that. I don't think such a link should be shared on a widely accessible platform like Hacker News. I find LWN articles to always be of great quality, and the subscription cost is definitely worth it if you can afford it. Also, "subscriber-only" content becomes publicly available after only a week. There is consequently no reason to share a subscriber link like this on Hacker News. The discussion could have waited a week.

mm... they were shared by Guido himself on Twitter, so I am wondering how "private" those links are.

Jonathon Corbet, the editor, encourages occasional subscriber links posted to Hacker News: https://news.ycombinator.com/item?id=5688938

I wasn't aware of his stance on this specific matter. I guess I was needlessly worried, then.

https://lwn.net/op/FAQ.lwn#slinks

"Where is it appropriate to post a subscriber link?

Almost anywhere. Private mail, messages to project mailing lists, and blog entries are all appropriate. As long as people do not use subscriber links as a way to defeat our attempts to gain subscribers, we are happy to see them shared."

This year I've been spending a lot of time learning Python as I can tell, even being a longtime Rubyist, that Python is the better language for teaching and for general purpose usage...and since Python 3 is already pretty mature, I figured I should just pretend that Python 2 doesn't exist...

Well, after a month of studying, using, and teaching the language...all I can say is, the conflict between 2 and 3 definitely lives up to the hype :)...Most of the changes make sense to me, and even as a "do-whatever-you-feel-like-aesthetically" Rubyist, I appreciate what Guido did/attempted to do in the clean-up. But things like lambda...there obviously was no easy answer...I love lambdas, but it's so functionally limited and awkward in Python that I also see Guido's point about just removing it from the language (ultimately, he gave up on that)...

But what about the built-in reduce()? Again, it's another function that I instinctively reach for as a Rubyist...and yet it's so awkward in Python that, again, like lambda, maybe it should die? But in this case, Guido halfway-won, and now it's been pushed into the functools package. Mmmkay. And so it is with so many of the 2 to 3 changes at the interface level...as a newbie, it's just mostly amusing since I have no legacy code to port over, but I definitely understand the strife.

But the conflict is still hard to avoid as a newbie...many of the most used guides (LPTHW, Codecademy's Python track) are just done in Python 2...LPTHW says right up front to stay the fuck away from 3.x. I don't think Codecademy even bothers to mention what version they're teaching...obviously, beginners don't need to get into the version wars, but as soon as they get past Codecademy and start Googling around, they're going to be in for some surprises.

Hell, the act of Googling is itself affected by the version-wars...everytime I google for commands/subsections in the official Python docs, the version 2.x docs are always at top. Sometimes the 3.x docs don't even show up. At least I know that there's a 3.x and how to manually switch to those docs...imagine all the novices who are also Googling for references...it's not hard to think that the cycle of 2.x indoctrination is propped up by the simple fact that 2.x docs/help are always at the top of the Google results.

I really wish there was more thought put into allowing code to be compatible with both Python 2 and 3. The fact that the six library (which supports writing Python 2 and 3 compatible code) isn't in the core Python library is a big fail IMHO.

I've written a few things that are meant to support Python 2 and 3 from the same codebase and it was a bit of a nightmare finding all of the gotchas. Stupid little stuff like changing dict.iteritems() and replacing it with dict.items() (which still exists in Python 2 but doesn't do what you expect!) in 3 are a big pain to deal with when writing code that has to work with python 2 and 3.

This page has a lot of good advice, but the fact it's so long is just a testament to how painful the 2 to 3 transition is for people: http://python-future.org/compatible_idioms.html

People are always going to whine about any change. The Python community should have been more adamant about dropping 2.x updates. Spreading out the pain doesn't make it easier...more code is still being written in 2.x. Ugh, I think I said this 5 years ago. Imagine the tens of millions of new 2.x code that's been written in the last 5 years. Oh well, good luck.

It's challenging. On the one hand, you don't want to break what is working (Python 2.x), so you'd like to support it so that community can continue to be productive. On the other hand, you don't want your braintrust split across two different versions, and you don't want the confusion that comes from having two major versions.

For example, the (Epic) Learning Python 5th Edition by Mark Lutz, had to be written for two simultaneous audiences, 2.7 and 3.3. There are all these disclaimers noting where things are different between the two versions, and there is a lot of cognitive overload trying to read a book can't assume you are on Python3.

In the best case world - people would treat this like a y2k situation, and realize that if they didn't get with the program, and migrate over to Python3, they'll end up like Perl, with some other newcomer that isn't so bipolar charging forward and winning mindshare.

Unfortunately, their are a lot of Python2 people who are happy with Python2, and we're in the situation we have today.

There are a lot of Windows users who are happy with XP too. I remember trying to explain to people 20 years ago why HDTV was better than what they had. Most people were happy with 480 lines of resolution, or thought they were. It's human nature. By being accommodating, you've made the problem harder.

Apple is the only company that says screw legacy. Of course users complain but they just grumble and know to accept it. Apple and their users, as a whole, benefit greatly by "getting all the wood behind one arrow" strategy.

There's lots of grumbling about Swift but I bet we get a very large developer adoption rate within 24 months.

But Swift is an actual example of something with no legacy to worry about. Instead of being Objective C++2015, it's an entirely new language. And, more importantly, Apple can give guidance that people have to follow. As Swift matures, regresses, and has the bugs worked out of it, and library support is fully developed, Apple can arbitrarily identify a point at which Objective C just has to die, and the developers (if they want to develop applications for the App Store, which is 99% of them), will just have to do so.

The nice thing about Windows XP, is that by and large, a Windows XP user could use Windows 7 with next to zero training. Or, in fact, zero training.

This is why the first version of Windows 8 pissed so many people off - removing the Start Button all of a sudden made the Operating System unusable to a huge portion of the Windows user base.

I understand they've brought back the start button.

Swift has legacy builtin. It is forced to use Objective-C's OO mechanisms.

I've ported a 100KLOC project. Took me a week or so to do it. But it took me months to iron out the last bugs. Had no problem with libraries (so I just had to support the language)

Fact is 2to3 is nice but it doesn't give you any guarantee about its code coverage. So you go almost just as fast working by hand.

But the lack of guarantees, that makes working in production very dangerous.

Tried to support 2 and 3 at the same time, but that's just too exhausting and error prone (one has to check in both python2 and python3)

Projects with 100% test coverage don't exist, spare time project have even less test coverage.

For me unicode was the driver to change. And it paid off. And I think that's the only P3 feature that actually improves expressivity (now I can clearly express unicode strings). The yield stuf, etc. is fine but nothing /that/ impressive.

For performance, forget PyPy, a 5x/7x improvements is not enough : you still can't write high perf code with that. If PyPy was 50x faster than CPython, that would be something.

So basically, after a lot of efforts I'd say write Python3 code because it helps Python or because you use unicode. Any other reason seems a bit weak to me. And that's sad, I've bet on Python 4 years ago and it didn't evolve much (it surely became very stable, which is not funny but damn useful!).

I guess the point of the 2-3 war is precisely that : 2 and 3 are different but not different enough... So people have hard time to make a choice.

The article mentions stackless Python. My understanding is Guido didn't want to merge it because it would break backwards compatibility with extensions.

That was a long time ago when Python's userbase was much smaller. I wonder if Python be in better shape today had they merged stackless back then.

I think Python is in great shape. On Unix it is the dominant scripting language.

https://www.google.com/trends/explore#q=Python%20programming...

OK... so from a _practical_ perspective, can someone tell me why I should move to python 3? None of the arguments presented to switch are strong enough :-/

A practical perspective depends very much on your specific use case. Server programming? Scripting? Scientific computing? Games? Tooling?

I am honestly curious if there is a good argument for any of the examples you mentioned.

Scripting and Tooling becomes more robust wrt Unicode. You write a short script which works on some ASCII data. Later that ASCII suddenly becomes utf8. With Python 2 you could easily run into weird issues, when there is a unicode object instead of the expected str. With Python 3 decode is more explicit and robust.

Scripting and Tooling are the major uses of Python for me, so I like Python 3.

I always work with utf-8 strings that get passed around from native C code to python. So far I have never had an issue. The argument for me for unicode support seems weak. Is there anything else beyond string encode/decode/manipulation? You mentioned tooling... like what?

It's not about a need for new features. It's about basic quality control. I recently ported a medium-sized production system (the back end of "sitetruth.com") from Python 2 to Python 3. It took about a month. Not because of the language changes, but because several major third-party packages were discontinued for Python 3, and their replacements were buggy.

Specifically:

- Python 3 forces you to use CPickle instead of the Python version of Pickle. In some multi-thread/multiprocess situations, CPickle has some memory allocation error, Python's memory becomes corrupted, and things go downhill to a crash. The Python version is fine. I submitted a bug report, but nothing will happen unless I come up with a simple test case, which is hard. Meanwhile I found out how to use the Python version, which works, despite Python 3, and am using that.

- PyMySQL (a "drop in replacement" for MySQLdb) originally didn't implement LOAD DATA LOCAL. When it was implemented, it wasn't tested for large data loads. I kept getting random database disconnects, until I figured out that it was trying to send the entire bulk data load as one 16MB MySQL connection packet. This only works if you configure insanely big buffers in your MySQL server. There's no reason to send a packet that big; LOAD DATA LOCAL will use multiple packets when necessary. It was just a lame default.

- HTML parsing uses different packages under Python 3. The HTML5parser/BS4 combination blows up on some bad HTML, usually involving misplaced items that belong in the HEAD section. The HTML5 parser, obeying the HTML5 spec for tolerating bad HTML, tries to add to the tree being produced at points other than after the last item. BS4 is buggy in that area. I wrote a function to check and fix defective BS4 trees, came up with a simple test case, and submitted a bug report. I have a workaround for now.

- Python 3 finally has TLS support in SSL. (That's also been backported to Python 2.7.9). SSL cert checking is now on by default. It doesn't work for certain sites, including "verisign.com". This is because of a complicated interaction between a cross-signed root certificate Verisign created, a feature of OpenSSL, and how the Python "ssl" package calls OpenSSL. It took weeks of work to get that fixed. Because it's a core Python package, it will remain broken until the next release of Python, 3.5, whenever that happens.

- Running FCGI/WSGI programs from Apache requires a different package than with Python 2. There are 11 different packages and versions of packages for doing this. The Python documentation recommended one that hadn't been updated since 2007, and its SVN repository was gone. There are six forks of it on Github, three of them abandoned. I finally found a derivative version from which much of the unnecessary stuff had been stripped out, and it worked.

- Python's installer program, "pip3", doesn't know which packages work under Python 3, and tried to install a version of one of them that only worked with Python 2.5-2.6. You have to know to install "dnspython3", not "dnspython", for example.

These are all bugs that should have been found by now, and would have if Python 3 had a more substantial user base. We're six years into Python 3. I shouldn't be finding beta-version bugs like these at this late date.

Python's Little Tin God's position on third-party library problems is that it's not Python's problem. His fanboys follow along. (Comment on comp.lang.python: "You have found yet another poorly-maintained package which is not at all the responsibility of Python 3. Why are you discussing it as if Python 3 is at fault?") As a result, PyPi (Python's third-party package list) has no quality control. Perl's CPAN has reviews, testing, and hosts the actual packages. Most of Go's key packages are well-exercised within Google and maintained there. PyPi is just a link farm.

That's why Python 3 isn't getting used. It's not a need for new features. It's that Python 3 doesn't work out of the box. Its supporters are in heavy denial about this.

Re: FCGI/WSGI with Apache, after a lot of frustrating trial & error I found https://code.google.com/p/modwsgi/ to work the best.

If you can dump Apache, uwsgi and nginx is (imho) a much nicer way to host python apps.

NumPy/SciPy have been the blessing and the curse of Python.

You've probably read this over and over, but for me there were 2 reasons why Python 3 wasn't a top priority: 1) lack of support in some libraries --- this got a lot better; 2) lack of support on Google App Engine.

On the other hand, I didn't find the new features in Python 3 appealing enough to make me fight the above drawbacks.

Last but not least, while I use most of the tools I've developed in Python on a daily basis, they are just that: tools meant to make my life nicer.

If the Perl6 rewrite is validated by it having a larger user base than Python 3 in 5 years' time, part of me that has died will be reborn. Unlikely, though.

What Perl6 rewrite? Perl6 is a specification. Are you referring to various compilers that have been implemented over the years as rewrites?

What Perl6 rewrite?

The rewrite of the language.

It was decided that if you need to break backwards compatibility anyway, you might as well do so in a big way. In contrast, Python3 breaks backwards compatibility as well, but only offers moderate improvements.

Maybe it would have been great to just say "python3 will be based on pypy"..

Definitely could use a another killer feature or two.

I wonder how much of the Python 2.x stickiness is due to OSX still having 2.x as the default version. I mean, even Yosemite still defaults to 2.7!

I imagine very little. If that were the case people would be using old versions of Ruby as well as Python.

Just make it faster and everybody will move. No need for fancy new stuff... I'm eager.

"Just make it faster and everybody will move" is demonstrably false. See PyPy.

Of course, "no regressions" is another condition. With PyPy lots of C extensions break.

PyPy is another implementation. Might as well be another planet. Extensions break or are slower. Different toolkits. DB Drivers not working, etc.

I agree. Introduce a JIT in CPython and optimize the hell out of it like Javascript does.

The way to make Python 3 more attractive is to backport the best bits to Python 2.8...

Officially there are no plans for 2.8. Version 2.7 is the last one (but they'll keep supporting it with bug fixes AFAIK, hence why it's up to 2.7.9).

You can lead the python horse to water but you cannot make it drink... :(

My point is that there ought be a 2.8, and it ought to keep growing.

I sat through a similar talk about (lack of) 3.x adoption at PyConSwe last year.

Who did the people who invented 3.x think their customers were? What did they think people wanted and needed?

I want Python 2.8.

This is Python 2.7, but is based on PyPy (so it's waaay faster), and has requests + gevent + lxml built-in, and adds some small niceties from 3.x (yield from, a, b* = foo()).

Feel free to implement that in Pypy right now?

Haven't hacked on PyPy before. Would be interesting to do someday :)

"For another, conventional wisdom holds that reference counting and "pure garbage collection" (his term for mark and sweep) are roughly equivalent performance-wise, but the performance impact wouldn't be known until after the change was made, which might make it a hard sell."

AFAIK there exists RC GC's which are performance equivalent to MarkSweep, but these aren't super common out of academia? What is the state of GC performance in python currently?

Python should institute "migration" notices on all of the 2.x documentation, making it clear that the intent of the community should be to move to Python 3. I know they plan to support Python 2 for a while longer, but no reason they can't make the statement that everyone should be using the latest versions available.

I noticed a while back that what really made me want to use python 3 is new features. Several script I use currently has a bunch of try-except that looks for functionality, and then monkey patch some python 3 feature into python 2. This will only get worse until library support allows me to switch.

I think that Python3 should had got only single big braking change, like unicode story, for example. Then python4 could get rid of print statement etc., one breaking change at a time, with a tempo suitable to overall migration.

These articles that assert low take up of Python never quantify in any scientific way why they think there is low take up of Python 3.

It's opinion presented as fact that Python 3 has low take up.

Without tangible proof I call bullshit.

Quick impression from the article: CPython holds back the larger Python language from its potential. Maybe?

I would like to learn Python to use it for web apps - what version would you recommend now to begin with?

If you're starting now, there's really no reason not to begin with version 3.

The Python 2/3 problem has created a decision point.

Namely, Python3 is sufficiently different for applications supporting distributions.

If you are doing Software-as-a-Service hosted webapps, fine, you can choose your platform, but if you are shipping software, you have to make concious choices about usually supporting what the distros have.

And the Linux distros are inconsistent.

This problem sounds solveable by saying "users, install a newer Python", though this seldom is effective -- and long lifetimes of things such as RHEL 5 (yes, still afield - and some folks have to support version 2) ship versions that are less compatible with Python 3 compatible hacks than newer Pythons.

As a result, this intent to "clean things up", I feel, has massively undercut Python's growth rate. Maybe it's not declining, but there's been what feels to be an inflection point.

I suppose looking at download curves for hundreds of long-standing PyPi projects relative to the growth rates of other systems could provide this is a thing or not.

Anyway, I do love Python. The problem is not the GIL. Most folks who are making web services get by far with a pre-forking webserver (mod_wsgi, etc) and something like celery for backend jobs. multiprocessing is ok enough for some other cases.

It doesn't matter whether Python 3 is attractive so much, and that's what I mean about a decision point - the confusion gave people a chance to shop around, and some people are trying things in other languages now.

For instance, Go seems misapplied - it has a different expressiveness and domain area. I'm writing a fair amount of clojure, which also feels a bit more low level (sometimes, just in places). But I felt compelled to look around.

The crux of the theory is this - changing something singificantly will allow people the opportunity to think is this something they still want to do.

I still believe Python strikes a great balance between expressiveness and readability, and it's surpression of "clever" in programming makes it ideal for a lot of problem domains. And it's kind of old enough that people are going to want to look around.

Still, I begin to feel some of the directions being made in 3 are out of touch, just as the resistance to some more expressive language features (crippled lambdas, I vaguely recall) were that way. This happens when those that write the language don't neccessarily use the language, and the (percieved) BFDL approach of "fixing regrets" I am not sure it looks after the good of the whole, the way the 2->3 transition happened. Those should have been evolved slowly, keeping things compatible, rather than creating what is essentially a new language.

I'm still pleasantly surprised by how widely deployed Python is to the rate at which people talk about it (say, vs Rails), I think a lot of that is because it's NOT complicated, and you don't need to talk about it so much. It's a quiet workhorse.

But I've also started new projects in Python 2 - because I've needed them to work everywhere. Python 3 is almost sort of having the Perl 6 stigma to it in my mind, it's available now, but it's made a compatible break that has shaken trust.

Since Python 2 is essentially the deployed standard, there's no real reason for most apps that must be distro installable to work on hybrid support - until the distros move to a version that makes it easier to be compatible, it's more important to support where the users are than risk possible bugs and implementation trouble. Resources are better spent elsewhere.

Reply on news.ycombinator.com