Hacker News Reader

Oops! I thought 3.3 was already out. I see it's only a release candidate.

Get this into your head - Python has a different development model than C. There is no standards body, or requirements for backwards compatibility. Anyone who gets into Python thinking that there will be full backwards compatibility is fooling themselves.

Those who want backwards compatibility can do so on one of several ways: 1) keep using the old code and don't upgrade, 2) contribute time, effort, and funding to the core developer team in order to implement backwards compatibility, 3) fork and maintain your own version of the source, 4) make your own run-time hooks to convert old-style Python code to the new-style runtime, or 5) create a new version of Python which does have that backwards compatibility.

I know that REXX and a few other languages have made a point of backwards compatibility even as semantics change. They have different goals, and a different funding model than Python. If enough people disagree, then those people can take the existing Python code base and run with it. But that's not happening. (I think there are some unofficial patches with VC2010 support, mostly used for internal company builds.)

There's, what, a few score people working part time in Python code development? Even Guido works on it only half time. How do you expect that small of a team to pull off a clean 3.0 release without making mistakes? The 3.0 release had a new io system, which was a lot slower than 2.x. There were a lot of bugs, like unicode filenames, which had to be figured out, where previously it was the responsibility of each developer to figure out the Unicode issues.

"Can you help explain this one to me? Why is it ... impossible for Python 3 to have just a couple "from __past__ import" features?"

That's an easy one. Lack of resources. The Python code developers did not want to maintain both old-style and new-style class implementations. They used Python 3 as a way to get rid of old-style. As a consequence too, it means that I can stop putting in the silly 'object' in "class Spam(object):", which was the Python 2 way of indicating that I wanted a new-style class. If 5 people said that they were willing to support old-style classes, and showed that they were committed to doing so for many years, then there may have been a different decision.

You can see that even in the PEP 3000, written during the 2.6 days: "But I expect there to be at least one or two new 2.x releases even after 3.0 (final) has been released, probably well into 3.1 or 3.2. This will to some extent depend on community demand for continued 2.x support, acceptance and stability of 3.0, and volunteer stamina." (Emphasis mine.)

So many other things had deeper semantic changes, like dict.keys returning a view rather than a list, that a simple syntax change is trivial. I don't think a "from __past__ import comma_exceptions" would have had much benefit. I can be wrong, but that's exactly the thing that the 2to3 converter could handle, and there are patches to make distutils convert 2.x code automatically and import hooks to do that transformation at run-time. One such is pymolurus.blogspot.com/2012/04/pep-414-like-import-hook-for-python-32.html . It isn't a catch-all solution, but that approach could help some people.

BTW, as far as I can tell, clang does not support K&R C.

"Get this through your head": the question here is about whether Python still has a chance of being successful, not whether they are bad people for what they did. My argument is firmly on the "no": Python is in decline after years of stagnation caused by a fork in their effort and I believe that they could have done some things differently at little effort to have had better chances.

Personally, I don't even have skin in the game anymore: I am now using Clojure, and am piecemeal porting my code over. This is actually an easier time than the brutal experience I had dealing with Python's / operator change (which I adopted ahead of schedule, thinking it would help me: all it did was hurt me as my code no longer could be used on 2.5).

I thereby wonder: what are you defending? Do you think Python is succeeding? That in another couple months we are going to suddenly hear of a massive turnaround? That it will still be relevant in another two years when 3.3 is fully deployed everywhere and most libraries might be Python 3 compatible? What is your overall position?

You asked me a specific question. I said that the reason is the lack of resources. I didn't realize that we were talking about the decline and fall of the Python world.

I remember reading "Decline and Fall of the American Programmer." And "Rise and Resurrection of the American Programmer." It's a special kind of consultant who can get paid both coming and going.

Python is in decline? How do you reckon that? In my work, Python is the de factor high-level language for chemical informatics, molecular modeling, structure visualization, and the like. This is much improved over the last 10 years, and won't be dying in as short a time-span as two years in the future. No, most projects haven't migrated to Python 3. Yes, it will be annoying. So was porting client code from IRIX to Linux. That happens.

Really, you need to plan for it. And it sounds like you didn't plan for the division operator change. The ideas of that change first surfaced during user testing under the 1.x days, with Alice. That change decision was publicized in spring 2001, with PEP 238. The "from __future__ import division" was made available in Python 2.2. The runtime even has the -Q option so you can tell it to print warnings wherever there's a problem. The nice thing about this change is that it's purely a syntax issue which can be handled on a per-module basis, and mixed with modules which weren't compiled this way.

These all worked for 2.5, so what was the problem you experienced?

What I'm "defending" is my sense that Something is Wrong on the Internet. It sounds like you're doing things the wrong way. You expect certain behaviors which were never promised or implied, you don't read the update notices, and you don't use the tools which help you achieve the goals you want. No wonder you get so frustrated with Python! But I don't see what the Python developers could have done better, given the resources they had, and given that they care about goals that do not interest you.

You're going to run into the same problems with Clojure. Looking now, it seems that 1.3 broke code. Some things were removed, there were changes to the contrib namespace, and some third-party modules didn't work. Various places say it was a hassle to deal with. Sound familiar? How are you sure something like that will never happen again?

> I didn't realize that we were talking about the decline and fall of the Python world.

The article is called "I Am Worried About The Future Of Python" and the specific thread I chose to participate in directly described a way that the Python 3 schism was contributing to the concern (WinRT support). To quote the lead comment for this thread: "I definitely feel like Python qua programming language has become a missed opportunity in many ways".

I then provided my experience, as someone who has been developing for over 20 years in numerous languages (even designing them and teaching them in a university setting), of a unique set of challenges and mistakes that I see Python having run across--a perspective that one reader called "a brilliant if unpopular analysis"--and yet I'm now simply being dragged into the same defensive argument that comes up whenever the 2/3 topic is breached: to turn it into a very personal attack against me and my abilities led by highly inflammatory and insulting wording like "get this into your head", quite despite the fact that I am hardly in the minority with regards to these overall opinions.

Regardless, I am hereby so confused as to what conversation you feel you are participating in that I am just going to bow out: thank you, regardless, for taking the time to respond.

(Oh, I will respond to the semi-unrelated Clojure discussion, as I should probably make certain I don't inadvertently cause them problems: I do not use third-party libraries written in Clojure, as there is really no need given that all of Java is accessible; this already stops most of these problems. Also, the way Clojure 1.3 broke code is closer to the way Ruby 1.9 broke code--a change I stated elsewhere in this thread was a non-issue for the Ruby projects I'm involved in--than the way Python 3 broke code, and was in fact even less of an issue than that.)

I deliberately used the inflammatory phrase "in your head" because you used the phrase "What really seems to happen with Python is that people get it in their heads "developers should not be using this old feature"". The 'people' you are talking about there are the Python core developers. Those are specific people, not abstract entities. I figured that since you used that phrase with them, then I would reverse it, when saying that that wasn't the point.

I've been using Python now for 15+ years. I've read articles about bemoaning the lost opportunities of Python for almost as long. It needs static types otherwise all is lost. It needs tail recursion otherwise all it lost. It needs macros, it needs to get rid of explicit "self", it needs blocks, ... and so on. People thought for a while that Python development was proceeding too fast. There was the language moratorium so others could catch up. Now you say it's going too slow.

If you want to give me specific details about how Python is losing in popularity, please do. You haven't. Nothing I've seen tells me that there's anything more than the usual churn of people moving in between languages, of hipsters talking about the language de jour, of industry people insisting on certain trends. But what you've written describing your experiences with Python's 2-3 transition does not mesh with my own experiences and readings of the experiences of others. How did the change in division cause problems? Why weren't the existing mitigation mechanisms good enough for you?

I truly am going to stick to my word and ignore the rest of your commentary about Python: there is a long thread with a lot of context started by an article that is at least attempting to disagree with your premise; the things you are asking me to explain are thereby all "assumed state" for this tiny sub-part of the conversation, so you can take up your issues with someone up-thread who might feel less insulted by the overall direction of the argument you are making.

However, I will defend my wording, as I think that is unrelated: the phrase "once you get it into your head that" (even with that direct "you", rephrased from "people get the idea into their heads", about others) is a statement about the virility of the idea that someone now believes; in contrast, "get this into your head" is a direct command given to a person used to indicate that they are being dense and thereby are unable (or unwilling) to understand the idea in question.

To make this distinction more clear, you can look at something generally considered positive: "once saurik gets it into his head that something should be built, he builds it" or "people get the idea in their heads that things should be built, and then they actually build them" are both fine; "get this into your head: once you determine something should be built, you should actually build it" is both harsh and patronizing: it is an entirely different concept for the sentence, used as an attack.

As someone who started this sub-thread I have to say I read it with pleasure: it is informative and broad, I learned some from it, and I wanted to thank you, saurik, for your input and perspective, even though I don't fully agree. I have nothing to say ad rem, at least nothing that wasn't said already, but I wanted to comment on that unfortunate phrase. As I see it, there was no malicious, "harsh and patronizing" intent behind it; I agree that it was not particularly polite, but I feel it wasn't very brutal an attack either. So please, don't hold it against your adversary (or, God forbid, Python community) or at least please try to not be offended - I'm almost 100% sure it wasn't meant to be offensive in the first place.

Again, very informative discussion, thanks.

To my mind this is a brilliant if unpopular analysis. I would really like to see the ability to import modules "from the past" but it seems extraordinarily difficult to implement that now that 2 and 3 have diverged so much. And one of our problems is exactly what you describe: a large, very-well-tested, 2.X library that we rely on is no longer maintained by anyone. So we'll have to port it ourselves, and surely introduce a bunch of stupid bugs. Grumble.

A saving grace is Python's historical emphasis on unit testing. That makes fore-porting things without horribly breaking them at least plausible. But still painful.

> Please realize before disagreeing: I develop compilers, and I have developed virtual machines. I know what kind of effort it takes to have parameterized grammars and internal transformations.

Please be sure that I totally respect your work. Still, while I agree some of your points are right, I believe you're taking a shortcut here:

> For an example of a community that gets this: C. No one develops in K&R C anymore. However, there is still tons of code out there, even code inside of some core projects you use every day, that was written in K&R C and never needed to get serious modifications made.

The syntax evolved, but behind this syntax K&R C behaves in the exact same way, all the way up to the latest C revision. You can't say the same for old vs new classes, which behave in a not so subtly different way, and you can't say that either for the new str/old str, and indeed you propose a different name, which is perfectly legitimate, as they are different things.

The problem is also a vision of scale. I'd say the designed impact of python 3 breaking changes is not for next year, or even the next two years, but for the next 10-15 years. I think the decision came from the realization that there were retrospectively bad or overlooked design choices, and that they needed to be fixed, and thus cut out, now. The most critical things dropped from python 3 are actually stuff carried over from python 1. Python's design and philosophy is around a form of minimalism. The Zen of Python says:

    Now is better than never.
    Although never is often better than *right* now.

I'd say Python3 breaking changes fall in the first case, except the u'', which was a total fail and fell in the second case.

I'd say from anecdotal experience that every single python project I write today works on both 2.7 and 3.2. I sure wish python 3.2 received 2.6's from __future__ import unicode_literals, which would, along with 3.3, remove the biggest part of my headaches. The real problem, as stated, lies in the dependencies, and there's no denying that although you can compile K&R C today, such code was not written in a vacuum and certainly (recursively) relies on a bunch of libs that may or may not work today anymore (because of the code itself, the plaform, or the compiler evolving). It is also in plain sight today with Ruby, where a significant number of historical gems simply don't work anymore in 1.9.x (worse, some appear to work, and silently corrupt stuff, up to a point where it blows up. I'm looking at you mysql, which silently fails to transcode strings, the very situation Python introduced an "upfront error" breaking change).

The point I want to make is that the situation is simply not as clear cut as some want to make it. The short term cost is that we have a rocky transition and that we won't benefit from the new features right now, while the long term benefit will really be seen 10 years down the road. The question you seem to ask is: will Python still be relevant by then? To which none of us has an answer. If not, Python will have paid the price of its 9th zen statement: practicality beats purity.

Now I will return to the pending migration of three real world ruby 1.8 + rails 2.1 apps to ruby 1.9 + rails 3.2 (and hell was the migration of some other app from python 2.6 to 3.1 so much easier).

> The syntax evolved, but behind this syntax K&R C behaves in the exact same way, all the way up to the latest C revision. You can't say the same for old vs new classes, which behave in a not so subtly different way, and you can't say that either for the new str/old str, ...

A lot of code operates in a regime where it takes strings as input, attempts to find substrings in those strings based on either pure ASCII literals or other strings also given for input, cuts ranges and substrings out of that string, and then generates more strings as output.

This is in the same way that much of that code was designed so it could accept either an str or a unicode, and as far as I can tell that same code should either work quite fine now taking either a bytes or a str; to the extent to which this is not the case, it could easily be made to be the case.

> ... and indeed you propose a different name, which is perfectly legitimate, as they are different things.

FWIW, I didn't "propose" the new name: "bytes" is the new name: that is what the old str type is now called in Python 3, and in fact if you are using Python 2.7 (even Python 2.6) "bytes" and "str" are aliases for each other, as even the people who designed Python believe they are equivalent for the purposes of portable code. Sadly, without keeping around "unicode" (and preferably just deprecating "str" but leaving it as a warning-laden alias for "bytes") that was pointless.

The reason being that the only place you could actually have compatibility between these types was Python 2.6+... except, most "serious Python" has been stuck in Python 2.5 until possibly (I think I see a sea-change) this year. Yes: that is four years during which these transition mechanisms were largely pointless: neither Python 2.6 nor Python 2.7 were actually deployable targets for major and serious libraries.

Personally, I found this surprising: I have been using Python 2.7 since it came out, and I have never had a difficulty upgrading within Python 2.x. However, time and again when I come across a large company using Python and I ask them "what version?" they say "2.5". Meanwhile, both Jython and App Engine were likewise stuck at that level.

That means to "remain relevant" the primary deployment target for most systems needed to be Python 2.5. The idea of supporting Python 3 or even Python 2.6/7 was largely seen as a hopefully-fun diversion at best, and a needless chore at worst.

Thereby, I will argue that what was really needed was actually not Python 2.7: all of the effort put into Python 2.7 with its support for bridging the gap should have been made to Python 3 in reverse. You can't tell me that supporting some of these things as the same time, at least in ways that make the remaining porting discrepancy "realistic", is possible, as Python 2.7 does it: it is obviously possible, as we have an existence proof.

Now, you can totally (and dalke does) tell me they didn't have time to do that to Python 3, but my argument is that all of the effort exerted into Python 2.7 would have been better spent doing the same thing to Python 3, but with the defaults reversed: I believe that the effort would have comparable, and the result would have been to avoid the last four years of hell.

> The short term cost is that we have a rocky transition and that we won't benefit from the new features right now, while the long term benefit will really be seen 10 years down the road. The question you seem to ask is: will Python still be relevant by then?

This "rocky transition", for people who actually use Python to develop libraries or applications, is simply "stagnation".

One of the things I found so weirdly surprising when I finally decided to "bite the bullet and move back to Java-land" was the amazing number of high-quality maintained libraries for relatively new concepts (even ones so new they were "buzzwords"): it was like a kid suddenly being thrown into a candy store.

Whereas I had spent forever, on numerous occasions, trying to piece together semi-working mechanisms out of multiple competing and seemingly now all disowned or disinterested Python libraries (such as for PDF editing), I suddenly had my pick of numerous fully maintained libraries.

This is not the state of the Python ecosystem when I entered, and I argue that the reason this changed is because of the Python 3 split: the result is that the people still sticking with Python during this "rocky transition" are putting themselves at a "practical" disadvantage for the idea of a "pure" language.

> If not, Python will have paid the price of its 9th zen statement: practicality beats purity.

Right: it seems like that tenant was thrown out the window with respect to the changes made in Python 3.

> Now I will return to the pending migration of three real world ruby 1.8 + rails 2.1 apps to ruby 1.9 + rails 3.2 (and hell was the migration of some other app from python 2.6 to 3.1 so much easier).

Is this difficulty caused by Ruby, however, or by Rails? I had had an application that used Sinatra on 1.8, and it took me about an hour to get it fully compatible with Ruby 1.9 (which I did years ago, at the first opportunity).

(Note: my application was handling Unicode data correctly both before and after change: something that would have been noticed immediately as much of the package information I used was in random languages from Russian to Chinese, and I am quite a stickler for having things encoded correctly in my database; yes: I did have to thwack it some on Ruby 1.8 to get it to work correctly, but the result after thwacking I believe required no modifications on Ruby 1.9, a result that I might argue indicates that most people having porting problems are actually fixing bugs, not porting.)