Hacker News Reader

Comment by tisme | Hacker News Reader

I've done quite a bit of python development. One of the things that really bugs me about python is how they keep breaking older stuff. Other languages have been much more careful about maintaining backwards compatibility and I think that is a big factor in the retention of users.

Having to re-do any part of your code from one release of a language to another became a real deal breaker for me.

For an interpreted language that problem is even worse because you don't know you have a problem until that bit of code gets hit.

View on HN

Be careful what you wish for. The extreme alternative is Java, whose slavish obsession with backwards compatibility has effectively crippled the language. There's enough cruft in the standard library to keep newbies guessing for years, plus fundamental language design failures like type erasure and "beans".

At some point you need to burn bridges in order to move forward. Doing this frequently destroys the community. Never doing it also destroys the community, it just takes longer.

Personally, I think Python is managing pretty well. Yes, it's occasionally painful, but it's less painful than stagnation.

This exact point is the basis for semantic versioning. Major version changes, e.g. 2.x.x to 3.x.x is expected to break backwards compatibility for the sake of progress. If you are happy with 2.x.x. stick with it, just don't expect any new features, just maintenance in the language and libraries built around it.

http://semver.org

Well said.

I think in Python's case there was really no compelling reason to burn the bridges between 2 and 3. Burning the bridges should be done when something will be significantly improved.

Python's Unicode support was significantly improved between 2 and 3.

I write a fair amount of Python 3 and I don't agree at all.

If your concern is that code written for 2.x won't work in the future, it will be some years until 2.7 is no longer getting security fixes etc. The mere existence of 3.x doesn't prevent you from using 2.7.

I don't think C# ever broke backwards compatibility and they have no problems adding plenty of nice features every release.

One of the things that really bugs me about python is how they keep breaking older stuff.

That's a fair point, and it applies to a lot more than just programming languages. Operating systems and browsers also come to mind, for example.

On the other hand, if you insist on maintaining strict backward compatibility indefinitely, you have increasing drag on every useful new feature you want to introduce. You also can't remove edge cases that should ideally never have been there, even if they make it easy to introduce a bug.

In programming languages, this is the C++ effect. Building on the familiar foundation of C was a good decision by Stroustrup in the early days, and I'm sure it contributed greatly to C++'s success. On the other hand, today I believe that C++ is holding back large parts of the programming community, by being good enough in its niche that huge numbers of projects stick with it, yet lacking the expressive power, broad standard library functionality, and clean syntax/semantics that we take for granted in numerous modern programming languages.

In general, the goals of stability and progress are always going to be in conflict for any platform-like software. Such software essentially defines a standard for others to program against, and the entire point of standards is to create stability and common ground, but sometimes old standards don't adapt well to incorporate new ideas.

I suspect the best we can ever do is restrict major changes, which in practice means those where the old code cannot be automatically converted to get the same behaviour on the new system, to major releases. Minor changes that can be automatically converted are much less of a problem, as long as the "breaking" version of the platform comes with a simple conversion tool.

To be fair to the Python developers, this is essentially what they've done with the jump from Python 2 to Python 3. There is a tool to deal with converting the trivia, and most of the breaking changes were in the initial jump and acknowledged as such.

There probably is a case for making fewer, if any, breaking changes in minor releases. On the other hand, if you're looking at an estimated period of five years to migrate the bulk of the community from one major version to the next, there is probably a fair case for allowing a few smaller but incompatible changes in minor versions as well, as long as their effects are clear and only within a tightly controlled scope so they don't unduly disrupt everyone they aren't there to help.

You're spot on. My basic expectation is that software that I write today will run 10 years from now, unmodified. Things like this give me 'upgrade fever', the fear that if I upgrade a machine something will break and I won't find out about it until it is too late. So I stick to writing software like that in compiled languages now. That requires a bit more work (ok, sometimes a lot more work) but the increased reliability across upgrades is worth it for me.

Other than that language itself being incompatible with previous releases there is the added burden of having to maintain a whole ecosystem, not unlike many frameworks and their plug-ins.

Many people will write some module or other and will make it available for others to build on, and then an upgrade will break the module. The module creators have since moved on and are no longer supporting their brainchildren.

Fortunately with open source you actually can fix these problems - most of the time - but there is not always time or opportunity to do so.

Backwards compatibility is what made microsoft a dominant market force, I believe that you mess with it at your peril.

It is becoming harder and harder to find backwards compatibility for binary interfaces even in the open source world, where libraries move so fast and finding old versions of libraries is becoming more and more of a pain. If you dynamically linked against something 10 years from now unless you kept all of those libraries around, and their supporting files the program won't run anymore.

Backwards compatibility in Windows definitely made Microsoft a dominant market force, but it has also led to the Win32 API stagnating, and it becoming increasingly more fragile as time goes on. Also in all the old code paths, all those not often taken branches there are various bugs just waiting to be found and to be exploited. Yes it has gotten much better, but still.

If you're talking about breakage between version 2 and 3 that is sort of what the major version number bump implies. I don't do much python work these days, but if stuff is (deliberately) breaking between minor releases I would agree that's not very nice.

Is this a 2x -> 3x gripe, or is there something specific in this release that breaks compatibility?

If anything it should help, what with the u"" syntax being allowed now.

I gave up before this release.

edit: oh my, that's a lot of downvotes for answering a question.

Python deliberately broke stuff in the 3.0 release, in order to get rid of some (minor) cruft, but other than that it is as backwards compatible as anything else.

Complaining about backwards compatibility on the 3.3 release news item makes people think you have some specific issue in mind.

Could you provide an example where a minor Python release broke your code and it was not a bug in Python?

> you don't know you have a problem until that bit of code gets hit

The easiest way around this problem is to have tests and run them before you upgrade your production boxes. It's not that hard to do.

> they keep breaking older stuff.

What are specific APIs or things they keep continuously breaking? Did you have issues with transition from 1 to 2 and now with 2 to 3.

Is there a LINT for Python? It looks like there was a question about them on Stack Overflow, but it was closed. http://stackoverflow.com/questions/5611776/what-are-the-comp...

pylint is probably the most commonly used:

http://www.logilab.org/857

Pylint is the most comprehensive, even if too noisy by default. After editing ~/.pylintrc to disable some overly strict checks it's pretty decent.

Aside from pylint which everyone has already mentioned, there's also pyflakes.

http://pypi.python.org/pypi/pyflakes

The problem is in the type system. Dynamic typing makes it difficult to ever change APIs, because a change could break anything that depends on them. pylint and other such tools help a little bit, but they can only do so much when the language doesn't support static typing.

Another Python feature that makes upgrades hard is monkeypatching, where one piece of code can inject arbitrary code into another piece of code. In many dynamic languages, this is often used to patch the behavior of core classes like String. I don't know how common this is in Python, but I've definitely seen it before. By erasing the distinction between interface and implementation, this makes it difficult to ever change them implementation of anything.

People say that writing more and more unit tests will solve these problems. But guess what? The more unit tests you have of an API, the more unit tests you have to change when that API changes. Unit tests are good and should be written in any language, but they are hardly a substitute for static typing.

The end result of all of this is that dynamically typed languages usually follow a trajectory where there's an initial burst of enthusiasm about some cool syntax, followed by a lot of code being written, and then a gradual descent into a compatibility tarpit, where nothing can be changed because of fear of breaking working code. Only additions can be made, and the language gradually grows uglier and uglier. Dynamically typed languages approximate Bourne shell more and more as time goes on-- a dozen slightly incompatible implementations, ancient quirks that bite hard on newcomers, and a resignation that this is the "best it gets."

Sometimes there's a burst of irrational hope towards the end of a language's lifetime. Perl 6 and Python 3 are good example of this. Developers go into their happy place and forget about the big bad compatibility bear that's been chasing them. But it's just a fantasy-- dynamic languages can't escape from the tarpit in the end, and nobody adopts the new thing.

> Another Python feature that makes upgrades hard is monkeypatching

You can't alter core classes (like str) in Python, and patching classes from other modules is considered a really weird, rare, bad practice.

Monkey patching is a clear violation of the API contract and is expected to break when versions change.

It's really not clear to me how your claims can be backed up. Lack of static typing is a fundamental feature of Python, it's part of the design philosophy. The majority of the language functionality is built in the standard library, where modules and interfaces can be swapped out and deprecated easily. Modules in pypi can have different codebases for different interpreter versions, and programs/projects can declare their dependencies using virtualenv; multiple Python interpreter versions can coexist on the same system. Your claim boils down to saying that static typing allows languages to develop faster and cut down maintenance costs, but I don't think you can conclusively demonstrate either point.

virtualenv is a symptom of the disease, not the cure. People get stuff working with a specific version of all the libraries and of Python, and then they forget about upgrading. And why should they upgrade? It will just break their code in mysterious ways. It would be different if the compiler could tell them about changes in type signatures, but it can't.

In the real world, monkeypatching happens. And you may not always know that your libraries or framework are doing it, either. Ruby on Rails does extensive monkeypatching, even of core classes such as String. I don't think you can patch str in CPython due to implementation issues, but that's not the point.

Ruby's culture is FAR more tolerant of monkeypatching than Python's culture - monkeypatching str, for example, just isn't a thing you would do. So what constitutes 'the real world' differs between the two.

Java is statically typed and it is ugly as hell. It also breaks stuff between major versions.

Implying that a language is doomed to be ugly or hard to evolve because it doesn't use static types is not a logical conclusion.

Besides, Perl 6 is not Perl 5 continued, and never pretended to be. Perl 5 continues to evolve (e.g. Moose) and is not getting uglier. You could argue that it was already ugly when it was created, but it is not growing uglier.

Another Python feature that makes upgrades hard is monkeypatching

Although possible, it's my understanding that monkey patching is greatly frowner upon in the python community. I've been programming python on and off for a decade and can only think one time when I've monkey patched something, and I felt really bad about it. The only library that I know of that uses monkey patching is gevent, and it doesn't do it by default but only if you've explicitly told it to.

Reply on news.ycombinator.com