Hacker News Reader

Comment by onphonenow | Hacker News Reader

The python maintainers went out of their way to break things - and were smug about it. That’s what it felt like to the community.

A big part of 3 was Unicode strings. In 2 you could mark a string as Unicode with a u””. This would have been a great way to let libraries and code work with both 2 and 3. They banned this, but you had to still use b””. This was the attitude and just one example.

The list of stuff that got hard / slow was long. Migration path was unnecessarily difficult as was compatible code. Somewhere around 3.4 it was like a light switch flipped and they started being more reasonable. ASCII handling improved (yes, not all internet protocol stuff is Unicode), they began making it easier to target 2/3 etc. but it was horrible to start. I’ve thankfully forgotten some of the details :)

View on HN

> Migration path was unnecessarily difficult as was compatible code.

I feel this take is outright wrong. Python provided it's 2to3 tool[1] which took care of the bulk of the work required to port Python2 code to Python3. The only code that was not supported was eggregious errors in the code that worked by coincidence, such as handling bytes as strings and vice-versa. Porting old code to Python3 is a breeze that consisted of running 2to3, run tests with python3, and if anything breaked then just touch up the code to get it to work. I know it because I personally ported half a dozen projects throughout the years.

I'll go as far as to claim that most problems porting old python code to 3 were either upstream dependencies dragging their feet or internal human/organizational issues.

[1] https://docs.python.org/3/library/2to3.html

This is totally false - code produced by 2to3 no longer ran on 2. So you ended up w a chicken and egg problem. Things like u would allow folks to keep compatibility with 2 while working in 3.

Your comment is a perfect illustration of the issues. Lots of user blaming. No actual solution. As I said, it did start to get massively better at some point. Instead of condescending lectures on org issues they for example began allowing u”” in 3, which did not mess up Unicode handling in 3

> This is totally false - code produced by 2to3 no longer ran on 2.

I don't see where GP made a claim that it would. Why would one expect code written for a newer version, using features that don't exist in an older version, to run under the older version?

This was the issue. Why should a new version make it hard to run old code for no particular reason except to break things.

Folks using u”” were already being careful with Unicode . Supposedly Unicode was important enough for 3 to default to it.

The cost benefit of then destroying code using u”” made no sense to me. It was unnecessary. This was followed by the oh just upgrade thing. I’d love to see the “quick” upgrade guidos employer did. My guess is a total lie that it was easy and it probably took years and VC type money.

It’s not about using new features, it’s about not unnecessarily breaking old or allowing for cross compatibility.

> using features that don't exist in an older version

Except they broke the old way for existing features, required you to use the new way and in most cases you could manually write code so it would still work with both. Meanwhile the only tooling provided by the python 3 crowd screwed over anyone with an existing customer base stuck on python 2, which at least early on should have been a foreseeable problem. Not to forget that anyone stuck on python 2 was literally Satan and projects where outright "shamed" to drop python 2 support.

Given dependency chains the business case for these “quick” switches was often poor as well. Yes, Dropbox had $$ and guido - but that wasn’t everyone and most code was 2 at the time.

Many folks would have been ok updating to 3 compatible approaches if it didn’t blow up their 2 story. That’s what ultimately happened in lots of cases when it became more doable. The rip and replace everything at once was a weird goal

If you published libraries on pypi you were in exactly that nightmare scenario. How do you support users on both versions of python (because let's be real, it took _ten years_ to change so you had to support both major versions) without two totally separate codebases and all the massive headaches that entails. There were some partial solutions like the library six and all kinds of related tricks to write python that was compatible with both 2 and 3 at the same time.

> This is totally false - code produced by 2to3 no longer ran on 2.

That's what "breaking changes" and "backwards incompatible" means.

What exactly is hard to grasp?

> So you ended up w a chicken and egg problem.

You really don't. Your code and your upstream dependencies need to be ported to python3. Once your dependencies are updated, all that's missing is you doing your job.

Python2 has been on the path to deprecation for how long? A decade?

You're fabricating problems where there are none.

> Your comment is a perfect illustration of the issues. Lots of user blaming.

There is no issue. When I had to port projects to Python3, I just ported them. No drama, no hangup. You're pretending there were problems where there were none, and you're throwing a tantrum when this fact is pointed out to you.

There is really no justification for this. It's high time people like you stop making up excuses and start to own up their misconceptions, misjudgements, and mistakes.

Some projects were rewritten from scratch in entirely different tech stacks in less time the likes of you complained they could not update their projects from python2 to python3.

“It's high time people like you stop making up excuses and start to own up their misconceptions, misjudgements, and mistakes.”

What is it about python 3 that brings out this lecture stuff.

I would really like to see the inside of this “easy” change at even the bigger most well resourced places. Google / Facebook / Dropbox etc.

At small places where a program is working and programmer is gone they are not investing in ports.

Anyways - they did eventually get a clue. It’s much easier NOW to handle this, but it wasn’t at the start. Yes, they did put u”” back in. No it did not destroy the world.

> Yes, they did put u”” back in. No it did not destroy the world.

This saga isn't completely over, over here: We still have code running python2 that we don't have the room to upgrade, and libraries that have to work with both python2 and python3. Meanwhile, people are pushing "code consistency" to the point where people outside our team are taking those libraries and running the "black" formatter on them - but "black" doesn't understand python2, so it removes the "u" prefix and breaks the library.

I like black and python 3 - but that’s pretty funny! In fairness black is opinionated on purpose - even if it understood u”” it’d remove it if it’s targeting 3 probably because it tries to reformat to one style. Amazing how long the ripples last though

>What is it about python 3 that brings out this lecture stuff.

I think some people convinced themselves that using "" without the u"" was disenfranchising to people who don't want to use ASCII to represent their language. This transformed a practical argument over syntax into a moral crusade.

Ok, so what happens when several of your dependencies never ported?

This isn't strawmanning - this is literally the case for a codebase I'm working with today.

You should not be using those dependencies today. Looks like they have been unmaintained for a decade and can contain a ton of vulnerabilities.

Though I find it hard to believe that there are some useful dependencies that have not been ported and don't have better alternatives. Typically a "never ported" dependency is never ported because it is deprecated in favor of something better.

We're in a rather niche market. It's geocoding related stuff that only talks to a trusted endpoint.

Does no one work in a business? Does no one work with science instruments purchased more than 5 years ago? Businesses have lots of useful dependencies that have not ported. If you are running a payroll system and it’s working, there is a real aversion to going to a new code base, and the cost benefit is not there for many businesses. You do realize python 2 is downloaded 3 million times per month now? Yes that’s down - but it’s not nothing - probably 300 - 400 thousand plus users easily

> Ok, so what happens when several of your dependencies never ported?

If you're consuming dependencies which were never updated in the past decade then you have more worrying problems to deal with than porting your code to python3.

That’s not true at all - in high compliance industries if the dependency has been validated and is working changing it brings risk.

So why did for example Ruby not have such a painful transition from 1.8 to 1.9/2.0? My theory is that the Ruby people make the migration in many steps, it started even back in at least Ruby 1.8.6 and they did it with a lot of care on how to make the transition as painless as possible while the Python team just broke everything and bet on that tools like 2to3 would just magically solve it.

Right. I think a key miss was underestimating value of code that would work w both 2 and 3. This avoids need to upgrade all at once.

Relax things a tiny bit in 3, back port to 2 via futures, get a library like six going. It could have been a ton easier.

In fairness by 3.5 maybe they’d realized this, but a lot of lecturing was done before. Initially recommendation was to upgrade all dependencies and programs to 3 and not try and make dependencies work w both

I think rubys TDD culture at the time also probably helped. My (purely unsupported) opinion is that the test suites were likely more fleshed out on ruby so its less of a risk to switch. That said, I still saw lots of 1.8.6 installs in the wild for quite some time after 2.0 hit.

> The python maintainers went out of their way to break things - and were smug about it. That’s what it felt like to the community.

The other side of that coin is that the python maintainers went out of their way to push back the Python 2 EOL. They kept on pushing the date back and back again, as per the sunset page:

"We did not want to hurt the people using Python 2. So, in 2008, we announced that we would sunset Python 2 in 2015, and asked people to upgrade before then. Some did, but many did not. So, in 2014, we extended that sunset till 2020."

The drawn out demise of Python 2 was, frankly, painful.

I have no time for whinging snowflakes complaining 12 years (2020-2008) was not enough time to migrate their code to Python 3. Hell, even the original 7 years (2015-2008) should have been long enough for 99.999999% of the community.

If they hadn’t extended there was talk of actually forking python. Basically people needed a 2.x compatible python, and most features being put into 3 could have been backported. So you’d maybe keep u””, keep print, add futures stuff for folks targeting both and then pull in 3 stuff to reduce pressure to move. They were smart not to drop 2 cold.

It's not the other side of the coin, it's a direct consequence.

They had to push back the EOL because they screwed up.

it's a volunteer project. nobody stepped up to maintain 2.x as far as I know

> I have no time for whinging snowflakes complaining 12 years (2020-2008) was not enough time to migrate their code to Python 3. Hell, even the original 7 years (2015-2008) should have been long enough for 99.999999% of the community.

I'm old enough to have read this when it came out...and it changed my view on backwards compatability (from Joel Spolsky of Trello, FogBugz, and StackOverflow fame)

"Code doesn't rust": https://www.joelonsoftware.com/2000/04/06/things-you-should-...

If a company has production Python2 application / service (with hundreds of thousands of LOC), what business value does it bring to migrate it to Python3?

At that point, if you've got to make severe changes, folks might decide to use a language that doesn't impose breaking changes (and business cost) on them. YMMV.

I mean, code does rust in many ways.

Not just talking ASCII was one of the first ways.

Drift in 3rd party library support is another.

Security support of the language and libraries is a massive one.

Simply put as hard as you try to stand still the rest of the world is not going to.

> and were smug about it

I feel like a lot of the people who had this reaction were reacting to being told (correctly) that how they handled strings was broken. A big reason that projects had difficulty upgrading, especially difficulty using the automatic 2->3 upgrade tooling, was because of broken handling of unicode and broken handling of binary strings. I remember reading a quick assessment of a large open-source project that there were thousands of calls to a single string-handling function that would each need to be inspected by hand to determine whether it was correct usage that would be correctly translated by the 2->3 converter or buggy usage that needed to be fixed before running the 2->3 converter, with some people guessing a 50/50 split. The effort was going to be gargantuan, and though the ultimate cause and responsibility was poor design choices in Python 2, the immediate cause was that the code was wrong, and some people didn't appreciate hearing that.

No one really blamed programmers for getting strings wrong in Python 2. It was unreasonably hard and extremely rare for people to get it right, so there was not really any shame in Python 2 programs and libraries being pervasively broken in that way. But some people still felt it was smug and condescending to make reference to this elephant in the room. This put the Python maintainers in a bad position, because they were trying to explain the benefits of upgrading and also the costs and difficulties, both of which were intimately tied to the brokenness of current codebases.

As a user and spectator to the whole drama, therefore, I felt that some maintainers got overly defensive about issues that virtually every codebase suffered from. And some of them took a "so what" attitude towards bugs involving non-Western languages, saying that if they personally didn't care whether their code handled Chinese or Bengali filenames then it wasn't a bug to do random things with them, which sounded just as bad ten years ago as it does now.

> I feel like a lot of the people who had this reaction were reacting to being told (correctly) that how they handled strings was broken

Can you be very precise about what you mean by "broken" or "incorrect" code?

In python2 you can easily mix encoded and unencoded strings. This doesn't actually work, but at worst it'll produce mangled strings. You used to see this sort of thing online quite a lot with webpages having all sorts of weird characters and question marks showing up.

Python3 enforces the difference between encoded and unencoded strings. It forces you to deal correctly with unicode. If your code base was already handling unicode correctly it wasn't much hassle to migrate. If your code base was making a mess of unicode handling --like many at the time were-- you'd run into that headfirst.

There are many codebases in the world that never need to deal with Unicode. Therefore, new language features which deal with Unicode are unwelcome when they replace the existing string API (which was often used to handle raw bytes)

I think this was the right decision for the long run though. What we have now is the right situation where the strings we work with in the language are 'magic' and we don't need to think about representation or encoding or whatever. And then at the periphery we encode/decode based on what the outside world is working on. It has become clear that this is the 'correct' approach for programming languages and is almost universally the approach in new languages. E.g. Rust does something similar (they don't hide that the internal encoding is UTF-8, but it's consistent).

I'm glad that this is what we have for Python now, despite the pain it caused. We'll be using Python for another few decades probably, so we'll reap the benefits.

> eat your vegetables

Nah, I'm good.

The era of "move fast and break things" was disastrous for a lot of things.

12 years is moving fast?

Reply on news.ycombinator.com