Hacker News Reader

Comment by xorvoid | Hacker News Reader

Idk.. the alternative seems like a Python 3 nightmare. People start depending on things and then you’re faced with two choices: (1) break lots of people, (2) hold on to every bit of ugly broken legacy stuff in the name of backwards compat.

At this point, I feel like the entire solution space has been explored between all the different languages and there is not a great solution from any of them. Personally, I’m much happier with Go breaking things than becoming a C/C++ hole of legacy going back like 50+ years.

On the timing and speed (only one version warning before removal): we can debate the right speed until the cows come home. But, Python 3 took longer than a decade to kill Python 2 as they were trying to give people “plenty of time”. In the end (IMHO), people don’t change unless they’re aggressively forced to because there is always something better to do with your time than upgrade software to prevent some possible future hypothetical breakage. That calculus changes when it’s not hypothetical at all.

View on HN

In my opinion it's quite simple.

Upgrade cycle should be one major version per year (may be less, but not more).

N+1 version does not break compatibility with N version, but introduces warnings for things that's going to be removed. Every warning must have very easy and clear way to resolve it, ideally automatic way, if language and tools support it. Developer must not redesign his application to get rid of warning, but rather just make few obvious mechanical moves.

N+2 version removes previously deprecated things and breaks compatibility with N version.

So you're going to spend few hours once a year to migrate to newer dependency and resolve their warnings. If your software was rotting for 10 years, you're going to spend a week to gradually upgrade, resolve warnings, etc. That should be acceptable to everyone. And those who prefer to live in rot, should pay someone to maintain those outdated libraries (or just live with bugs and vulnerabilities, that's their choice).

With programming languages and foundational frameworks it might be worth to be a little bit more conservative and extend that deprecation period to a few years. Just don't keep deprecated things infinitely.

> That should be acceptable to everyone.

I used to build things with frameworks and languages with yearly upkeep requirements.

> And those who prefer to live in rot,

What rot? If my application didn't change I'm not sure why my dependencies should need to.

Regarding golang, I hated go get and the go workspace idea when I first started with go.

Then I got over it, now I try to use modules and I just get annoyed, the whole system is confusing and flaky.

All this change is doing is pushing me further into rust. It has problems too but they tend not to do this kind of hand wringing.

The change does not affect either the Go language or the way that you compile the code or fetch dependencies, what's the problem?

Go is much more stable than Rust, you know how many Rust libs need nighlty to work?

> Go is much more stable than Rust, you know how many Rust libs need nighlty to work?

Rust libs that use nightly do so because they're opting in to new features, not because released features are unstable.

Even then with the releases of const generics, proc macros, and a couple other big-ticket features, fewer and fewer Rust projects require nightly anyway.

How many do? Which ones? I’m curious what you’re still running into.

(This is me being curious about your Rust comment, not implying anything about the relative stability of the two languages)

For example: https://github.com/rust-lang/rustfmt

rustfmt is not a library, and folks don’t depend on it as one. The only nightly features it uses are “hey let me use compiler as a library please,” which is also not, in my experience, common in the ecosystem. Even furthermore, it’s distributed precompiled in your rust installation, so that doesn’t end up giving you a nightly dependency to use it.

Anyway, thanks. Nightly does exist to be used, but our users largely report using stable, so when people say things like this, I am trying to see if there’s any blind spots.

I'd differ on this a bit. Your N+1 scheme is fine with me if you want to use the newest and greatest features, but I think old code / libraries should still work so long as they don't use the latest and greatest features.

I think the compiler should just ship with older versions of the language and different translation units can be compiled with older versions of the compiler. So long as the newer versions can interact with the older ABI (which ideally rarely changes), there's no problem.

In that cast, they'd have to either

* Fix all bugs in old version, or

* Ship known buggy code

Neither seems great. Better to just have archival versions with a "here be dragons" warning.

> Ship known buggy code

Thank god there's no known bugs in released software!

I'm not sure I follow.

Go 1.17 will release with a lot of known bugs. Continuing to ship it a year later with known bugs is not different.

But more deeply, LTS ships buggy versions of the software. That's the point - only the truly dangerous bugs (e.g. compilation-triggered RCE or something) get fixed, everything else stays compatible even if it's otherwise undesirable or suboptimal. Code depends on those bugs.

But Go never updated major version yet?

The timeline seems way too short for a lot of usecases to be honest.

For young projects, where it's kind of expected (trade-off of being an early adopter), or for small projects (which can be easily maintained by a third party), it's acceptable.

But for bigger/core projects this is far more problematic.

Just for comparisons, a lot of Linux distributions will maintain a specific version for ~5 years if not more (Debian, RedHat), and if for smaller components, they can do things like backport patches, or even create the patches themselves, however, for larger ones, they are reliant on upstream providing stable and maintained versions for the life of a specific version of the distribution.

Also, I've seen quite a lot of large and complex projects taking several years to reach production. In these contexts, having to constantly rework components (and re-test them, both individually and as part of the overall system) is not really feasible.

There is no ideal solution, maintaining backward compatibility is costly, and can lead to messy code bases, and it's legitimate to want to avoid that by deprecating API/functionalities. However, downstream (distribution, users of a given library) needs stability and generally cannot afford to rework their solution every 3 months (keep in mind that downstream generally relies on many upstream components, if a lot of these components break at the same time, even in minor ways, this can result in a huge headaches).

IMHO, a more desirable compatibility period, at least for big/core components, would be around 5 years.

The linux distro doesn't keep the head supported for 5 years though, they just support older versions for longer. Ergo, what you're basically arguing for is an LTS version of golang.

There's definitely also a tradeoff. If you make the timeline too long then people will just assume that they can use the old version forever and get mad when that doesn't work out. Say because the project that needs "several years to reach production" actually ends up being used for 5+ years

This can also backfire - no one updates Go because of breaking changes and now you've created more problems. Simply miss one or two upgrades and it cascades into an uphill battle. Code base is stuck at N-8 and while version N's popularity plummets, libraries are in disarray.

I know with C, I can rely on it for 45 years. There is something to be said about this and the peace of mind that comes with it.

Obviously, this is a spectrum with extremities and many ways to go about it. We should not discount "Move slowly, and not break things".

> I know with C, I can rely on it for 45 years.

Right, but that's at the expense of 50 years of progress. For the overwhelming majority of languages I strongly contend that's the wrong tradeoff.

I think you need at least two major releases (or two years) where you can do things the old way or the new way.

Some organizations have a long path to production for runtimes and would prefer to skip releases. Requiring a code change to be tied to the runtime update makes things harder and encourages staying on the old release forever.

Your "move fast, break things" speed must be inversely proportional to the penetration of the language: penetration into books, academia, courses, hardware, firmware, software, software that depends on other software, and so on.

Soon I will be able to let go to Python 2.7, but not just yet. My industry is the precise opposite of "move fast, break things" because here the "things" happens to be what are called humans and their "break" scenario involves shrieking as their lungs are burned to leather, and no amount of Jolt soda poured into the floating holographic head of Mark Zuckerberg will suffice.

If you have a piddly language you and five people circulate on Usenet, go for it. Break it twice a day. But once your language gets serious, be ready for inertia. It's a property of mass.

Agreed. Python 3 and IPv6 are case studies in how not to do an upgrade.

While I can appreciate the care and engineering skill involved in upgrading a technology while still having backward compatibility, it turns out just to enable the worst possible behavior on the consumer end.

I think I like Apple's way of just forcing breaking changes down everyone's throats (e.g. removing CD/DVD, PowerPC->x86, 30-pin->Lightning, dropping headphone port, etc.). Sure the tech media complain for a cycle, but then everybody adapts and the world's a better place.

There was literally no choice when it came to IPv6. Everyone who thinks they've got a clever compromise is forgetting that it's not a software problem where it matters.

Embrace Extend Extinguish

I vote for IPv5 protocol, which will allow to cal any IPv4 host via IPv5-IPv4 NAT transparently at nearest bridge.

WHY I cannot do `ping6 127.0.0.1`? WTF? Is IPv6 is too small for whole IPv4 address space?

You've listed a command line you'd like to run, but have you spent any time to decide what it is that you think it should actually mean?

Maybe you think it should just be exactly equivalent to ping6 ::1 - your ping6 could be updated to allow this but, like, why? That's not an improvement at all.

OK, so maybe it should be equivalent to ::127.0.0.1 - but that's very strange, an operating system could choose to have this do what you expect locally but it's not clear to what end and again it doesn't offer any wider improvement.

IPv6 is big enough that it contains the entire IPv4 address space - in several different places in fact although ::a.b.c.d is the most transparent. But, that's completely irrelevant to the problem, which I suspect means you still haven't quite grasped what the problem even is.

You're wrong here. Folks actually deploying wanted better co-existence options for folks on IPv6, and it's shown up. You basically route ipv6 as IPv6 until you hit a network border that doesn't support IPv6, then go over to IPv4 for the rest. This let's me do IPv6 only devices without giving up IPv4.

As it is, they locked the upgrade cycle, because you can't do IPv6 only without losing access to IPv4 (historically).

In the end folks did get work arounds going using 464XLAT

https://datatracker.ietf.org/doc/html/rfc6877

I expected (20+ years ago), that IPv6 address space will be extension of IPv4 address space, so `ping6 IPv4_ADRESS` will use IPv4 protocol internally, bind6(IPv4_ADDRESS, IPv4_PORT) will use IPv4 socket internally, and so on. It's easy to implement:

  an_ipv6_function(address) {
    if (address & 0xffffffffffff0000 == 0) {
      // Use legacy ipv4 protocol ...
    } else {
      // Use modern ipv6 protocol ...
    }
  }

Actual result: I cannot ping an ipv4 address using an ipv6 tool, or connect to an ipv4 server using an ipv6 only tool.

That already exists, it's called an "IPv4-mapped address". The IPv4 address 198.51.100.15 can be represented in IPv6 as ::ffff:c633:640f (normally shown as ::ffff:198.51.100.15). See in particular at https://man7.org/linux/man-pages/man7/ipv6.7.html the IPV6_V6ONLY option which can be used to disable that mapping.

Yes, I know about this mapping (I can read man pages too), but

a) ping6 ::ffff:127.0.0.1 doesn't work too;

b) it's not an IPv4 address, because the binary representation of ::ffff:127.0.0.1 != 127.0.0.1. It's the IPv4 address space mapped to an IPv6 address space (IPv4-mapped-on-IPv6).

c) IPv4 addresses in IPv6 address space are deprecated: https://datatracker.ietf.org/doc/html/rfc4291#section-2.5.5.... .

In short, IPv6 and IPv4 cannot coexist, so they must be separated, so we must keep IPv4, because it's mandatory, and we may skip IPv6, because it's optional. So, all internet providers in my area are skipping IPv6, because they can.

All that problems are created by lack of backward compatibility in IPv6 protocol.

Why? «Because backward compatibility with IPv4 is unnecessary. IPv6 is so much better. Everyone will jump to IPv6 till end of the next (1995) year.»

Why not IPv10?

Right. The specific problem is that 2^32 isn't enough. The most naive "solutions" were from people who hadn't even recognised why it's 2^32. For example, let's just have numbers bigger than 255 between the dots ... why can't we do that?

But once you get past those, another layer of "solutions" see the immediate technical consequences of 2^32 not being enough but don't grok the bigger picture. For example let's just turn existing 32-bit IP addresses into a 64-bit IPng address by adding zeroes, and then all the old addresses are now networks with up to 4 billion nodes. But wait, all those addresses are assigned too now, so we still have an address shortage. Oops.

The least stupid people saw the new address scheme itself as logical but just felt like surely IPv6 should have resisted the urge to fix anything else. Sure, you're embarking on a multi-decade project that likely cannot ever be repeated, but let's just leave all the sharp edges and weird anomalies exactly as they are to minimise upgrade costs. This belief is less stupid, but it is still pretty stupid, that cost difference is a drop in the bucket, and this was our only chance to fix some pretty huge mistakes made when the Internet was newborn.

It would actually be cheaper for a lot of big outfits to go IPv6-only today (and then use edge translators) than drag their heels and try to stay on IPv4 as long as possible, but contrary to popular belief almost nobody really does "cost-benefit analysis" to decide what to do - they make a decision based on emotion and any such analysis is purely there to support that decision.

Buying more addresses is a budget line item you can see, but a lot of big IPv4 costs are hidden in existing practices that "always" have cost money but would vanish under IPv6, and some IPv6 costs are only there if you did no preparation. If you just count the need to replace those IPv4-only printer-copiers you purchased a year ago across the company, and forget that you intentionally chose not to ask bidders if they were IPv6 capable you're going to persuade yourself that it's cheaper to keep waiting.

IPv6 could have been a straight forward expansion of the address field without replacing ARP (which could easily have been extended for v6 addresses, just need to define an address type value) and header processing and address autoconfig and X and Y and Z.

It still would have taken a long time to deploy, because you still need hardware and software support in lots of places, but it might have been a little less long, and a little less bad along the way.

I don't think we got much of use from the new ways either. I can announce IPv6 ranges to my LAN from two different ISPs, but my clients will use an IP from one and send to the gateway from the other, so that doesn't help me.

ARP is a broadcast protocol, so that's pretty bad news, whereas IPv6 Neighbour Discovery gets to use (link local) multicast.

On the cheapest budget cobbled together network, either one turns into a network broadcast. But even a bargain basement 1990s Ethernet chipset does onboard multicast filtering, so irrelevant neighbour discovery packets needn't wake up your OS whereas every ARP query must be drawn to the attention of every host's OS to confirm it's not for them.

However, spend a little more money and the network switch understands multicast, so now the data isn't even sent across links which don't have anybody listening to that address, whereas of course nothing can be done about broadcasts.

You couldn't have easily done this trick in IPv4 because the address space is too small, but in IPv6 choosing the Ethernet multicast addresses to make this work falls out very easily.

I feel like Apple's way of forcing is because you cannot buy new Apple products with old features. In software you can keep using the old version and complain about the new one. This problem is especially visible in poorly designed ecosystems like Go and Python. Do you think introducing generics to a language like Go won't cause a great divide in the community? Sure, it won't be a breaking change but it will still divide people into the "old way" and "new way" camps. I guess the language which changed the most along the years without breaking things is C# but then again .NET actually introduced a lot of breaking changes unrelated to the C# language.

Bad example actually.

Lots of us on Windows are still stuck with .NET Framework, because no matter how Microsoft keeps pushing it, even their own products aren't fully ported to Core, let alone 3rd parties.

That's what I said as well. C# good, .NET bad

One doesn't go without the other.

Ah, and don't forget FireWire.

IPv6 is great... the only problem is, that IPv4 has be patched enough with workarounds, to make IPv6 unecessary for most "normal users", and there is nothing to sell there.

Yes, a simple voice call might use STUN, through two nats, fail to puncture a hole, and then be proxied over a server somewhere, but "it works". Noone is willing to may 1$/€ more per month to get IPv6, noone really needs it, half of IoT doesn't support it (ahem, esp8266/32), it's costly to implement and costly to educate users (especially those, relying on nat for "security").

Most of the IPv6 progress is because the governments mandate it for many stuff, big cloud providers are running out of addresses to buy, vendors want to sell new routers and switches... basically, everybody except the users.

> But, Python 3 took longer than a decade to kill Python 2 as they were trying to give people “plenty of time”.

I disagree, I think you're learning exactly the wrong lesson from the Python3 debacle. It took a long time for Python 2->3 because the Python3 developers ignored the need for easy backwards compatibility, just like the Go developers are doing. Python3 required all software, including all your transitive dependencies, to be simultaneously updated. Python3 was dead-on-arrival for many years until there were finally concessions made in Python3 to make backwards compatibility less painful (in this case, so that code could more easily be written to work in both Python2 and Python3).

If the go developers want people to use "install" instead of "get", sure! But give time so that the change can be distributed across the ecosystem before removing the functionality. There's no hurry.

I mean, the Python2 vs Python3 split was primarily caused by language changes, not tooling. And Go’s 1.x comparability promise is rock solid. It seems difficult to compare the two.

Except py3 needed to be a breaking change, because unicode handling. It doesn't matter if the API rearranging was delayed and the only change 2->3 was string/bytes handling, it still would have taken a decade.

What's the alternative? Deprecate warning old-style string handling and slowly nudge it out? Great, that's not actually decideable, since you need to know how a str is being used.

str are at the very heart of python. This would be a gargantuan task just to write the linter.

Ruby made a similar change in character encoding starting at roughly the same time; it's (barely) remembered now as a non-event because they didn't take the "opportunity" to bundle in a bunch of other breaking changes which collectively required much more pervasive dinks to pre-existing code than the encoding issues.

This and that have literally nothing to do with one another.

AFAIK Ruby's change was that they introduced a notion of encoding to strings. Python literally change the string types and its semantics.

Adding a bunch of other BC breakages was a good idea. Having been responsible for the migration of a large codebase the strings change was by far the most disruptive and problematic one, the rest did not even come close (the distant second was the change to rounding IIRC). Ripping the bandaid and adding a bunch of other BC breaks to clean things up or prepare for the future was I think an excellent idea. Most of those were easy to shim or work around (the syntactic BC breaks were especially trivial).

The problem, as GGP's comments correctly notes, was the incompatibility between Python 2 and Python 3: when Python 3 was first released it was impossible to write a non-trivial program which could run on both version.

Meaning if you wanted to port a project you had to convert everything at once. And as if the odds of that working were not infinitesimal enough, libraries had to fork their own codebases, and either try to maintain completely incompatible codebases concurrently or drop support for Python 2 while literally all their users were still on it.

> Ruby made a similar change in character encoding starting at roughly the same time; it's (barely) remembered now as a non-event because they didn't take the "opportunity" to bundle in a bunch of other breaking changes which collectively required much more pervasive dinks to pre-existing code than the encoding issues.

No, its often misremembered that way; Ruby 1.9 had lots of backward-compatibility breaking changes, there was a long time when lots of things were stuck on 1.8, and much of the other stuff that happened with the Py2 -> Py3 conversion was mirrored. The big difference was that the Ruby community and ecosystem were smaller and less diverse.

Ruby's also a bad comparison because not many folks are running ruby on windows.

Huge chunks of industry and academia run python on windows (usually through (ana/mini)conda). That's a whole 'nother world of encoding hurt.

> Ruby's also a bad comparison because not many folks are running ruby on windows.

Especially around the 1.9 transition; Ruby was barely usable beyond the level of toy code on windows. The Ruby on Windows story is actually a lot better now.

Which things in particular? Rails supported both Ruby 1.8 and 1.9 simultaneously from Rails 2.3 through Rails 3.2, without it apparently being a huge struggle for the developers, dropping 1.8 only at about the same time that support for it was being dropped by the core Ruby devs altogether. And other posts from maintainers of smaller, but widely used, gems describe it as being a bit of a bother to set up, but not nearly the struggle that it took to run anything in both Python 2 and Python 3. See, e.g.,

https://thoughtbot.com/blog/the-journey-to-ruby-1-9

One key here is that the common subset of Ruby 1.8 and 1.9 was large enough that you could ship a single library for both, and a lot of large ones did, as in the above examples. That wasn't the case initially in the Python transition, and by the time Python3 had started making enough concessions to back compatibility to make this practically possible, people were a lot less willing to try.

(I personally was managing upgrades or a large Rails app at the time; Rails upgrades were, and remain, a bear, but as an accident of code style which just avoided the tricky spots anyway, the changes required to move that code between any two Ruby versions have never been more than a day or two.)

> Which things in particular?

Lots of things; there was for several years a “caniuse” equivalent for Ruby gems tracking if they were on 1.9 yet because it was such a big issue. Bigger issue outside of the Rails-centered ecosystem.)

> I personally was managing upgrades or a large Rails app at the time; Rails upgrades were, and remain, a bear, but the changes required to move that code between any two Ruby versions have never been more than a day or two.

Yeah, a lot of Rails use exercises a fairly narrow slice of Ruby language and core/stdlib directly (that js, outside of library code), while leaning heavily on Rails-provided APIs, so that's not too surprising.

It didn't need to be such a disruptive breaking change that fragmented the ecosystem.

They should have made the unicode handling changes opt-in per package or file. Then, start warning about anything that hasn't opted in. Then, only after a significant migration period, change the default to the new unicode handling.

It's well known that you should make the "correct" choice the default. So unicode should have been opt-out (from __past__ import legacy_strings). Result: Everyone batch-adds unicode opt-out to their whole codebase, but new files use the modern way. Gradually people remove the opt outs as they go over old code.

The correct way should be the default, but in the case where you already chose the wrong default, you shouldn't introduce the correct way of handling things and make it the default all at once.

Upgrading to a new major version should never require changes to your code if you fixed all deprecation warnings in the previous major version. Otherwise, you make incremental upgrades impossible.

The new alternative should always be introduced as opt-in with the old default deprecated, and then made default in the next major version.

This couldn’t work. For starters, the names of classes, attributes and functions are str in both versions, which is to say, bytestrings in Python 2 and Unicode strings in Python 3. This cannot be handwaved away with a pragma.

    str = int

is perfectly valid python. So having str mean different things in different modules is already possible.

You missed the point. It’s not about names themselves. What happens if I want to lookup bytestring-named attributes on a class defined in a Unicode-string module or vice versa?

I'm afraid I don't understand your question completely, so I'll answer what I guess you mean. Let's say you have a function in one module that expects a bytestring. This function is called from another module, and a unicode string is passed. What happens?

Well python is a duck-typed language, so it will handle the unicode string as if it was a bytestring, and this may or may not work correctly. It's up to the modern code to interface correctly with the legacy code. This could mean manually converting between bytestring and unicode string.

That ignores how python scoping works. Built-ins are actually the last thing python checks.

You can override the built-ins by re-defining the symbol in any of: local functions, enclosing functions, or the global scope and it will impact the entirety of that scope.

You missed the point. It’s not about names themselves. What happens if I want to lookup bytestring-named attributes on a class defined in a Unicode-string module or vice versa?

Ahh, the wording screwed me up.

If I'm understanding your comment correctly this time, it would look something like:

getattr(thing, b'attribute_name')

I guess you could make it work as long as everything is ascii right? And if it's not then raise exception.

If it's not ascii, you would just do:

getattr(thing, u'attribute_with_unicode_characters')

This would work in both python 2.7 and 3.9 (probably works with other 3.X versions but it's the one I actually tried).

The changes that were needed were far too invasive for that to work in practice. Any library that touched anything even vaguely stringy or bytey would have needed to be operating in either bad mode (2) or better mode (3), but that would have been an almost-invisible part of its public API, which starts causing serious trouble when you want to use bad mode libraries and better mode libraries simultaneously, since the two are fairly incompatible, so that automatic translation would not be possible, so you’d have to know which mode a library was operating in, which undermines the whole endeavour.

In practice this isn't as big a problem as you're making it out to be, because UTF8 is compatible with ANSI.

Nearly all of computing has made a similar transition in the 2000's and 2010's, and I've certainly debugged my share of encoding issues, but in general it worked for a large majority of use cases.

I think most people would've taken a few encoding issues over the 10+ years long migration.

If it were ASCII and UTF-8 that this was all about, it wouldn’t be as big a deal. But it’s not.

Python 2 was a code pages world. Not an ASCII world. Also Python’s Unicode isn’t and hasn’t ever been UTF-8 anywhere; rather, it’s sequences of Unicode code points (note that that’s code points, not Unicode scalar values−this and the representation of str are a pair of things I find quite bafflingly bad about Python 3, because it means Python’s Unicode strings aren’t, y’know, Unicode strings, because they can be ill-formed, and they’re stuck with bad fixed-width representations so that any even vaguely interesting string will be stored as (ill-formed) UTF-32).

Python 2’s str type was a mess, used for bytes data, for strings of uncertain encoding, and for Unicode strings. In a vacuum, you might say you could make that work by splitting it up (and indeed the types module was an early endeavour to do that sort of thing back before it became evident it was insufficient). The key is that this confusion and widespread misuse was rife through the standard library and other third-party libraries. If the Python 2 → 3 transition had been just str-as-bytes/str-as-who-knows-what/unicode → bytes/something/str, it might have been manageable. But the libraries were so shot that a straightforward migration was impossible, because str was too many things and it was quite impossible to tell statically what should be done about it. (And impossible dynamically to resolve as well.)

Python 3 IO isn't UTF-8. The entire point of the change was to support _arbitrary_ encodings (most of which are legacy code pages).

Python would sometimes guess the output encoding on Windows to be ASCII even when UTF-8 was supported by the output device. So you'd end up with encoding-oblivious Python 2 programs that worked perfectly reading and writing UTF-8, and Python 3 programs that failed upon encountering a non-ASCII character.

They've since improved the defaults on Windows, but the changes in Python 3 that caused me a lot of pain were when they made IO to be non-Unicode.

Man, it's it's harrowing in hindsight to look back on porting 2->3 on linux, and remembering my coworkers that had to do it on Windows with encoding-oblivious libraries.

They're still constantly finding bits of code that break cause some customer decided to switch from "delta" to “Δ” in some machine learning log output.

> What's the alternative?

Have an import switch that enables a legacy mode, that will convert your strings into and out of the library? You can even require that the application developers cast their strings into the correct ones if you don't want to calculate it on the compiler.

That's just not possible with python's computational model. That works for "from future import print" cause that's just an additional function.

str is too deeply embedded and used in too many dynamic and unique ways for that approach to work without doing some really ugly or nonperformant or community-fracturing things.

You're completely missing the point of GP's comment.

Their point is not that Python3 should not have broken backwards compatibility, it's that Python 3 should not have been entirely incompatible with Python 2. That decision is what hampered migration for years, because it turned out not to be a feasible migration strategy, it would have required the entire ecosystem to migrate in lockstep which when you think about it made no sense.

Python 3 migration picked up steam once the core team backported what could be backported to Python 2(.7), and reintroduced syntactic compatibility features to Python 3.

At that point, and with a few shims for standard library stuff, libraries could create cross-version codebases, which meant they didn't have to try and fork their own codebases. And dependents could migrate at their leisure.

> Their point is not that Python3 should not have broken backwards compatibility, it's that Python 3 should not have been entirely incompatible with Python 2.

No, my point is precisely that py2 and 3 are not compatible. Even with the backports, py2 and 3 are not fully compatible. Hence the "py23" or "six" dialect, which is the subset of 2 and 3 that overlap, with some shims to facilitate it.

Py2 and Py3 string apis are not compatible. They cannot be made compatible. They can never be compatible. There's no "from __future__" that could make them compatible without a stupendous amount of work.

Not to be pretentious, but I'm developing the opinion that anyone who thinks that the Python team could have just shimmed their way between 2 and 3, does not really understand the Python compute model.

Google literally invented a py2 to go transpiler (grumpy) to avoid converting. Do you think for a second if there was a way to just "from future" their way to a compatible str object, they wouldn't have taken that route?

> No, my point is precisely that py2 and 3 are not compatible.

Which still misses the point, py2 and py3 not being compatible is different than them being completely incompatible, because there is a useful, workable, common subset of Python 2 and Python 3.

The entire issue with Python 3 was that there originally was not, the core team had to be dragged kicking and screaming into carving it out through backports and reintroductions because their original strategy was not workable.

> Py2 and Py3 string apis are not compatible. They cannot be made compatible. They can never be compatible. There's no "from __future__" that could make them compatible without a stupendous amount of work.

So?

> Not to be pretentious, but I'm developing the opinion that anyone who thinks that the Python team could have just shimmed their way between 2 and 3, does not really understand the Python compute model.

How can you claim that when "shimming their way between 2 and 3" is exactly what ended up happening?

> Google literally invented a py2 to go transpiler (grumpy) to avoid converting. Do you think for a second if there was a way to just "from future" their way to a compatible str object, they wouldn't have taken that route?

Have you considered stopping beating up your strawman?

I should define "shimming their way". I mean smoothly interoperating python2 code with py3 mechanics in a piecemeal way.

Writing 2/3 python is harder than writing either 2 or 3. Porting 2 code to 2/3 is about as hard or slightly more work than just converting straight to 3. But at the end of it, the benefit is your code can run in both environments.

So let's say instead of pulling the plug, py leadership says "Python '2.8' will only run 2/3 code." How is that any different? You're gonna have the people who already ported, who have no need for 2.8 cause they can run 3, and the holdovers who have yet to port from 2.7.

That's what I mean by there's no shim.

> The entire issue with Python 3 was that there originally was not, the core team had to be dragged kicking and screaming into carving it out through backports and reintroductions because their original strategy was not workable.

Maybe I got into the game too late then, but I don't recall it being this much of a dust-up.

"six" was released on pypi as 0.9.0 on Jun 28, 2010. 1.0 was in Mar 2011. Even if the py core team was dragged kicking and screaming, that's still 4 + 5 years of people who could have been porting, and weren't.

Basically, my primary point is, it was a damn hard problem, and I doubt there's much the core team have done better at the time. All the porting tools in the world won't get people to upgrade if they are resistant for any reason.

E: py3 was released in Dec 2008. So I guess those first two years might have been tough. But to me that seems exactly like the ripe time to work on the translation infrastructure and things like six, where you have public release but no immediate pressure to switch.

> E: py3 was released in Dec 2008. So I guess those first two years might have been tough.

3.2 was released early 2011, and P2 compatibility features were reintroduced as late as 3.5 (% on bytes) in 2015, though for my money the last key one was the reintroduction of the `u` prefix in Python 3.3, in 2012. That's the point at which the migration really became feasible without suffering.

Furthermore that's assuming all dependencies were ported, which was not the case: waiting until porting was feasible (and they had the time and manpower to) was also something they had to wait on e.g. Django first added Python 3 support in 2013 (IIRC PyLint only supported 3.4 onwards), difficult to port your own software if the stuff you depend on hasn't been ported over.

We're not talking about deprecating language constructs though. This issue is only talking about deprecating a sub command in favour of another sub command from literally the same binary. This should take all of 10 seconds for the developer to adjust to. I'm all for pragmatism but lets not exaggerate the significance of this proposal.

And it might well be that the deprecation notice will run for two versions. At the moment no official decision has been made as it is still an active issue.

This exaggeration of Python3 migration is ridiculous and frankly has become boring. Yes, Python 3 broke compatibility. But we had more than a decade to migrate. It definitely does not deserve to be the poster child for migrations gone wrong.

The python language developers did not do it wrong. The community of python users, which include companies with ten year old unmaintained scripts, did not want it invest to transition until the absolute last second. The "Apple Solution" is to make the last second the first second. Who knows if that's the right thing to do.

The python language developers absolutely did do it wrong. You could not run mixed python2 and python3 code. Either all your dependencies were migrated and you could flip the switch and try things out, or some stragglers kept you running code with compatibilities for 2 and 3 while you waited.

And if one of your dependencies started using new features from 3.x it would break everyone stuck on 2.7, so you can't even use the new features in your library. So why would you bother migrating? So everyone was stuck in a game theoretical position where there was no first mover advantage because one straggler would invalidate all the investment.

> So everyone was stuck in a game theoretical position where there was no first mover advantage because one straggler would invalidate all the investment.

Clearly not, as most libraries are now Python 3 only, so the switch did happen.

I do see a lot of people complaining but I see no practical solutions being suggested. "Just don't do a breaking change" isn't a practical solution when the problem being fixed was such a core feature of every language, strings.

>Clearly not, as most libraries are now Python 3 only, so the switch did happen.

You don't build a convincing argument that the python developers did nothing wrong when saying this 1.5 years after python 2.7 has been EOL.

>I do see a lot of people complaining but I see no practical solutions being suggested. "Just don't do a breaking change" isn't a practical solution when the problem being fixed was such a core feature of every language, strings.

A system should have been put in place to allow python3 code to import python2 code. Then people have an incentive to actually run python3 and use python3 features.

More people are stuck on Java8 which has been EOL for a very long time. They are often stuck because Android hasn't upgraded afaik. But Java11, 14, etc can use almost all Java8 code unless it uses I think sun.misc.Unsafe or something like that. The point is that people can use modern Javas with Java libraries targeting older versions.

> The community of python users, which include companies with ten year old unmaintained scripts, did not want it invest to transition until the absolute last second.

Try shipping python3 scripts to systems that only have python2 interpreters. Doesn't work very well unless you also start to ship the interpreter and all its dependencies. I learned my lesson and just avoid distributing any Python code to customers now.

What language lets you ship scripts that use features of version X and also works on versions <X?

Docker tends to be the easy solution for all this stuff, and it's language agnostic.

> Doesn't work very well unless you also start to ship the interpreter and all its dependencies.

That basically applies to everything that isn't a statically linked binary.

Agree with you here. If you try to deploy a C program to ancient centos then it can also fail with glibc being incompatible.

https://developers.redhat.com/blog/2019/08/01/how-the-gnu-c-...

But it will run on a modern system if you compiled it against the CentOS version. I have a VM running to do just that. Trying to run Python 2 code on a modern runtime on the other hand will have the Python community feed your remains to the pigs.

Much different from Python, though. As long as you compile your C program against the same version of libc you're running against, the program will still work, no matter how old the code. You just can't run a program that was compiled against an incompatible libc. A Python 2 interpreter can't run Python 3 code, and vice versa, no matter what, because the actual language changed.

You can't always easily compile an old C program against the new C libraries you are running. Even if you can compile it, changes in libraries can break programs. E.g. say in the ancient library, memcpy with null pointer parameters worked, if the size was zero. The new library checks it and aborts. There are all sorts of issues like this.

Every time libraries and compilers advance, you're effectively porting all the existing code. You have to fix any build breakage, and re-validate everything.

Moreover, Python is a C program. Just get the old Python 2 C code and compile it against "the same version of libc you're running" and it will work, right? Where is the problem?

Why recompile it when the existing binary still works with the current runtime? At least on Linux the old version of memcpy will still be included in the runtime library to avoid this kind of issue.

The point is that the code needs the runtime.

> What language lets you ship scripts that use features of version X and also works on versions <X?

I didn't care about Python 3 features, but you can't run Python 2 code on a Python 3 interpreter either and the Python 2 interpreter was abandoned. Meanwhile you can run ancient JavaScript code on both old and modern systems, ancient C++ on both old and modern systems, ancient Java mostly on both old and modern systems, etc. .

> Docker tends to be the easy solution for all this stuff, and it's language agnostic.

I think I will rather go with statically linked binaries before I dump docker on unsuspecting users. Just the pain of getting that dependency whitelisted by their corporate IT makes my skin crawl.

I'm starting to think that everyone would be less outraged if they rebranded python3 to <insert_different_snake_here>

Shouldn't that be <insert_different_british_comedy_troupe>?

The Mighty Boosh just isn’t the same

I don't like overly tech-specific job titles, but "lead mighty boosh developer" sounds wonderful.

Every person saying "pythonic" instead of idiomatic would need to switch to "booshy" instead. The file extension would be .og (for old Greg).

Only pair programming would suffer as the main philosophy of the language would we coding in isolation.

> Only pair programming would suffer as the main philosophy of the language would we coding in isolation.

Well, that, and the constant drinking of Baileys from a shoe...

Look at how well that's worked out for Perl and Raku.

The Raku name was adopted not even 2 years ago, and it's not really the most popular programming language niche anyways.

I'm sure they would have been.

I'm also sure they wouldn't have migrated.

Yeah I guess it’s a trade off

It is easy to look at Python 3.x today and say the migration should have been easy, but some of us were there, and remember how intransigent some core devs were about things like u"" and %, and how Python 3 offered nothing better upon release.

https://news.ycombinator.com/item?id=22036773

Not to mention, as the cherry on the turd sandwich, performance of early 3.x versions was substantially worse than 2.7.

And it is still slower for small glue scripts. On my Windows 10 box hello, world script finishes with Python2 in 73 microseconds (the best time of 5 runs) versus 238 ms for Python3. That translates into noticeable slowdown of compilation time for a project extensively using python code in the build system.

It is not an exaggeration, the Python3 move was one of the worst releases ever in terms of managing the process. It took a decade for library creators to have it all settled. It was not end users fault that they depended on libraries that wouldn't upgrade for several years.

The philosophy of incompatible upgrades is brutal for popular languages, IMO.

The main issue is not that I give you X amount of time now please fix it coz you have generous time now.

The cost of such a change is ENORMOUS.

As an example, in a recently IPO'd company we had a team of engineers and we had 6 months of work for 10 engineers, round the clock, to fix the Python 2 to Python 3 migration issues alone! and the original creators of the code were long gone! .. and I could only imagine the plight of thousands of other folks in similar boats and other resource constrained boat-less entities!

Then there are distributed packages and libraries that are used by a large set of audience that are dev complete and no longer maintained as such. The cost of fixing those are much much higher.

It makes one more than wince at the thought of going through this exercise again.

The problem was not that Python3 had broken things. The problem was the scale of breakage and inability to use both python2 and python3 code. Moreover only after a few years since Python3 release it became possible to write libraries compatible with both Python2 and 3.

This is an interesting side effect of open source. Companies can not afford to break their customers deployments and so tend to bend over backwards to guarantee backwards compatibility. But in open source that link is much more tenuous and project reputation does not directly translate into monetary value in most cases, especially if there is no commercial model attached to a particular solution.

This is made worse by the fact that open source is typically developed with a very early public release to drive adoption, rather than that the first decade or so the solution is battle tested in-house and only released when most of the kinks have been worked out.

Any type of backwards compatibility breakage provides a decision point to either decide:

* do I put in the work to upgrade all my code so that it works with this new version?

* do I jump ship to some other library/language that I've been interested in?

In the Python 3 breaking case, it provided good reason for engineers at major enterprises to start building in Go instead of investing in the Python 3 upgrade.

Another debacle was Angular being completely backwards incompatible, giving an opportunity for React to gain mindshare.

In the language proper, the Go authors have made very clear commitments about backward compatibility. (See https://golang.org/doc/go1compat.)

What's notable about this situation is seeing that their compatibility commitment extends only to the stuff inside the language, and not to the tooling provided to work with the language.

I tend to agree. I'm one of those developers maintaining a few open source projects on the side and sometimes struggle to keep up with updates to the tooling and best practices.

For me, it has caused more confusion than help that the old way of using go get still works, even without any warnings to tell me I should start using the new way. I wanted to upgrade myself to using the latest best practices ASAP, but struggled to learn the proper new way, because all the old ways I do almost automatically still work.

As long as the messages about how to do instead are really clear and helpful, I find it should be better to help people move in the right direction as soon as possible.

To be fair, python way of handling versions is the worst and that even goes to the package maintainers. Especially what happened to python 2 -> python3. I don’t like the logic of no version specified = python 2. I have made the mistake of thinking i have the right python binary only to realize it gives syntax error for being python 2

> Python 3 took longer than a decade to kill

That was an inevitable conclusion from two facts:

1. Python 2 was EXTREMELY well adopted. Like, Java-level, but even deeper in the stack.

2. Python 3 had fundamental changes. (namely byte/string)

The only way for a shorter migration is a change to one of those two facts.

Go has repeatedly chosen "(1) break lots of people".

Not for the language itself, but the tooling around it.

This tells me that you've never used a well-managed language or framework. Those aren't the only two options.

Ruby 1.9 was released around the same time as Python 3, with a similar amount of large breaking changes, and no one was holding on to Ruby 1.8 the way the Python community held on to Python 2.7. You could keep using it if you wanted to, but no one did. Ember.js is another example that I've used that was refreshingly well-managed, with a clear (oftentimes automated) upgrade path between minor releases. Using semver and LTS versions, it didn't just break backward compatibility without a significant deprecation period. Those are just a few examples off the top of my head, but I'm sure other good examples exist.

When things are well-managed, you as a developer have the freedom to upgrade when it's a good time for you. When you have production software with real users or a business that depends on it, you can't always just drop whatever you're doing to upgrade, which is why backward compatibility is needed.

If it's done right, there's a self-imposed drive to want to move to the latest with all its improvements. On the other hand, with Python 3, they took away things that people cared about, leaving developers not only with no drive to upgrade, but an irrational impetus to hold on to what they had, even when the situation was fixed at a technical level.

Reply on news.ycombinator.com