Hacker News Reader

Comment by korethr | Hacker News Reader

Hmm, should perhaps this be worthy of a major version number bump? Yes, it's not as large or as breaking of a change as the 2.7->3.0 transition, but by removing modules from the standard library, backwards compatibility is being broken. Now, perhaps most people don't use those features anymore, but for those people who do use and rely on these modules, a major version number change would be a welcome signal that their code will no longer work with the new version.

But, after the initial pain of the 2.7->3.0 transition, I doubt we'll ever see another major version number jump, even if a logical use of version numbers would merit it.

View on HN

Python minor versions allow things to be removed and thus 'break backwards compatibility'. There is a process for warning of upcoming deprecation then removing it after that. For modules this is documented in PEP 4 (from 2000). Similar deprecation schedules are used by other important ecosystems such as Django.

We'd be on way more than Python 3 if the major version was bumped for each one.

This. There are plenty of backward incompatible changes in every minor version bump.

I think he’s hoping they’d adopt semantic versioning, probably won’t happen.

https://semver.org/spec/v2.0.0.html

It's also a bad idea. Semver doesn't distinguish between removing an API that has been deprecated for a decade and Python 2->3. Or, for that matter, between removing one function and removing large parts of the standard library.

Both are changes that will break existing code. SemVer clearly signals this. Why bother with how long something has been deprecated?

You don't see a difference between removing a rarely-used module that has been deprecated for a decade, giving developers 10 years to replace it on the one hand and overhauling the entire language, breaking not just some but most code, on the other hand?

I'm sure Python 2.7 broke compatibility, but you don't see people refusing to upgrade from 2.6 ten years after its release.

Of course there's a difference, but it's irrelevant here, because no matter how long breaking changes have been announced, they break existing code. I prefer a versioning scheme that signals this.

> I prefer a versioning scheme that signals this.

How about <Very breaking changes>.<Breaking changes>.<Bugfixes>? That's what Python already does.

Semver is garbage.

I was once a fervent devotee of semver. It's so predictable! It's structured! There's a system! Everyone loves a system, especially engineers.

I've since become disillusioned.

The problem with semver, in my experience, is that it's impossible to predict whether a change will actually break someone else's code. Of course there are certain classes of change that are more likely to cause problems for other people. Changing a function signature, or deleting a function outright, is obviously a breaking change.

But the line between breaking and non-breaking isn't a bright one. Move away from the obvious examples and things start to get murky. Even the humble bugfix can be problematic. What if a client application unwittingly relies on the buggy behavior? Now that fix is breaking for them. Is that a contrived concern? Maybe—though anyone who has written an emulator can attest that this a real problem.

What about a non-breaking feature addition? Let's say the new feature requires some extra branches in a function, but doesn't change the function's interface or behavior for people who don't use the feature. Fine, non-breaking. Now say these branches alter the function's performance, and a client application's batch job that used to run in under an hour now takes four hours. It does run, so it's not "broken," but four hours is an unacceptable runtime to the users. Is that still a non-breaking change?

What these examples demonstrate to me is that semver's breaking/non-breaking change concept is incoherent. It conceives of changes as universally breaking or non-breaking, out of context, but a change can only be breaking or non-breaking in the context of a specific client application. Even the seemingly obvious example of deleting a function is non-breaking for applications that don't use the function!

I think the way we release software reflects that we know this deep down, even if we don't admit it. Imagine how you might handle upgrading a library in an application you've written. The new library is a bugfix release. Do you upgrade the library, push it to production at 100%, and gallop off to lunch without so much as a glance over your shoulder? My guess is no. Personally I'd be running my test suite, reading the library's change log, making a gradual release, and keeping a close watch on instrumentation during and after.

The interactions between software systems are simply too complex, too nuanced, too specific to the particular applications. We put all these safeguards in place and do things cautiously because we've been burnt too many times. And we've learned that in actual fact, the line between breaking and non-breaking doesn't exist.

Red Hat 6 is supported for another 18 months and still ships with Py 2.6.

Python's slow-moving gently gently approach to breaking changes has not been good for the ecosystem. I'll be glad when 2.x is dead. 7 months 8 days 14 hours... https://pythonclock.org/

But that's just an enterprise distribution doing enterprise distribution things. If you don't upgrade in general, of course you don't upgrade Python.

"just"? From where I'm sitting that's a pretty large chunk of IT.

This has knock-on effects: authors that want to deploy scripts/apps with the minimum fuss will avoid adding deps to whatever /opt-based repo RH ships Python 2.recent (and the hoops you have to jump through to install and activate that). So they remain compatible with 2.6.

All of the other applications and 3rd-party modules shipping with RH 6 are also chained to Py2.6.

Many conservative shops (industry verticals) will refuse to upgrade _anything_ until they absolutely have to. I suspect we live in slightly different IT worlds (lucky you!). This is a problem I see frequently and that's why I'm suggesting Python needs more strict impetus for timely upgrades, not more decade-long opportunities for balkanization and incompatibility.

What I mean is not that Red Hat is insignificant, but that it's not special that it does not upgrade Python. Even in non-conservative shops that generally would just use the latest version, using Python 3 instead of 2 was not a no-brainer for a long time.

> but by removing modules from the standard library, backwards compatibility is being broken.

API 101 - there is very little cost to leaving them in, but a hidden major cost to having them disappear, usually for non-developers.

By the way, "batteries included" is one of the BEST features of python.

Have you tried to fix something in your house with a "homeowner's toolkit" which is usually something like a hammer, pliers, 2 screwdrivers, a putty knife and a few more basic tools?

It is REALLY tedious, like writing a C program with a few basic tools like stdio and ctypes.

More languages need "batteries included", maybe like Perl.

I think if the cost of deploying a script is 1, deploying it + a dependency is literally something like 100x. You have to make assumptions about all the environments the script will run in, and they are usually wrong.

> By the way, "batteries included" is one of the BEST features of python.

None of these libraries need to be nuked from existence as a result of this change. I'd wager they'll move into PyPI modules so that teams relying on them could safely continue to do so.

> Have you tried to fix something in your house with a "homeowner's toolkit" which is usually something like a hammer, pliers, 2 screwdrivers, a putty knife and a few more basic tools? It is REALLY tedious, like writing a C program with a few basic tools like stdio and ctypes.

That's one extreme, but I don't think that's what's being proposed. The proposed model is closer to what Rust does today, where the core is slim, opening up new potential use cases, and the more complex functionality built on top of it is left to the community to maintain.

Take a look over some of the modules they're deprecating, like smtpd. What kind of standard library requires an SMTP daemon built in? That's akin to a homeowners toolkit including a planishing hammer [2] for some reason.

> I think if the cost of deploying a script is 1, deploying it + a dependency is literally something like 100x. You have to make assumptions about all the environments the script will run in, and they are usually wrong.

Depends on how it's done, honestly. Check out this Rust "scripting" system [1]. It has full support for third-party crates.

[1] https://github.com/DanielKeep/cargo-script

[2] https://en.wikipedia.org/wiki/Planishing

Sorry, my "batteries included" comment was independent of the changes proposed. I just love the rich library python provides (compared to other scripting languages).

I hope you didn't take my reply to be aggressive in any way! Standard library is always a point of contention in any environment and it's always good to explore all angles. It's kind of the ultimate bikeshed in a lot of ways. Python indeed is a very batteries-included language and there's a lot to like about that. I wonder if there's a way of preserving that by differentiating between a 'lite' and 'core' distribution so we can still meet the goals of the embedded / small system community?

> Take a look over some of the modules they're deprecating, like smtpd. What kind of standard library requires an SMTP daemon built in?

Just because it's not widely used doesn't mean it cannot be included in the stdlib. For instance, where I work, we have a MTA test framework that makes extensive use of smtpd to receive messages processed by our configured MTA which we then write to disk and subsequently make assertions on the message (e.g., contains certain headers, has certain recipients, etc.).

Having various protocols (client and server) in the stdlib is not a bad thing and, personally, I think it's very useful for testing purposes.

You couldn't set up a library dependency on smtpd or similar for your test project?

> API 101 - there is very little cost to leaving them in

That's not true as newcomers will typically bias towards bundled modules instead of 3rd party ones. If it's well accepted that the bundled modules are bad in some way then you are encouraging further use of them by leaving them in. That's real cost, and not just on the Python maintainers side of thing.

You can hide them or mark them deprecated in such a way that new scripts don't see them, but old scripts still function.

How exactly do you distinguish new and old scripts? They are all just files from the interpreter’s perspective.

that's a good question.

maybe you could make it explicit for new scripts?

    from __past__ import cruft

Just how is a beginner, or in fact anyone, supposed to go and choose the "right" 3rd party module, when it seems half the stuff in pip is either half implemented, or even just a hello world example.

An official list, containing only one library for each purpose, could solve the issue.

By simply asking that question you're already more likely to get a better result than if there was a known-wrong choice bundled into the standard library.

But really that's just a tutorial/discovery issue, which has many solutions. And not even a skill isolated to programming - how do you pick the "right" item on Amazon to buy?

C also doesn't have a standard package manager. I think that's the big problem.

Built-in functionality is less important if you can easily use packages.

> By the way, "batteries included" is one of the BEST features of python. > More languages need "batteries included", maybe like Perl.

Agreed with both points, the lack of batteries makes Lua quite annoying for me personally. Python and Perl are much more comfortable because you don't have to hunt down the most common operations.

Of course, Perl 6 has decided not ship batteries included, replacing batteries with a fusion reactor instead.

> Have you tried to fix something in your house with a "homeowner's toolkit" which is usually something like a hammer, pliers, 2 screwdrivers, a putty knife and a few more basic tools?

Sure. But did you read the list of proposed deprecations?

It's not like that homeowners toolkit becomes any more useful when it's also got a buggy whip tool, a set of special allen-like keys that only open the case of a TRS80 or a Commodore64, and one of those picks for getting stones out of horses feet.

Most of the proposed things to take out seem obviously "right" to me. AIFF sound file support? MacOS9 binhex? CGI? _Maybe_people are still using those things, but it seems to me like they're edge-case enough to let them know now that they'll need to do "pip install aifc" or "pip install cgi" sometime after late 2021 if they update their python installation (which they can choose to put off until at least 2024 or so, just like all of us who're still sometimes using Python 2.7...)

Yeah, I went into the article prepared for a "NO! Leave that in!", but I didn't realise just how much very old cruft was in there.

The approval process for getting libraries okayed for production is a massive pain in the ass, and for good reason. Without a batteries included language, it'd be easier to write all those tools in-house than hope that maybe in two months I can get a library out in production, even though it's just JSON parsing or something. The "this is an official part of the language and held to high standards, guaranteed to be suitable for poduction and needing no approval beyond already being approved to use the language" makes Python way more preferable IRL than languages I love, but don't even include a random number generator in the standard library.

It's also quite possible, that the people who use these features use Python 2 anyway.

Does this mean we're going to have Python 2 installed on things forever? (Particularly anything that touches NNTP servers, as there's no replacement for that lib)

Which lib?

The one for touching NNTP servers.

It's simple code. It can be moved to a separate package and imported with the same name.

If it's simple, then what's the point in removing it? Dealing with standard application level protocols like SMTP, NNTP, FTP, IMAP, POP, and HTTP shouldn't require installing third party packages.

So that they can be improved / maintained. "stdlib is where packages go to die" is kind of true. You can use contact http from stdlib for example, but that's how we get urllib, urllib2, 3rd party urllib3 and requests.

For the point of removing the modules, check the "rationale" part of the PEP.

The original post you responded to mentioned that NNTP did not have a replacement, so I don't see how it could be improved/maintained if it's just not there anymore.

> But, after the initial pain of the 2.7->3.0 transition, I doubt we'll ever see another major version number jump, even if a logical use of version numbers would merit it.

Do you think the transition would have been less painful if 3.0 had been called 2.8?

If Python 2.8 had come out and failed to run almost all code written for Python 2.7 then that probably would have been truly disastrous for the language.

some of these modules have actually been deprecated since before python 3.0

Even if they don't bump up the major version number, people are probably still going to treat it as such. I think it makes sense to bump it up.

Reply on news.ycombinator.com