Sunsetting Python 2
python.org
python.org
Dependencies? I helped upgrade a couple third-party modules, too. That's part of the job. If you choose to use a dependency, you're vouching for it. That means if the current version of the language no longer supports your favorite library, you need to fix the library, or find a new one. If we switched from phillips to torx, and your favorite tool brand didn't make torx drivers yet, you've got to either convince them to start, or switch brands. Professionals don't get to use this as an excuse to badmouth torx, and stick with phillips.
If people spent half as much energy upgrading as complaining, this would have gotten done 5 years ago.
Apple took only 3 years to go from first announcing a new CPU architecture to releasing an x86-only OS. 2 years later, Rosetta stopped working, so there was no way to run old apps on current hardware at all. That's got to be one of the advantages of proprietary systems. It's amazing what you can accomplish when you have no choice. People complained a little but they got it done.
This is the best analogy I have yet read. Thanks - you nailed an argument I have had at multiple gigs/clients way to often.
> Professionals don't get to use this as an excuse to badmouth torx, and stick with phillips.
D'accord. Exactly. I had very fruitful discussions with professional builders about tools, tool brands and their respective strengths and weaknesses - but never has a professional decried Bosch Pro vs Makita vs DeWalt. One has ones favorites - and that is fine. But badmouthing - I have never encountered.
The switch to py3 is more akin to using a new building material. Easy to start with once you know the differences, but it’s a bitch to retrofit if you depend on a method of construction incompatible with it.
“Go change the dependency” is a naive ideal. Most non-tech businesses have no budget for that kind of tech debt work.
If you don’t push your clients/employer to have room in their budgets for these types of things, you are the one being naive.
The thing about debt is that ignoring it doesn't make it go away magically. If business relies on tech for crucial functions, it's only a matter of time before debt comes due.
That's fine. I'm sure there are some that also don't have a budget for fire insurance, or a security system, or whatever. For anybody who wants to gamble with no fire insurance, or not fixing tech debt, that's a conscious choice they get to make. Now they have to be responsible for the outcome(s) of their choices.
Congratulations. You're fired.
"Hey boss, we're using an outdated programming language for which security updates are no longer provided, because we were told there's no budget for fixing technical debt."
"Congratulations, you're fired."
Same outcome either way... might as well make some effort to do the right thing along the way...
Python2 will almost certainly still remain in widespread use and it will still get security updates through alternative distribution channels. It's not that much work fixing such relatively rare issues versus migrating a major codebase.
But hey, if some 3rd party group effectively forks the language and maintains a Python2 branch after the official EOL date, good on them. But I'd argue that using that is risky in the same way that using a Pale Moon or Waterfox is (arguably) riskier than using mainstream Firefox.
Security through obscurity, baby...
Python3 broke compatibility for spurious reasons. The language didn't change that much, it could have kept compatibility with minor concessions.
> Java or COBOL maybe have done a better job of not breaking backwards compatibility...
Literally every single other major language has done a better job at it. That's a big part of why they are major languages. Also, most of these are statically typed, so any minor breaking change is far less risky to pull off.
> ...but Python 3 isn't exactly Perl 6 here.
Well, maybe if Perl6 didn't happen Perl wouldn't be irrelevant today.
C++'s upgrade woes make Python 3 look downright trivial in comparison.
If you're talking about ABI breaking, that's a different story. Python breaks the ABI in every major release as well. That generally requires a recompile, not a major code migration.
For what it’s worth, Perl 6 can load Perl 5 libraries: https://github.com/niner/Inline-Perl5
People will continue to use py27 going forward with likely a pay-for-support runtime because that’s cheaper and less risky than changing a million lines of python code.
Like it or not, there are a lot of teams that choose to incur technical debt to avoid what is seen as needless work on something that is working fine at the moment.
There are many maintenance mode projects where the switch to 3 represents absolutely no upside and is all downside risk. Not everyone is developing web apps in constant flux that are exposed to the Internet. That is a tiny fraction of python’s use cases.
If you built software using Python 2 at any point in the past 5 years, you have no right to complain about the added technical debt of updating to Python 3. But not wanting to incur that technical debt in the first place would also have been a good reason not to use Python 2 in the first place; whether that means adopting Python 3 early or simply using a completely different language.
A security system and fire insurance is at most around $2000/month, and it's a cost that is understood and often required by law. Diverting a $100k+/year employees to spend their time on tech debt is quite a bit more expensive.
Companies rightly try to minimize time devoted to tech debt.
Every non-tech business that maintains technical infrastructure finds itself doing that kind of tech-debt work regularly (if, often, chronically behind the vendors recommended schedule) for every platform they use Py2 to Py3 may (or may not, really) have been a bit more painful than a (multistep, because those types of shops never do it timely enough that a single-version-bump makes sense) .NET framework, Visual Studio, SQL Server, and Windows Server upgrade (and, yeah, those often happen together as multistep upgrades in slow enterprise shops), but it's not a fundamentally different kind of problem (and replacing dependencies that didn't make the same upgrade, often because they were abandoned years before the firm made it's upgrades, is often a part of that.)
I don't think there's a time I can remember when Norm Abram (probably now one of the greatest or at least most prominent carpenters of a generation) ever mentioned the particular tool brands he was using.
And looking back on earlier seasons of This Old House, he was hand-nailing thousands of nails a day (nowadays he and everyone else often uses pneumatic tools for this).
I'm sure he prefers a certain brand, but unlike popular YouTube personalities and more 'celebrity' builders, he isn't a brand ambassador for Craftsman, DeWalt, Milwaukee, Ryobi, etc.
But yeah, when you talk about 'builders' (the more common carpenters you encounter on most job sites), it seems like most of them will die on a sword defending whatever brand(s) they've sunk a fortune into.
I'm a DeWalt guy, mostly because I invested in a 20V Max drill a long time ago and have a ton of batteries for all my electric tools now. But I used to use Makita before switching systems. I wouldn't call myself a master carpenter by any means, but like AvE, I find good and bad in almost all the 'top tier' lines from tool manufacturers.
You want it written in C for a 16bit bargain basement ALU? I can do that. You want two implementations, one in Pascal and the other in PHP? Not a problem. You want my advice? No? That's fine too, the advice is expensive and who knows after all these years if it's even worth anything.
At my first job we used to build some hydraulic models in hard wood that where works of art.
Our wood shop was so good they made a museum quality piece of furniture for our retiring boss and it was better than the best Chippendale furniture when new.
I do like the grips and ergonomics of some brands more than others.
The one piece of concrete advice I got was “Whatever brand you get, go for the 20V, not the 12V”. Last week I saw that same guy at work showing off his 12V driver. So I’m pretty sure nothing matters, and I’ll probably just get the brand with the prettiest color.
Good for you. Some of us had code bases of considerable size and complexity though.
The fact is that for working software on the python platform, this upgrade represented work that had to be done that for a legacy app that was still chugging... little benefit. If you already coded around the python 2 limitations for Unicode eg, then python 3 was not a help at that point for something already in service. It was just more cost for no benefit.
> If people spent half as much energy upgrading as complaining, this would have gotten done 5 years ago.
These aren’t exchangeable. Why do people on the internet think bitching (either constructive or not) is some valuable currency? Often times the people complaining had no means or position to of the work. 4 promotions later I sure as hell wasn’t going to participate in that 2 to 3 mess on a project produced years ago, but I could still opine in the situation. I also had little incentive to fund it.
Your analogies are bizarre. No idea what you’re trying to convey with philips v torx but I’m going to wager it’s explanatory power in this case is shit anyway. I can still demolish it, having actually worked in manufacturing there were times we told a supplier (ie Python) to fuck off and piss up a rope because what they were proposing was not compatible with our existing tooling and it would be too costly to convert for little benefit to us.
I respect apples prowess in the consumer space, but there’s a reason you don’t see their products regularly put into industrial roles where your timeline is more than 5 years. Apple products are disposable, many applications in industry are expected to last. Python is a general purpose programming language (or at least billed itself as such). Your comparison is poor.
Name one.
Er, why not? It's not like there's some kill switch in Python 2 that will make it stop working after January 1st, 2020. If it works now, then it'll still work, you're just not guaranteed fixes anymore. At least, not for free. As stated in the article, paid support options exist from several vendors.
> PSF
These are two very different kinds of institutions!
People who need COBOL support from IBM are paying a lot of money. Giant piles of money can get you many kinds of help that people won't volunteer to do for free... among them, maintaining ancient software in amber.
If you need Python 2 support and you are willing (and able) to pay the kind of money that IBM's customers pay for COBOL support, you'll be OK. For a start, Red Hat (aka also IBM!) shipped Python 2 in RHEL 8, which means they'll be supporting it until 2029 at the earliest.
In the vast minority of cases.
Not in most electrical engineering (except power plants), nor in computer engineering. 60 years ago was 1959. What software/computer project from that time is still running?
Even moving outside of the electrical domain, how many physical products outside of civil engineering is expected to last that long. I certainly can't expect support for my car for longer than 25 years.
But I get your point, those are special cases for preservation. For mainline things, especially with today's processes for continual integration, dependency checking and advance build tooling, dependency rot is something that should be accounted for in all software project plannings. If your dependencies are a few months out of date and you don't have the time to update them and re-run tests (people write tests right?) things are just going to hurt more and more later.
I've yet to see a CPA adequately depreciate software assets, anywhere. Maybe we should help them out by building better depreciation schedules. There seems to be a lot of CPAs that don't believe software depreciates over time, and maybe that's the largest disconnect in labeling it "tech debt", because accountants hear that term and think they can ignore it on a balance sheet they don't care about, but "tech depreciation" might actually scare them straight (until they find the tax advantages in calling it that).
In a previous job that was mostly in an engineering/manufacturing department, but with a lot of Perl/Python automation scripts, we had an internal conference. One of the keynote talks was when not to automate using SW. It went into the cost of maintaining SW over the long term - including the fact that authors leave, and people who understand their scripts require higher salaries. Most people who write these scripts are not hired for SW roles, so their replacements likely cannot debug/extend.
Isn’t that comparable to java? Oracle gives you paid security updates on version 8. You can keep using the previous version without security updates or migrate to OpenJDK 8, which have their own security group. More options, seems better to me.
I think in this case at best the analogy grossly oversimplifies the issue. If it really were just a “screwdrivers” problem as explained then the 2 to 3 migration would have been mostly trouble free and would have happened. Clearly it did not go that way so that analogy can not possibly be appropriate.
If you haven't put in the priority to update your Py2 to Py3 apps by now, I really think your shop has the wrong priorities.
It's not just about Py2/3. Dependency rot is one of the worst form of technical debt. It often shows broken CI, broken security scanning, lots of generally broken processes that will just keep hurting a team further and further down the line.
CG/Visual Effects industry is still firmly using Python 2. Only in 2020 are they taking the first step to transition to Py3 [1]. Users and studios are held back because the Python runtime is used inside major applications; Nuke, Houdini, Maya as well as libraries and APIs. None of them have released a version that runs Python 3 yet.
The reasons for delaying it (mentioned in a footnote on that page) makes sense to me. Previous years were focusing on coordinating updates to GCC, Boost, C++14, Qt, and waiting on Python bindings for Qt.
Also, I've worked at a couple studios many people have probably heard of and none of them have unit tests covering much of their code. The focus is on tools that facilitate in-house artists where responsiveness to needs are valued over architecture and completeness. Requirements change for each project and previous requirements are often sacrificed (until a new project needs them in a few years).
I'm itching to move to Python3, but even for standalone tools I've felt it better to choose a completely different language (or Python2) instead trying to mix Python2 and 3 because having them co-exist creates more headaches in managing the environments, dependencies, and coding styles.
I am not sure how you could possibly know this. You have no idea what else they were working on instead.
I didn’t think it was the most amazing comment either, however, it was intended more to illustrate how things are rather than how they ought to be, I think some interpreted as a strong opinion in favor of the circumstances which was not intended. Based on how it scored (somewhat surprisingly) it clearly resonated with more than a few.
1. Python core dev pretended Python 3 was good and ready by, like, 3.1. It wasn't.
2. While your problem may have been painless (I'm glad), that doesn't mean that everyone who complained was just complaining.
(I use Python 3 as the default now, but as someone intimately involved with an async IO library at the time it came out, I maintain that a) the transition was botched b) core dev did not listen to any of the problems people pointed out for _years_.)
To give you an idea of what botched means: several people, names withheld to protect the guilty, had to wade in 6 months worth of shit to get a 2.x release that made TLS vaguely OK, in, like, 2014?
I would expect any stable release of software to be "good and ready." Can you explain what was wrong with Python 3.0 and 3.1?
> core dev did not listen to any of the problems people pointed out for _years_
What problems were those?
I appreciate the general point that that's far more recent than Python 3s initial release but it's also a large exaggeration to say "until this most current release".
Also to understand why it took so long is because Python 2 kept improving after Python 3s release. Python 2.7 was a great release and many further features from Python 3 got backported in it's 2.7.x point release.
[1]: https://bugs.python.org/issue19977
[2]: https://www.python.org/dev/peps/pep-0540/
[3]: https://docs.python.org/release/3.5.0/whatsnew/3.5.html
3.5 fixed % formatting for bytes, further making 2/3 transitions easier (or harder, depending on your use/abuse of strings vs bytes).
3.6 added the first 'exciting' new feature: f"formatted string {literals}" if you don't have an asynchronous type project that can make good use of async/await.
(Or in language-lawyer terms: CPython 3.6 introduced this behavior and documented it as an implementation detail, and for Python 3.7+ it's guaranteed a feature of the language proper.)
% python3.7 -c "import sys; print(sys.argv[1])" "$(echo -e '\xff')"
Traceback (most recent call last):
File "<string>", line 1, in <module>
UnicodeEncodeError: 'utf-8' codec can't encode character '\udcff' in position 0: surrogates not allowedThe key surprising thing that's going on here is this clever hack (clever, but a hack):
> In Python, file names, command line arguments, and environment variables are represented using the string type. On some systems, decoding these strings to and from bytes is necessary before passing them to the operating system. Python uses the file system encoding to perform this conversion ... > > On some systems, conversion using the file system encoding may fail. In this case, Python uses the surrogateescape encoding error handler, which means that undecodable bytes are replaced by a Unicode character U+DCxx on decoding, and these are again translated to the original byte on encoding.
https://docs.python.org/3/library/os.html#file-names-command...
This is meant as a way of fudging the fact that (a) for UI purposes, you want to treat filenames as text strings; (b) your Linux filenames are probably all encoded as UTF-8 (or your locale encoding); (c) but they might not be -- they could be arbitrary bytes, except only NUL; (d) and if they are, you really want to not munge the name when you go back and try to operate on the file.
The fudge is that filenames get decoded as (by default) UTF-8... but if invalid, the offending bytes get stuffed into the UTF-16 surrogate space. Then filesystem APIs encode as UTF-8, except they look for that surrogate hack and turn those to the original bytes, so it all round-trips.
It goes pretty wrong if you try to hand such a hacked-up string to something that just expects to encode normal real Unicode with UTF-8, though. That's what's happening in your example.
The magic words are `os.fsdecode` -- that's how you get back bytes round-trip clean from that hack.
Just let it go...
IIRC the messaging from Guido and Python core dev circa PyCon 2008 and 2009 was that it was good enough for people to start thinking about how they might migrate. This resulted in experiments with different approaches to migrations, maintenance of versions for Python 2 and 3, etc, but at that time there wasn't a clear end date and no rush to migrate. It seems like a lot of people misinterpreted what was actually said. Similar misinterpretations seem to happen within the wider internet community around the GIL and perf considerations. That cliff is finally here ten years later after a lot of experience with migrations.
The post you're responding to starts verbatim with `Python 2 to 3 (at least by 3.3 or so)` and your reply is about Python 3.1, released 3 years earlier. The amount of changes was quite big.
That seems like a very specific example...
_Everything_ I relied upon, every single dependency in my projects, gradually migrated over the course of a year and nothing broke.
Furthermore, I now have asyncio, aiohttp and friends to play with, and it's been pretty good.
Also, your screwdriver analogy makes no sense. The choice here is not about "your favorite tool brand". Mixing and matching tool brands is straightforward. The more appropriate analogy with screwdriver heads would be to imagine that you now need to change out all your hand tools, manufacturing lines, assembly robots, and equipment from multiple vendors from phillips to torx all at once. Sure, you can do some prep work, but you've got to coordinate a cutover at some point, in sync with tools that you have no control over or have to recreate from scratch. All for zero benefit in the end, other than the hope that they don't make the same mistake again with Python 4.
Also I think you're too harsh on the analogy, most of the time there's a way to write python2 and 3 compatible code, so which means the transition can go smoothly.
I’m not sure what more they could have done. Python 2 was released almost two decades ago, with an original planned sunsetting that got pushed back to 2020. That’s plenty of time and respect for their users.
>All for zero benefit in the end, other than the hope that they don't make the same mistake again with Python 4.
You’re free to stick with python 2 for as long as you’d like. You’re free to fork the project and continue develop it. To say there is no benefit to new and evolving languages is odd and short-sighted.
Not broken 2? None of the breaking changes in 3 were necessary.
It's tedious yes. Boring even. But most projects get away with 2 weeks of investment. And yes, it pays back. Python 3 is a vastly superior language when it's about introducing less bugs or debugging existing ones. It's just not stuff what had be sold to the public. People want to hear about better perf or fancy features, not better error messages or banned logical errors.
The hardest code base I saw ported was the Twisted project. It's a good counter example: it was long, hard, and required very gifted people (thanks Hawkowl !).
But most projects are not like that. They are full of dumb operational logic that can be ported 80% by 2to3, and a bit of manual fix.
The fact is, people love to complain, so you will hear massively people with that particular example from the movie industry, or this guy who coded this Fortran extension that was in such a tough situation. Well guess what, that's not what most migrations are about.
Most migrations are Django/flask websites, math teachers exercises, physicists scripts, sysadmin tools, etc. Straightforward stuff. I should know, I've been moving from industry to industry for 10 years, all my clients are different, but most of them end up with similar stuff in their repo because Pareto is a thing.
Nonetheless, the screams from the first ones scared the later.
I'm not saying we would've cured cancer if people didn't have to deal with this bullshit, but it's definitely a setback.
Given the reach and popularity of Python, of course you will always find plenty of testimonies saying they suffered. Again, it's important to remember you can't make everybody happy, especially if your job is important. If porting is easy for 90% of projects, you did a great work, period. 10% of millions of users having it hard is still going to raise a lot of voices against you by the sheer power of numbers, but you did ok.
There is also dishonesty. Most people reporting this problem or that problem didn't actually encounter it. They are reporting somebody else experience they heard about because they wanted to make a point. Or vaguely tried something and ran away after they saw 10 minutes of fiddling didn't solve it.
In 10 years going from company to company, I met only 2 projects that were hard to port, but much, much more complainers. When I looked at what was really going on, the fact was just that they were afraid to port, and so just repeated all the stuff other people told them would go wrong. Or they tried running a few scripts for 5 minutes on Python 3 and gave up seeing too many error messages.
I know there are honest situations in the lot. But again, the most honest people are not making noise, because they just ported their code and noticed it wasn't the hardship they've been told.
...but the last to actually finish migrating. In fact, it's still ongoing.
> Given the reach and popularity of Python, of course you will always find plenty of testimonies saying they suffered
Everyone suffered. Everyone had to deal with Python2 versus Python3 bullshit. It's not just about migrating some codebase.
Everyone had to use an inferior version of Python, whether it was because library choice was limited (Python3) or because the language was intentionally left without feature backports (Python2). There was a huge amount of plain "brokenness" in the ecosystem and it was a huge waste of everyone's time.
Conda ships on 3.7, and I'd consider it the definitive "scientific" stack.
Updating software version is part of the job. Like creating tests, writing documentation, training newbies and dealing with customers. It's not hard just because we don't like to do it. Porting to Python 3 was not hard for most people. They just really, really didn't want to do it.
I get it. I didn't want to add that on my plate too. Like I didn't want to migrate from my Centos 7, I didn't want to move from mysql to postgres and I didn't want to learn the entire setup of Webpack. 3 times. But "suffering" is a big word that has no place for the vast majority of projects.
You are stuck with a tough choice: Do I start out with Python3 and tons of broken packages? Do I limit myself to Python2 and face a costly migration later on? Do I run the extra cost of supporting both? This the choice you had face for the better part of ten years of migration. Perhaps it's not obvious that all Python-based software was worse for it, but that's what happened.
I'm not talking about some web backend service where most of what you do is trivial stuff. You can write and re-write that in almost anything, it doesn't matter.
Dynamic typing on the language side, and a less than perfect test suite on the user side are not a good combination for large projects facing a project wide migration.
The existence of the cloud means that you can scale wide instead of scaling tall, which makes developer time and complexity management more valuable than hardware efficiency. Python is optimized for primary applications in exactly that kind of environment, where the performance characteristics of something like C++ don't make up for the clumsiness of the language.
In my very brief experience with python over the last few weeks this is very common. Half of our dependencies were abandoned before python 3 existed, when mercurial shuts off the hg we'll even lose the source to some of them. Python 3 get's the blame but the real problem is that the company has ignored maintenance for decades, it's entirely there own fault. There's no "business value" in maintenance until the whole lot needs to be rewritten.
Another underlying cause is package tools like pip, they make it too easy to take on dependencies with zero thought given to their maintenance windows and management think the can just ignore upgrading/replacing them regularly.
Are you perhaps thinking of when Atlassian removes the Mercurial repositories hosted on Bitbucket?
Mercurial isn't going anywhere or shutting down anything; it's open-source software with some very large users and committed developers.
(The majority of engineers at Facebook all work in one enormous Mercurial repo. Facebook naturally employs a few people to work full-time on Mercurial, and I don't think they're the only ones.)
Atlassian's decision about Bitbucket is regrettable. Especially regrettable is to actually delete repos for so many open-source projects that may not have active maintainers (rather than keep them online but read-only.) To my mind it marks a stain on their reputation that should make anyone think twice about relying on Atlassian for years to come.
But you should be able to avoid losing the source to any of your own dependencies. Between now and May 2020, go through all your dependencies and make sure you make your own clone of all the source repos.
The original programmer is long gone, and I while I am a semi-competent Python scripter and a slightly above average C programmer, I'm not looking forward to having to dig deep into this code, and patch it up to keep running.
Where did you come up with this "2 weeks" estimate? That has not at all been my experience.
Many months.
Step zero is education. Your team has been programming in Python 2. You need to make sure they know the differences in the environments, how to use create cross-compatible code, and how to use six. They also need to setup a second dev enthronement, and become comfortable switching between 2 and 3 regularly.
Step 1/2 is prioritizing. Determine how important is the move to Python 3, and what other features and dev work will you have to sacrifice to make it happen. While having a nice plan in place may provide some level of comfort to management, you can be sure it will thrown out, amended, extended, and/or ignored throughout the project. Upgrading Python dev environments brings no near or mid term value to the company. So expect developers to continue their current work-load while also attending to this tech debt.
Step one is getting the libraries into shape. This is easy if your project only relies on actively developed libraries with good teams behind them, but nearly impossible for abandoned libraries with no new commits in the last few years. Sure in retrospect, it was a bad idea to use these libraries, but at the time, they were incredibly powerful or popular. Generally, for abandoned libraries, it's easier to find a newer Python 3 library than work on fixing someone else's code. But this may mean rewriting large parts of an app, and may alter functionality, so constant communication with product mangers and a flexible approach is necessary.
Second step is working on your own code. One or two modules is no problem. But more than a dozen takes time. Scripts like 2to3 are not helpful, you need to use tools like six and modernize.
Third step is testing. If you have a good team, then you should already have good tests with known coverage. In that case, you should make sure your coverage matches, and take a very hard look at the code that is not covered. If your tests cover less than say 60% of code, you need to invest quite a bit of time of either testing directly, or building many more tests.
Step 4 is partial rollout. There will inevitably be issues you didn't think about or catch, so you need to plan the roll-out carefully and either split traffic or at the very least be able to quickly revert.
Step 5 is to watch carefully for customer complaints. Any complaint may or may not be related to the switch from 2 to 3, and you need someone on your team who stays on top of that, and can communicate the issue to the correct module owner.
Step 6 is deciding when to drop support for Python 2 altogether, as there will have to be a time where you have the two environments running side-by-side while you're testing. After all of this work, you'd think this part would be easy, but in every organization some folks will be wary about dropping support for something that already works.
Obviously, this isn't just a linear progression, you can do some of these in parallel, and will likely have to take a step back a few times. My back-of-the napkin, experience based, non-scientific estimate suggests that a 100k LOC project can be managed by two developers in two-three weeks, but a 1M LOC project (including libraries) is roughly 5-8X that much work.
I suppose you don't use Java.
The Apple comparison is a bad one since Apple controlled the whole thing.
Isn't what code review and CI/CD is for?
As a manager one should be extra careful to be _seen_ eating your vegetables, flossing your teeth and getting your code reviewed.
(What you do in private is a different matter. In some companies office politics can require some skullduggery.)
I wager that in many cases, fixing something that isn't broken never reaches the top of the TODO.
I think many people underestimate the challenge that the 2 to 3 migration presents for large enterprises. The core issue is that even though the migration for any given module is normally really easy, the total effort required to migrate is still essentially O(n) in module count/file count, because even with current tooling you still need to have an engineer look at every module to do the change safely. Even if it only takes ~5 minutes per module to make the changes and validate that it works correctly, this becomes a giant undertaking when you have tens of thousands of files to migrate.
The fact that it takes a long time also creates other problems. Your business isn't going to hit "pause" on other development, so there will be changes constantly introduced into modules you've already "swept". It's going to be hard to make sure 100% of your engineers and code reviewers are knowledgeable about the specific requirements to make sure the code works in both 2 and 3, so you would really like some automated safeguards to make sure they don't introduce anything that won't work in 3. Pylint helps with this, but won't catch everything. Unit tests are obviously essential, but:
1. Even a well-tested project won't have tests that cover 100% of code paths and behavior.
2. You're stuck running the tests on both python2 and python3 for the duration of the migration, which doubles the resource (compute, memory, etc.) cost of your Python CI and regression testing infrastructure for the duration of the migration.
I see far too many commenters attributing the delayed migration to laziness or complacency. In reality, most big companies have passionate Python advocates who really want to be on Python 3, but the scale of the problem and the lack of tooling to tackle it with a sub-O(n) amount of effort make the overall project risky and expensive for the business.
Compiler errors sure would be handy here :^)
> If people spent half as much energy upgrading as complaining, this would have gotten done 5 years ago.
Yeah, right. Look at the creator of Python himself. Took about 3 years to migrate just his employer, and that's a top-tier software shop [1].
I've heard many people talk like you over the years, always being dismissive of the cost to others in python 2 vs 3. The complaints against the migration were not unfounded.
1: https://blogs.dropbox.com/tech/2018/09/how-we-rolled-out-one...
Contrast this with the Postgres website, which tells me I'm browsing old docs (because Google still offers old links), see e.g. https://www.postgresql.org/docs/9.2/tutorial-window.html
Are there plans to add a banner to Python 2's docs? I think it would be quite instructive.
As a casual python user, it is quite common to stumble in some issue, after somme googling land in a doc page, try my solution, and find it does not work. Then googling after the failed solution, noticing that the previous doc page was describing a Python 2 feature that do not exists (or works different) in Python 3.
I just wish they'd named it something different. If there's neither forward nor backward compatibility, I'd argue they're not the same language and should not have the same name.
https://perldoc.perl.org/perlpolicy.html#BACKWARD-COMPATIBIL...
Also they are color coded.
I'd support that if every Python 3 page had a link saying "if you're still a sorry SOB stuck on Python 2, here's the link to the related library." It's just as bad for Python 2 developers, it's roughly a 50/50 shot of whether you get docs for 2 or 3.
Or email the mailing list for more help?
- https://github.com/python/cpython/commit/46ed90dd014010703c7... - https://github.com/python/cpython/pull/13638 - https://github.com/python/cpython/blob/master/Doc/README.rst...
And because of changes in PostgreSQL versioning, people don't release that the oldest support version (9.3) is rather old.
PostgreSQL 9.6 was released in 2016, 10 in 2017, and 11 in 2018.
But PostgreSQL 9.3 was released way back in 2013...not quite as old as Python 2.7 but close.
In 2019, I feel pretty confident about using Python 3, having used it exclusively for about 18 months now.
For my personal use case at least, this timeline worked out well for me. Hopefully it works out for most everyone. I can't imagine they made this decision without at least some data backing it up.
Python is an open source project I've used and contributed nothing to, so I don't have the right to be a back seat driver. Were it a commercial project and I was a customer, I would be quite upset.
If the deadline wasn't 2020 but 2024, then you wouldn't get more time, simply we'd be at "not yet ready to migrate" state right now, as major libraries would not have switched yet.
Library, as is most open source libraries out there, sure.
For a big company-internal Python codebase that also has to wait for all the major libraries to migrate first, that time frame can quickly shrink to 1-2 years, which is very little.
Beggars can't be choosers; if your company wants or needs major libraries to have migrated 4-5 years before the deadline, then your company has to participate in making that migration happen. Or accept that it's going to be done later than you'd like, because the needs and motivation of these library maintainers are quite different from the motivation of Python core maintainers.
“The free libraries we chose to depend upon haven’t migrated.”
Do you have programmers?
“Of course.”
Put them to work.
“We have other priorities.”
Well then. Your choice, your outcome.
I'm supposed to go and tell all the various projects that I depend on that they have to change their upgrade timescales to suit mine? Even if I had enough time to help them all meet that kind of timescale, why would they agree to fit their changes into my required deadline?
Maintainers are under a lot of pressure. Having users come along and say "we'll add 50 developers to your project if you agree to ship v3 compliance by next month" is not reducing that pressure, it's adding to it.
More programmers != better results or faster delivery. Your solution is just going to create more problems.
Programmers ~= Resources ~= Money
=> If you want your dependencies to be upgraded in a reasonable time frame, try to contribute to financing the maintainers (or support them in a manner _they_ find suitable).
At no point do you have to wait on anyone else, you choose to.
The Python 2->3 window is an example of where being too nice to too many users harms the project. Once Python 2 is dead, people will choose between being the sole maintainer of a Python 3 fork of some Python 2 library versus a being a sole maintainer of their entire Python 2 stack from the language layer on up.
Still better than my last programming job where the vendor of a vital piece of HW had provided a C++ API (also vital), but refused to provide dlls that would work in anything newer than VS2010. At least with open source you can do something about it.
Sure, they will have to cope. One of the ways people cope is by accepting security risks and continuing to run with old versions. Another is to decide that Python isn't for them, and move resources toward better platforms with commercial support for older code.
The fact is, that until a few years ago there were too many libraries that were not compatible for most organizations to make the switch. So, effectively the window for those organizations has only been a few years.
I think he's blaming organizations, not engineers.
I also think organizational dysfunction is a problem that isn't a technical problem that the Python project has the capability or duty to solve.
No one is doing that.
OTOH, it's not PSFs job to protect developers from bad management.
If there wind up being major security issues, there will be little choice but for someone to take up the mantle and fix them. Because companies aren't going to switch, and when you've got a botnet ravaging the internet they can't flip a switch and do a big-bang rewrite, someone will have to push out a patch for them.
This isn't the end of Python 2, it's just the Python team washing their hands of it. Ask PHP or Cobol users how their attempts to kill legacy codebases worked out for them.
This whole endeavor was complete folly from the Python foundation to begin with. A big-bang rewrite because they didn't like the syntax, and they didn't even fix the GIL while they were at it.
Personally, I would guess that many organizations will be running in Python 2 for at least parts of their codebase come 2020.
A lot of devs like to not worry about new things.
I had to manage a mature codebase upgrading Ruby versions, and it was painful. And that is fairly mild by comparison; not the big jump from Python 2 to 3.
Is it though? Especially since as late as 2014, everyone was under the assumption that in 2015 security support would end. We were already trying to migrate long before that. The five year extension seems pretty reasonable to me.
Were I a commercial customer, and had been told to switch to the new version ten years ago, then it's on me if I still have started the migration yet.
This sounds a bit like telling someone, oh, you could've easily started driving an EV, so that once charging stations were actually out there, you'd have been ready...
While car analogies are notoriously bad, it's much more like "I'm going to buy a plug-in hybrid and use gasoline until charging stations are widely available". You can write code that works in Python 2 that takes zero effort to run in Python 3. I (and many others) have been doing this for years. Just look at the number of packages on PyPI that run, unmodified, on 2 and 3.
[Edit: gas -> gasoline]
It's still hard to get bilingual Python correct today, and it was considerably harder before 3.3.
This is simply untrue— there are a dozen small stumbling blocks that mean that writing bilingual code has a real cost. And it's not as easy as just having the right CI setup, especially if it's ten years ago and most of your dependencies haven't migrated.
Q: Ten years ago how would you have explained to your manager that you needed your team to start writing 2-3 compatible code, which obviously takes considerably more time and effort than sticking with 2 compatible code?
At this point, all the code I write is 2/3 compatible, and the things that annoy me are all py3 only features. Backwards compatibility stuff is pretty easy. Most of the easy problems are easy, and six has solved all the hard ones for like 7 years.
And I've been writing some of my code 2/3 compatible since 3.4.
import subprocess
import re
import yaml
output = subprocess.check_output(['ls', '-l'], universal_newlines=True)
matching_lines = []
for line in output.splitlines():
if re.search(r'total', line):
matching_lines.append(line)
with open('matching_lines.yaml', 'w') as f:
f.write(yaml.dump(matching_lines))
Python 2.7: https://repl.it/repls/OrchidDirtyNotification
The much worse case of this that I dealt with a few years ago was reading in and modifying XML files, and then saving the modified XML contents as strings in a yaml file (a keyed cache). The input XML files (which I didn't control) were not consistent about having the encoding marked, and that really made things a mess for ElementTree.
A taste: https://github.com/ros-infrastructure/rosdistro/search?q=utf...
I know that's not the version of Python you were targeting, and I know you didn't write this program to be cross-platform, but I wrote a similar program and I was repeatedly surprised by those sorts of problems. I don't entirely agree with mikepurvis, but I do feel his pain.
You can write code that's compatible with both Python 2 and 3, but I don't think you can expect to be using Python 2 exclusively and write code that works well in Python 3 unless you're constantly running it with Python 3. Especially, if you don't have much firsthand Python 3 experience. This is much easier if your code is isolated from those dependencies and you write tests that run in both Python 2 and 3, but that's quite a bit more of a commitment than you seem to be implying.
I get your point though about not being able to run full integration tests due to dependencies. Still, if you were making a good effort to keep the code Python 3 compliant, it would make upgrading much easier.
Also, my other suggestion- contributing to Python 3 compatibility for modules you depend on! That will give you Python 3 experience :)
- 2008-2012 Python 3 becoming usable (byte formatting, six library)
- 2012-2016 Libraries (Django, etc) becoming compatible
- 2016-today Applications (Trac, Ansible, Chrome[1], etc) becoming compatible.
[1] https://bugs.chromium.org/p/chromium/issues/detail?id=942720
https://github.com/getsentry/sentry/issues/8806#issuecomment...
https://github.com/getsentry/sentry/issues/1152#issuecomment...
I mean, even supervisord is migrated at this point, and it was a long-time laggard.
Actually, I can't wait to remove it from my hard drive on Jan 1, since it's one of the most poorly designed apps I've ever used. It's really not surprising at all that they're not able to migrate it, assuming the quality of the app is indicative of the quality of the code. I wouldn't be surprised if the same was true of a lot of applications that are refusing to migrate. (I'll probably have to find an alternative to ebook-convert, the command line tool that's the only part of Calibre I use, or maybe I'll rewrite it in Python 3 myself.)
> No, it doesn't [need to convert to Python 3]. I am perfectly capable of maintaining python 2 myself. Far less work than migrating the entire calibre codebase.
That's a very strange quote. I want to have this much self-confidence.
The gui, Calibre proper, is a total nightmare. Functional, but my god is it esoteric and ugly. I've heard people actually use that gui as their primary ebook reading software and I just can't fathom how they find that tolerable.
https://manual.calibre-ebook.com/generated/en/ebook-convert....
Actually, depending on what action you pin dates to, it was twice as long.
One other thing - you can’t run non-64 bit apps anymore. You can still run Python 2.
I feel like this the python community has been the epitome of class regarding communication during a transition.
There are an increasing number of applications which require .Net 4.7, we found one early this year.
Promising "10 years of support" for a product is great, but if/when other vendors decide to undercut this, you can still be SoL :(
Mind that there is a feedback loop between data of current usage and an eol announcement.
If they announce an EOL close by this creates pressure to migrate.
You can still get Microsoft Windows XP support today. If you're prepared to pay enough.
I wonder how many of the people who're upset by this have ever or ever worked for companies that've ever sent anything more than pocket change to the people Python?
You could work on porting them or be part of that effort?
Where they only release security updates and no breaking changes?
There is also the case of Oracle, which defines support as "you can read our knowledge base and we will answer a phone before telling you it can't be done", meaning their support is literally for life (of the company). And they still manage to screw it up by deleting old KM articles...
Everything else in IT goes towards shorter and shorter support windows, because the field is always in flux and change (or rather churn) accelerates every year.
At the company I work at, we've got a large amount of Fortran code from the '70s that works fine with the latest and greatest Fortran compilers (barring deprecation warnings which help guide refactoring efforts). Same with C/C++ from the '80s.
Food for thought, anyway.
Well, isn't it the benefit of FOSS, that volunteers can, and in the case of such a critical piece, so much used as Python 2, in all probability will, step up. Doesn't have to be the same people as the core team if just security fixes are involved...
Not to mention paid FOSS developers at places like RedHat, who want to keep supporting their LTS and enterprise customers...
Python 2.7 was released 1 decade ago. That's quite a long transition time.
More importantly, it would have stopped piles and piles of bleakly futured py2 codebases being written. That people were still writing new py2 code five years ago is terrible, and the long sunset is certainly greatly at fault here.
That people are still writing new py2 code now is just criminal.
Not everywhere had that foresight.
Also, macOS _still_ only ships with Py2. If I'm writing a quick script for my girlfriend to make her life easier, should I target Python 3 and force her to learn how to install xcode so she can learn to use homebrew so she can install Python 3, or should I just target Python 2 and move on?
Now many important libraries have already moved to being Python 3 making the whole Python 2 ecosystem feel frozen.
ANSI C is very different to the whole pre-ANSI zoo of C-like languages. Most of the code I saw from that age would never compile on modern post-ANSI compilers.
C has many warts and design errors but they got the portability bit down quite well, the biggest problems you will run into are assumptions by programmers rarely assumptions by the designers of the language. One major thing lots of people tripped up over was endianness (sp?).
Some compilers included support for architecture-specific things, like segmented memory, etc. Others allowed to do cross-function gotos. Types were not very well defined. Event the original language grammar from the book is only useful as an idea of what the language should be like, not as a proper syntax definition, I am not even talking about semantics here.
The code I had to deal with (written for 16-bit x86) had to be rewritten, it was definitely not a "fix-here-and-there" thing. It was fine, written by reasonable programmer and completely readable so I was able to understand and rewrite it in a week.
You probably were very lucky with that project of yours.
Even the way we have it nowadays C is... hard to formalise.
Ok, now do the reverse. ;)
The work involved to upgrade from Python 2 to Python 3 for most things will be more like what you did (fixing a few things here and there), certainly not anywhere near as hard as what I challenged with above.
Excellent point, conceded.
But backwards compatibility is the more useful direction to me, I can see however that if your job is to take a modern day application and compile it for a platform for which there is no modern day compiler that you're going to be in a special kind of hell for a long long time if the project has any size at all.
What people seem to be getting uptight about here, is that the volunteer Python language devs have collectively chosen to not fix any bugs or security problems discovered in the older Python2 language interpreter. I can't quite get my head around what the equivalent for that in C would be? There kinda _are_ no "language security features" in C, right? C is the archetype "How powerful a gun would you like to shoot yourself in the foot with today" language.
It still requires dialect selection: https://gcc.gnu.org/onlinedocs/gcc-8.1.0/gcc/C-Dialect-Optio... http://releases.llvm.org/7.0.0/tools/clang/docs/UsersManual....
Not to mention that a system library is only as portable as the code you write for it, since inherently there’s usually a very small standard library of functionality that can be assumed across platforms. And all the support for systems generally rests on the system developer shipping support with C headers, etc.
Now it could be argued that C dialects are far more compatible with each other than Python 2 to Python 3, it could also be argued that linkers allow for code targeting one dialect to call code compiled for another dialect...
Once you go down that rabbit hole, though, there’s an argument that as most scripting languages are basically VMs with JIT compilation, then just like .NET and JVM, there’s nothing preventing scripting languages from inherently supporting multiple dialects via compiled intermediary representations...
But I digress... :)
I'm sure people wouldn't gladly accept C code written for a 47-year-old compiler today.
The current standard languauge is C18.
Portability issues in C are mostly between architectures, not standard versions. The fact that "char" is normally signed on x86 but unsigned on ARM for instance is a common source of problems while porting code between these architectures. At least when you have python3 code you can be reasonably sure that it's going to run on any compliant python3 interpreter.
However the best analogy for python 2to3 is probably the prototyping changes of ANSI C, and we similarly offloaded the conversion of that code to external tools (protoize(1) and unprotoize(1)).
Even if it's just security fixes, there's still the process of testing and release management, and honestly, I don't blame the core Python team for no longer wanting to do release management of both Python 2 and 3.
That's opensource in a nutshell.
If there are any big shops stuck on Python 2 then they will probably throw a resource or two at this. If there are big dists using Python 2 they will do the same.
Which, I'd guess, is probably gonna be 3 or 4 orders of magnitude more money than they've ever spent supporting Python2 (or 3) devs in the 10+ years they've built their own businesses on it while they sat on their hands and ignored the project's urging to upgrade to Python3...
Not a lot of sympathy from me for "big shops stuck on Python2"...
The Python foundation is threatening to sue anyone continuing something called Python 2, or even a similar name.
Finding volunteers who are willing to get sued to work on Python will be hard.
What's "in it" for the volunteer? How much "fun" does "Supporting an old language where the original developers have moved on to a newer and more interesting version of the language, but there's a bunch of complaining people who still want the old language to be supported but they aren't offering to pay for it" sound? I'd rather sit in the park reading a book or walk a dog or something.
I mean, there's still people taking on COBOL contracts, because businesses consider their COBOL code to be important enough to keep maintained. But they sure as hell aren't "volunteering". They're getting paid rates that even bay area twenty-something FAANG-ers would be impressed by.
If Blackrock or The Vanguard Group or equivalent decided they needed continued Python2 support because it was a critical dependency on their ETF platforms (and they'd been foolish enough to not heed the "Goddamn it, just fucking upgrade to Python3 already!" advice from the core team for about a decade), I'm sure they could whip out their chequebooks and agree to a rate that Guido himself would agree support Python2 for them. But I'd hope and expect Guido (or whoever in the core team would be suitable candidates to do this work) to hold out for genuinely life-changing numbers of zeros on those cheques.
And if nobody is offering to write those cheques? Well maybe that says something about the seriousness of their complaints?
I 100% agree. I was just responding to the parent post.
While I generally share your POV, it doesn’t do justice to the situation to trivialize the upgrade like that. Anyone with C based dependencies will have a rougher time (but not that “rough”) due to ABI changes. The harder userland change is string handling anyway (which isn’t that hard either), not parens on print.
I understand using trademark as an advantage in cases of a hostile fork, but if the original maintainer no longer plan to do any bugfixing then it seems like a dick move.
They do have issue with people using a trademarked name and not making it clear that their work isn't endorsed by the Python Foundation.
[1] https://www.naftaliharris.com/blog/why-making-python-2.8/
If I'm not allowed to do that, then you are already adding a lot of friction to continuing to support existing python installs.
Although, I wonder if you could claim you need to call it that for the programs to work?
> Use of the word "Python" in the names of freely distributed products like IronPython, wxPython, Python Extensions, etc. -- Allowed when referring to use with or suitability for the Python programming language.
We can quibble about whether a Python 2 fork would technically meet the definition here, but it seems to me that it would meet it at least as much as IronPython does; going after such a fork would contradict the rule implied by the examples in this section, even if it doesn't unambiguously violate the definition. I think they've already granted the public the right to use the word "Python" in a Python 2 fork and couldn't successfully sue over it now even if they wanted to.
What's your basis for suggesting that they've threatened to do so?
I know this has come up at least three times I've seen, but I'm having trouble finding them all. Here's one:
https://github.com/naftaliharris/tauthon/issues/47
In this case there was plans to call something py28, to which Guido replied "OK, bring in the lawyers".
""" Since I was asked: The project's name (and its binary name) need to change. They are misleading. The rest looks acceptable according to Python's license. This is not an endorsement (far from it). """
---
gvanrossum: Isn't the whole point that we're trying to solve this without lawyers?
[redacted]: The whole point is that you've been sabotaging Python 2 for years and when someone does what needed to be done from the start, you come up with silly objections.
gvanrossum: OK, bring in the lawyers.
---
Just don’t appropriate a name what’s not yours, and you can maintain all the Python 2.x you want. Pathfinder[1] couldn’t call itself D&D 3.6 – Tauthon or anyone else can’t make things called Python.
[1]: https://en.wikipedia.org/wiki/Pathfinder_Roleplaying_Game
Also the longer a software is around, the lower the chance of some showstopper showing up (not 0% chance though)
What's most likely is that one important library in 2.7 stops talking to some upgraded system version (think OpenSSL for example) then what depends on it stops working.
It was also posted in this thread below by the current maintainer of the project but for some reason that comment was downvoted to [dead].
By reading a bit around about it, i do get the impression that the Tauthon developers face a bit of hostility from the Python community, so i'm not sure how viable it'll be in the long term. I suppose it depends on how stubborn the Tauthon developers are :-P
(Unlike other forums, banned users can still post, but their posts require someone with showdead on to vouch for it before it can be seen).
Well RedHat annouced a long time ago that RHEL 8 will drop support for Python 2 [0], so at least it appears they also want to leave Python 2 behind. RHEL7 is already receiving security fixes only [1], which doesn't look like a huge "support" effort to me. Especially, behavior bugs and ports of Python 3 features to python 2 won't be done.
[0] https://www.phoronix.com/scan.php?page=news_item&px=RHEL-8-N...
[1] https://access.redhat.com/support/policy/updates/errata#Main...
Nobody expects new features in python 2.7, so this is basically all the support that people are looking for. RHEL7 will have security maintenance till Jun 2024.
In that regard, maintenance for python 2.7 would involve backporting security fixes also for popular 2.7 third-party opensource libraries and frameworks even if those libraries themselves have already switched to python 3 only.
I wonder if Redhat have left themselves enough weasel words in their contracts to say "Oh Python2? No security updates to the interpreter! Oh, you wanted explioted-library-de-jour updated? Well that's not covered in your support contract here. Left me put you through to our professional service division. Please have your contract ID and credit card number ready when they answer - transferring you now!"
I'm 99.99% certain that if you ran "pip install numpy" on your RHEL7 box, and it's infected by a cryptominer the next day due to a know vulnerabilty, Redhat support are gonna laugh you off the phone when you call them up asking what they're gonna do about it...
Workstation: https://access.redhat.com/documentation/en-us/red_hat_enterp...
Server: https://access.redhat.com/documentation/en-us/red_hat_enterp...
Search for "python"
> I'm 99.99% certain that if you ran "pip install numpy" on your RHEL7 box, and it's infected by a cryptominer the next day due to a know vulnerabilty, Redhat support are gonna laugh you off the phone when you call them up asking what they're gonna do about it...
If course they don't do fixes for all of pip / PyPI, nobody does/can be reasonably expected to do. They explicitly only cover what they ship (which does includes numpy and scipy, but not pandas). If you can demonstrate an exploit with just "yum install numpy" I'm pretty sure they'll work on it.
People getting paid to work on Python (like at Red Hat) are not considered volunteers.
> If you need free help from volunteers, look at this help page.
Which links to the help page to learn python, haha. I think the point is that you can fork it or pay someone to fork it, but you won't find anyone else willing to fix your problems for free.
[1] https://discuss.python.org/t/official-list-of-core-developer...
`PSF` should be inserted before every occurance of the word volunteer in that page.
In what way does this imply that they're speaking for any "extended community"? Who're you trying to say is being mistaken here as being a "volunteer" but is not a 'PSF volunteer".
I totally get that there is an "extended community" around the Python language(s), including at least module authors, people who write Python frameworks, and people who write code in Python - and _possibly_ including devs at companies who have built products in Python or relying on Python. But it's 100% obvious that the linked article is referring to none of those people.
Who do you think this "FOSS org" was appropriately speaking for?
Google has flat-out stated that they will continue to support 2.7 in App Engine for an undefined period of time.
Amazon has inferred through their Amazon Linux 2 LTS support page that Python 2.7 is included in that support for 3 more years.
I don't give a fig about controversy. I just want to keep using Python 2.
Okay, here goes.
In brief, Python 3 adds work without adding significant (to me) value.
As others have pointed out, you don't get anything from converting an existing project from 2 to 3 other than compatibility with 3. If the Python 3 proponents were to succeed in killing 2 then that becomes compelling, however, since I intend to maintain 2, I don't care about compatibility with 3.
As I have become older my tastes have changed, and nursing an old friend seems easier and more fun than trying to stay up to speed with the new hottness. Especially since it just isn't that hot.
Some details:
First, I see 2 and 3 as different languages. The degree of similarity is not relevant (to my evaluation of which to use) since they are incompatible (in other words, incompatibility is binary not scalar.) Since the abdication of the BDFL it seems to me to be even more appropriate to treat them as separate. I don't know how or why but Guido was good at language design. I feel like he lost his touch somehow, and then he left anyway, so the magical "thing" that made Python so sweet in the first place is gone. (I won't comment on any particular PEPs. It's all been said by others elsewhere.)
I feel that Python 2 hits a sweet spot in language design as compared, not just to other Python versions, but to other languages as well. (If you can't use Lisp or Smalltalk or Prolog at least you have Python.) Python 2.7 is a local optimum with a clear view and steep sides, eh? Python 3 goes crashing down the hill and into the woods...
From this POV the cessation of modification to Python 2 is a boon. It's stable and can only get asymptotically more so over time. It also means that projects like Snakefood that do meta-programming can stabilize because the underlying language is stable. We are only now getting type checking and such, and e.g. MyPy has to expend resources to deal with changes to Python 3 or risk bit-rot.
Python 3 continues to include dubious design decisions that then must be dealt with by other projects/tools. I.e. syntax highlighters have to be modified to deal with "fstrings", etc. It drives a continuous process of change for the sake of change without introducing anything fundamentally new and useful. As a developer, your knowledge is depreciating as you sit there because the language is in flux. With Python 2 the thing you're studying and using is stable. In fact, it's possible that the total amount of code could contract in a stable regime. Especially if you have dev time to work on it (that isn't wasted on pointless new features that then engender additional work to learn and use.)
You can also spare cycles to work on things like improving the standard library documentation, or the packaging/versioning mess, or really anything other than tweaking the language (and, again, making more work for everyone without adding fundamental new facilities.)
And at some point, say 10 years from now, you'll reach your breaking point where you'll get tired of maintaining backports for every. little. package. that you want to you use. And when that day comes, then you'll face the even more daunting task of porting your stuff to the new current version of Python.
I'm saying this as one techie to another: don't do this to yourself. Not long term, anyway. If you want to support your own ecosystem for six months or a year as an exercise, sure, I'll be the last one to tell you not to work on a personal project you enjoy. But for Pete's sake, don't engage in this as a long-term project. This isn't how you want to look back on your deathbed and see that you spent a decade.
As a practical matter, I'll say that everyone I've spoken to seems to have the experience that Python 3 seems like a pain in the neck for a very short while as you go through the process of finding all the bugs in your codebase that Python 2 was letting you ignore. But once you've done that, you're not going to want to go back and un-fix them even if you decided to stick with Python 2, and if you've already gone through that work, why not enjoy the new language features and 3rd party packages now available to you?
(Going forward I'm more likely to use Julia, Haskell or Prolog than Python 3 for new code.)
What do you think it would have happened if python 3 included a python 2 interpreter (as a command line flag or even a library) so that everyone could switch to python 3.
I understand they where almost incompatible as runtimes so it would have doubled size but it would make easier to propagate the diffusion of python 3.
Do you think a project such as Bladders would have been still possible had this happened?
Honestly, I'm not so much a Python 3 hater, as a lover of stability. To me the idea of Python 2 being stable for the foreseeable future is exciting because it means I can concentrate on improving the tooling (e.g. tools like PySonar and Snakefood and such. I would add MyPy but of course they will be chasing Python 3 now.)
> Do you think a project such as Bladders would have been still possible had this happened?
Maybe... You can run C code from decades ago. That's pretty awesome. I guess it would depend (in this hypothetical) on whether the 3 interpreter was ever going to drop support for 2 code.
> what is it that you like about Python 2?
I came from C and Pascal so from that POV Python is a rich and delicate syntactic and semantic gravy over the same basic functionality plus shell (Python was originally the shell language of the Amoeba distributed OS†).
Things that might not seem that big a deal these days were a revelation to me when I started with Python:
- The basic datatypes, list, tuple, and dict, are enough to describe so much of the tasks you need to do.
- Slice notation is so useful, and works on the left-hand side (you can assign to slices) and the elegance (you can reverse a string or list or tuple by oof = foo[::-1].)
- The simple but effective OOP model, "dunderscore" methods, monkey-patching is possible.
- How can I forget! Indentation for blocks! So crazy but it works so well.
- long ints that don't overflow! So nice.
I can go on but really it was the whole thing, the syntax, the semantics, the standard lib. But in my opinion it reach a zenith sometime between 2.4 and 2.7, and even then Python overgrew its ecological niche. (What I mean is that if your project has more than 100,000 lines of Python you're pretty much fucked, but people do that all the time these days. I once worked on a horrible old "lava-flow" enterprisey Python codebase that could have been replaced by Excel but instead employed a gang of programmers. Pure waste.)
(Also, you're definitely not fucked with more than 100k LOC of Python, if the devs know what they're doing. Unit/regression/integration tests, type hinting, and a modular design goes a very long way)
I never said that. In fact, I admit that in some ways 3 is better than 2. However, that 2 is static (unchanging) while 3 is being actively changed means that, as time goes on, the cost of using 2 will decrease while the cost of using 3 will increase.
> Also, you're definitely not fucked with more than 100k LOC of Python, if the devs know what they're doing. Unit/regression/integration tests, type hinting, and a modular design goes a very long way
I was engaging in a bit of rhetorical flourish there. I know it can be done, but it's not Python's "sweet spot". For large projects, Java or Haskell or even Ada are preferable IMHO. And I say this as a Python partisan.
Cheers!
Because 3 is incompatible with 2 I evaluate the cost/benefit of using it compared against all other languages, not just 2.
From that POV, I find that e.g. Julia, Haskell, or Prolog are much more compelling than Python 3. Python 3 doesn't add anything radically better than 2.
(Most of the new language "features" to me seem like they should be libraries instead: fstrings, enums, dataclasses; I can't understand why people think asyncio is cool but ignore Twisted (which is a huge treasure chest of amazing capabilities!))
Since I see 3 as only marginally better than 2, the main difference between them is that 2 is stable and mature while 3 is in flux, and that consideration mashes the cost/benefit ratio waaaay over in favor of 2, IMO.
I don't want to come off condescending, I really want to understand your point. But I have no idea what you're trying to say.
Sure, technically py2 and py3 are different languages. So what?
Same here! Well met.
> But I have no idea what you're trying to say.
Sorry, I'll try again.
> Porting a python 2 project to Julia/Haskell/Prolog is rewriting the entire codebase.
Sorry, I was thinking of new projects there.
To understand my point keep in mind that I'm assuming a world where Python 2 stays stable and maintained (if only because I'm maintaining it) so the option of not converting to Python 3 is on the table. In that world, I don't see a compelling reason to adopt Python 3 when Python 2 still exists and is stable. (And FWIW I think some of the py3 folks realize this and that's why they are so eager to "cannibalize" the old for the new, precisely because py3 can't really compete on its own merit against its entrenched incumbent ancestor, eh?)
> In my company we have >2 million line of py2 code and we spent a few weeks converting all to py3 (biggest problems being str/unicode and None comparison) and less than a month later our prod runs py3.
You paid a modest but non-zero cost to convert and then what did you get? Was it faster? You didn't convert 2M loc py2 to, say, 1.8M loc py3?
If Python 2 wasn't going away (my premise) would you have converted to Python 3? If so, why?
PyPy is delivering speed improvements, and to me it seems like most if not all of the other language features of py3 can be done in py2 with libraries. Other than compatibility with py3, what was the payoff?
> Sure, technically py2 and py3 are different languages. So what?
Two things: first, the whole reason this mess is happening is IMO because py3 interpreter can't run py2 code. If it could I think the transition would have gone smoothly and been done by now. It wasn't necessarily a bad thing to correct some warts, but it was a deliberate choice to "kill" Python 2 despite the obvious fact that we're going to be stuck with it for years to come (COBOL! FORTRAN! Like Elder Gods, these things don't die.) Really, i don't mind py3! It's the effort to throw py2 onto the funeral pyre while it's still kicking ("I feel happy! I feel happy!") that I find objectionable.
Second, and IMO this is the big deal: py2 is stable while py3 is in flux.
If the language stops changing that means people can work on the tooling and implementation instead. If I was managing a codebase the size of yours I would be much more interested in PySonar (or Cython or Nuitka) than anything that py3 brings to the table. Changes in the language require changes in the tooling (even just your syntax highlighters) so IMO they should be really good to pay for the expense of learning them and supporting them. For example, algebraic data-structures with pattern matching would be a nice addition to Python. (But then, you can do that in a library too!)
In sum, the very fact that py2 is now stable and unchanging is a killer feature (IMO) that differentiates it from py3!
> In sum, the very fact that py2 is now stable and unchanging is a killer feature (IMO) that differentiates it from py3!
This is a good point, but I disagree. As long as they make future py3 versions backwards compatible, it doesn't affect me if they keep changing py3. To me, it's more important that the world will be operating on py3 as opposed to py2; this is the killer feature of py3.
> the world will be operating on py3 as opposed to py2
You're right, but like I said, for new stuff I'm using Haskell or Prolog or something, not Python {2,3}.
I don't want to advance Python 2 --well, by adding and improving libraries and tooling I do, but not because I plan to make new stuff in py2, rather to make maintenance cheaper-- I just want to prevent it from dying. I'm really only interested in maintenance. Otherwise I would just contribute to Tauthon, but they're modifying the interpreter/language whereas I just want to make the existing thing more solid.
Well met. :-)
Honestly curious as to why you would prefer to stay behind?
People still run old FORTRAN from 20 years ago in production. I'm sure some old COBOL code is out there. The reality of not wanting to re-write working code is somewhat real.
And people said what you said about it 10 years ago. And 20 years ago. And 30 years ago.
When those folks want to do scripting, they mostly use Python. And they mostly use Py3, from what I can see.
That's what I'm in right now :-)
I loved Python 2 for a long time. It was a beautiful language. But after seriously using 3, I would never go back. As in, I would turn down job offers involving Python 2 in any other context than "we're hiring you to help us upgrade".
Better how? Or better at what?
It's likely that our opinions will be different but I've got an open mind and I'm genuinely curious about your experiences and opinion.
(I hated PEP 572 at first but then some comment here on HN made me rethink and change my mind.)
print(b"byte = FF (\xFF)")
Never mind `print(len("ẅ"))`[0] and the like.
If there is a bug, did you file a bug report it ?
If there is demand for python2 maintenance, there will be people--like you--working on it and companies selling support for it.
All this announcement is saying that the python community at-large (aka "We are volunteers who make and take care of the Python programming language") is focused on python3 and isn't going to work on python2 any more.
Their position makes sense, even large communities have limited budgets on development capacity, and they want to spend it only on python3. You control your own capacity, so if you want to focus on python2 you'll likely find some like-minded compatriots.
Project repo: https://git.sr.ht/~sforman/Bladders Mailing list: https://lists.sr.ht/~sforman/bladders-announce Subscribe: mailto:~sforman/bladders-announce+subscribe@lists.sr.ht
At this moment it is total vaporware. I should have some time this week to clone the Python 2 code and set up a virtual server farm to compile it and run test suite on some different platforms, as a start. (Or I just remembered srht has a "builds" tool that might be just the ticket.)
[1] https://www.hostgator.com/help/article/what-software-and-pro...
Like Google App Engine lol. GAE Standard Environment doesn't support Python 3.
Python 2 is part of the first generation runtimes, python 3 is part of the second generation runtimes. They are not out of the box compatible, even if your python code works on 2 and 3. The first generation runtime included many APIs that are simply just not available in the second gen (you need to build them yourself). See link for more info.
GAE has been a favorite of mine for years, but now they seem hellbent on ruining it, and I don't understand why ='(
(On the plus side, other things are improving and Run looks good)
https://news.ycombinator.com/item?id=16808998
We knew in April 2014 that Python 2 was EOL in 2020.
Doesn't mean we should do anything except laugh at their incompetence (or perhaps be impressed by their ability to monetise other people's incompetence...)
Much respect. That’s retro-cool.
(And no, I'm 100% not offering to do any of the work required to make that come true... ;-) )
As an aside, a few weeks ago I interviewed at one of the top 5 investment banks for a software engineering role to maintain one of their main trading systems, written entirely in Python 2. My first question was what their plans were in porting what they called "one of the worlds largest Python 2 code bases," and it was "on the roadmap".
Surely the Python core team are aware that teams like this exist in some of the largest companies in the world, and I assume they've had at least some level of dialogue with companies like this to say "we're super serious this time, we're about to stop support for Python 2", or something along those lines?
Maybe it's a blessing in disguise that I was rejected?
If they start in the next few years, I imagine it'll be much harder to find ready and willing Python 2 developers with the experience to navigate such a big code-base? They could hire consultants, but I imagine the future hiring cost will also be astronomical.
But distribution vendors like Red Hat and Ubuntu have Python 2 in long term support distributions that they will give support for a few more years, so it is possible to use those. And after that they offer more years of support to paying customers. It's not necessary to hire programmers to do this for the companies needing support themselves.
That's because the Python community is too nice.
When Ruby or node broke compat, they just said "move or die". People moved.
When Python broke compat, they gave 7 years. People screamed, cried, complained, sweared it was impossible for them, that life was hard and the PSF was unfair.
They got an extension of 5 more years.
People still complained !
Hell, famous authors resented python 3 for years, refusing publicly to update their book.
On the other hand, the PSF had a very small budget, only few tiny donations and contributions by people or big companies that were saying they couldn't migrate that their lines of code. The same code that were making them a living, from a 25 years old amazing tech given to them for free.
My take on this is that if you are doing an open source project with a lot of volunteer work, don't be too nice. People have a tendency to abuse it. Save yourself trouble, and balance respect with users and convenience for you.
I don't think I disagree with your point in practice. Maybe in some details. I just don't think that not being credible in your plan to drop support on a given date is necessarily such a "nice" thing to do. You could say it's a "weak" thing to do. It doesn't help people prioritize the move and eventually hurts people by putting them into a jam on the day finally comes here. I think the ideal thing would be to break compatibility piece by piece, assuming it's really necessary, but be _understanding_ about it. Tell people "we understand that this is inconvenient for a certain number of people but have to cut it off at some place", not "stop complaining about free stuff".
Unless I'm missing something, this is a pretty disingenuous comparison. Ruby and Node breaking changes have been fairly small, Python 3 was anything but.
The platform can already run in python 3. It's possible that the team you interviewed with wasn't aware of that.
Like all big companies, personal experience varies with the team and the part of the company you're in.
I won't name names, but they're also in the top ten and based in America.
> Surely the Python core team are aware that teams like this exist in some of the largest companies in the world
Why would you assume they considered their users? Maybe they went purely for some abstract purity with tenuous benefits and didn't actually speak to their stakeholders. Remember, the "D" in BDFL stands for "dictator".
Recently though, I have been running into more and more projects that don't specify the version and are built using Python 3. I also used your rule of thumb, so seeing this was kind of surprising. Wonder if it's just the mentality shift that Python3 == Default?
For example, I decided to only support Python3 going forward. I've had person once open a PR to add python2 support, but I politely declined[2]. As such, PyPI shows that only 3.5+ is supported[3].
[1] https://github.com/RPGillespie6/fastcov/blob/master/setup.cf...
Both errors were familiar from earlier cases of trying to run with the wrong Python version, but I have no idea how the code managed to do that for both, and frankly it wasn't worth the effort to look into.
I liked the stance the post took: it was very matter-of-fact in its tone regarding python 2 support — consultants are there for that and they will charge a mint to give you time you should have taken to move your codebase to python 3.
Edit: Yes Manjaro has default Python3 too. :)
Although their package management requires manually specifying the minimum accepted Python version for every single package, so they have thousands of packages that would run just fine on Python 3.7 but no one has the time to manually check and label them as such. So their default is still 3.6 and using 3.7 as your default version is not well supported.
I wish someone told me Gentoo is a non-starter before I wasted an afternoon installing it.
While we're on the topic, it's pretty stupid how PEP 394, the one that says "python should point to python2.7", gives its reasoning as "because all Linux distros except Arch do it this way" and then all the distros say "python should be python2 because the PEP says so" for the next 9 (maybe more) years. How is anything ever supposed to change?
Or use a Linux distribution like Arch that has python symlinked to python3 by default.
How many more decades must we wait??
If I have a modern C compiler, I can still run most C code from 30 years ago.
When I have a Rust 2030 compiler, I'll still be able to run Rust 2015.
But with only a Python 3 interpreter, I'm strictly unable to run the vast majority of Python code out there.
I think there is a parallel with the .net Framework. .Net core is a similar breaking change (not the syntax of the language but very much so in term of core libraries and project types). I wouldn't be surprised if .Net full followed a similar timeline.
The tone in this document is excessive and over-the-top. If the author spent half the document demonstrating (with examples) why Python3 is so great, it might actually be useful and get people upgrading like right now as they're reading it. But instead, it seems the author simply wanted to convey how sick they are of Python 2. The author does not appear to understand their users; that's their real frustration.
In life outside of Python... The transition in C/C++ from C++98 to C++11 and then C++20 (and beyond) has been quite clean and professional. There were some glaring issues with C++11 that later got patched up (e.g. rvalues but no optional, no filesystem, make_unique, etc). Most importantly, compiler support and standard distributions improved a lot. There were often clear incentives to upgrade, and it was relatively easy to stick to an old standard where necessary.
In contrast, Python has become a bit fractured, with a number of notable well-discussed but unaddressed loose ends in Python3: http://pyfound.blogspot.com/2019/05/amber-brown-batteries-in...
I think the Python community has failed to present strong enough incentives for upgrading to Python3, and that's why users are sticking to Python2 and 'slowing everybody down'. The jump to C++11 was obviously needed: at the very least, `auto` saved much of your sanity and you no longer really needed to include Boost in your build. But you could stick to an old standard if you wanted; you could even transition a codebase module-by-module in some cases.
The Python3 community seems to have been groping for a similar killer feature (e.g asyncio, as discussed in Amber Brown's talk above), but the value-add isn't so clear. `six` is honestly a nice catalogue showing how many changes in Python3 are largely sentimental.
Perhaps the killer feature missing from Python3 is a flexible, built-in Python2 runtime. Then maybe the transition would not have met such pushback.
If people haven't found reason in the last ten years, then they aren't about to be convinced today. All the arguments for and against have already been widely discussed, and that the EOL of Python2 is here should come as a shock to exactly no one.
There's no reason to re-hash over and over what is already out there.
A bit? This is an incentive.
Me too. Anyone who's running 15+ year old free software which came with 6 or possibly 12 years of notice to upgrade to the new version (depending on how you count it) and is complaining that it's no longer going to be supported for free by someone else, is being quite amazingly arrogant...
Maybe it's unintended, users of free software are not always good at being gracious.
It's also arrogant to grab Python 2 out of the hands of people still using it and flush it down the toilet.
As choppaface said,
> Perhaps the killer feature missing from Python3 is a flexible, built-in Python2 runtime. Then maybe the transition would not have met such pushback.
If Python 3 could run Python 2 code (even at just the granularity of modules) then all of this mess could have been avoided. I love Python and I love GvR, but he stumbled badly here IMO (not as badly as Perl 6, but that's small comfort) by trying, in effect, to cannibalize 2 for 3. I don't enjoy saying this.
I've met GvR briefly and he didn't seem arrogant at all to me. But I do find the attitude of "Hey, F. U." to those of us who want to keep using Python 2 to be pretty arrogant.
That's a strange way to characterize "decide to no longer volunteer their own time." And by strange, I mean rude. If you want somebody to spend their time maintaining python2, why don't you cough up the cash?
> why don't you cough up the cash?
I am.
Second,
> "decide to no longer volunteer their own time."
If that's all they were saying I would STFU.
They are also telling everyone that, once they stop volunteering, Python 2 is dead. They insisted that the Tauthon project rename itself. It's one thing to say, "we won't support it", it's another thing to actively try to kill it, especially when they know people are still using it.
It wasn't. (And Tauthon is arguably a really cool name.)
But it does indicate that they want Python 2 dead.
> Van Rossum argued instead that if the Twisted team wants the ecosystem to evolve, they should stop supporting older Python versions and force users to upgrade. Brown acknowledged this point, but said half of Twisted users are still on Python 2 and it is difficult to abandon them.
http://pyfound.blogspot.com/2019/05/amber-brown-batteries-in...
Like I said, if people were just fading out of Python 2 that wouldn't make me call them arrogant, it's when they insist that other people have to ditch Python 2 that I start to feel like there's some arrogance there.
Like I said in a another comment, the only thing you get from switching to Python 3 is Python 3 compatibility. If Python 2 remains viable (which it will) there's much less incentive to use 3.
Python 2 isn't broken or bad or anything, it's just not Python 3. We aren't being urged to switch because 2 is (so much) worse than 3 but because it's a viable competitor to it.
Being less uncharitable, it indicates they want to avoid brand dilution (which is wholly reasonable.)
Look, I'm not really interested in proving that so-and-so is arrogant or not.
It's pretty clear to me that some people have some animosity to continued use of Python 2, and I find some of the language and attitudes smack of arrogance. I'm arrogant, and it takes one to know one, eh?
But I don't have a magic arrogance-o-meter so really what are we arguing about? Our feels?
Perhaps you could argue that the Python3 language could have changed it's name and "left" the trademark and reputation behind for some other random people to take over. But a stronger argument would be that the reputation Python has earned over the last two decades "belongs" to the ongoing team who built and supported it for way longer as version 2.X than they needed to and who've now been releasing it as V3 for over a decade.
If "people are still using" Python2, nobody is going to stop them. Nobody it deleting all the copies of the source code to Python2.7 or revoking any of the rights the granted when it was released:
"PSF hereby grants Licensee a nonexclusive, royalty-free, world-wide license to reproduce, analyze, test, perform and/or display publicly, prepare derivative works, distribute, and otherwise use Python alone or in any derivative version" (from: https://github.com/python/cpython/blob/2.7/LICENSE )
It you're uptight because the PSF owns and protects the trademark to the name "Python" where used in relation to a programming language _that they wrote and gave away and granted you extensive rights to do whatever you damn well please with their source code_ - that's a totally unreasonable expectation on your part.
Spend two decades building your own reputation for intelligent and responsive language design and stewardship, then walk away from all that reputation by allowing random people to piggyback off it, then get back to me and tell me how that's actually how shot should work... Guido and his team have put the hard yards in. They've given you pretty much free reign to do whatever you like with their source code. They are not only under no obligation to allow you to call what you do with that "Python", but they arguably have a responsibility to ensure that people who expect a historical level of stewardship and "benevolent dictatorship" of the language called "Python" are not mislead by people other than them using that name to continue to promote old and discarded technology and design decisions with any assumption that those new people deserve any of the historical reputation that "Python" implies.
Get over the "Wah! I can't use the name Python! I'm being oppressed!!!" childishness. I, for one, do not want _you_ specifically, and people like you in general, to fraudulently trade in the reputation that the Python trademark would bestow on your work if you were allowed to call it "Python".
Here: https://github.com/python/cpython/tree/2.7
Join the other 11.5k people who've forked it on Github, and an unknown number of people who've cloned it without playing Github's high-school popularity contest...
Nobody is ever going to take anything Python2 related away from you.
There's a lot of talk here about "arrogance", all from people upset that some other people have decided not to do any _more_ free volunteer work on a thing they've been trying to get people to upgrade from for 3/6/12 years depending on how you count it. Those people complaining about "arrogance" are putting words into the mouths of the people who've done so much free work for them over th=e last 20 years, accusing them of saying "fuck you" and of "grabbing things out of the hands of people" and of "cannibalising" the code they wrote 15-20 years ago while providing newer better code (which happens to not suit some people who're not prepared for whatever reason to update their own code).
GvR is _not_ the one seeming arrogant in this thread...
(And didn't the last line in my comment you replied to make it abundantly clear I knew exactly what arrogance the OP was incorrectly claiming, and what the actual arrogance on display was?)
No one is doing that. Python 2 is available, open-source software. The Python core team discontinuing development isn't grabbing it out of anyone's hands. Open source projects whose core teams have discontinued them in favor of an incompatible (either technologically or by licensing) alternative, or abandoned them altogether (or just not maintained them in a way that pleased the community), have often had maintenance picked up by alternative maintainers, who have often had to use alternative names for the independent continuation project. Nothing is taken out of anyone's hands by this.
> If Python 3 could run Python 2 code (even at just the granularity of modules) then all of this mess could have been avoided.
And we all saw how well starting with that goal worked for Perl 6.
> But I do find the attitude of "Hey, F. U." to those of us who want to keep using Python 2 to be pretty arrogant.
No one has that attitude (except maybe employers, but that's more about a desire to continue to get paid for using Python 2 than merely to continue to use it.) You are perfectly free to continue using Python 2 until the Earth is swallowed up by the Sun and no one will stop you, or even really care.
[1] https://www.businessinsider.com/oracle-customer-explains-aud...
There are tons of incentives to upgrade, but unfortunately very few to none can be considered killer features, and thus are harder to show off in a concise reminder about the sunset.
You have to give it a try and actually use Python 3 features to realize how much this improves your life as a developer.
Python 3 isn't perfect, but especially with later versions, it's a hell of an improvement over Python 2.
No, the quoted passage is exactly correct. Many major tools have dropped Python 2 support, and others have put that on their timeline. If you continue to stick with Python 2, you will lose out on the ability to use an increasing number of current tools, because the community is moving on. Relaying that fact isn't arrogant.
More like 12 years by now :-)
To clarify: long-term means automotive time scales: I want to be able to run my software in 10 years.
Python 2's EOL was first announced in 2008.
I work on a C program that's about 30 years old. We've slowly moved, at our own pace, to C99, then added some C++ here and there.
I don't ever want to have a forceful change, where I have to go and edit code which has worked correctly for over 15 years, just to make a compiler happy.
It's much easier/better when you are forced into dealing with the change. At least people can laugh at how annoyed others get when they cry "but you broke everything!".
It's called progress. Get with the times or get outdated.
Similarly, developers are not forced to upgrade to python 3 any more than they are forced to upgrade to GCC 4, in that they can keep using their old tools but they're SOL on maintenance if they don't meet the requirements of the new versions.
So, either your team chose really well 30 years ago, is actively working on an horribly insecure application, or has invested some modicum of effort periodically over the last 30 years to maintain support with the compiler (unlike the developers complaining about Python 3's upgrade window).
You can! Just buy 10 year old hardware, and run a 10 year old operating system. That's how banks, schools, governments, etc are running 40 year old software.
I've upgraded a number of projects at my work to Python 3 this year, but nothing that 2-3 developers couldn't do in a few weeks. Can only imagine the headache of migrating something the size of Splunk.
However, if you've been using the python SDK for Splunk - its been supporting python3 for a few years already (https://github.com/splunk/splunk-sdk-python/commit/4503db961...)
See this blog post: https://www.splunk.com/blog/2019/07/01/admins-and-developers...
And this helpful doc: https://docs.splunk.com/Documentation/Splunk/7.3.1/Python3Mi...
I honestly had no idea the SDK has supported Python 3 for this long. Granted, I haven't worked with it since 2015, but still surprised I missed that. Glad to hear they are making such good progress.
I've switched to Tauthon [1] for over a year now and have been quite pleased. Consider the switch yourself rather than rewriting your code.
I am, however, very happy to see that ROS2 (the next iteration of ROS) uses Python 3 by default.
There was a discussion on HN about this a few years ago, but I am curious to see what does ninite.com plan to do now?
I run a Flask course[0] and from the beginning I coded the app to work for both 2.7.x and 3.x in parallel but it's only been a few months now where Celery supported Python 3.7.
In either case, upgrading from 2.7.x to 3.7.x was super painless. It came down to making sure Celery is using 4.3+ and pytest requires 5+ if you plan to use Python 3.5+. There wasn't any app level changes that had to be done other than optionally removing some dual import try/excepts to work for both 2.x and 3.x. The biggest pain point was remembering to use print with parenthesis but I've been doing that for a while now so it's a non-issue.
This was based on a decently sized Flask app with tens of top level dependencies and many thousands of lines of code.
If someone wants to see how this upgrade process looked in real time, I recorded that and put it up on Youtube[1] the other week. The video also covers some gotchas you might encounter during the upgrade process (especially if you're running in production and deal with sessions).
[0]: https://buildasaasappwithflask.com/
[1]: https://nickjanetakis.com/blog/upgrading-a-dockerized-flask-...
Yes but check the changelog for pytest.
Version 5.x.x is only available for Python 3.5+. They dropped support for 2.7 and <= 3.4, unless you fall back to the 4.6 series.
To be honest, the whole “Python on other runtimes” movement, as a concept, is more or less over. It’s just too much effort for too little reward, now that CPython has good libraries for pretty much anything you can think of.
It's written like a hate letter with a passive aggressive tone of finger pointing towards users of Python as the source of the never-ending Python 2-3 split. It says, between the lines: "YOU are part of the problem, stop using Python 2. We don't care about YOU anymore. Stop using Python 2".
I can think of ten other different ways to be more constructive and, generally, nice.
Boring work though
Now that Python no longer has a BDFL, maybe we'll get the print statement back? ;-p
I agree that it probably contributed to python 2 inertia as it was re-exposing people to the idea that "typing python in the Terminal gets me python 2" and "I just used what I already had" - but it definitely wasn't stopping people installing a newer version.
https://developer.apple.com/documentation/macos_release_note...
Hopefully they will fix the shitty keyboard before that happens, so I can finally replace my old Macbook Pro.
Scripting Language Runtimes
Deprecations
Scripting language runtimes such as Python, Ruby, and Perl are included in macOS for compatibility with legacy software. Future versions of macOS won’t include scripting language runtimes by default, and might require you to install additional packages. If your software depends on scripting languages, it’s recommended that you bundle the runtime within the app. (49764202)
Use of Python 2.7 isn’t recommended as this version is included in macOS for compatibility with legacy software. Future versions of macOS won’t include Python 2.7.
I've looked at the PiStorms stuff MindSensors (Lego Mindstorms compatible). Unfortunately it all seems to be just Python2 based.
Looking at all the code it seems you're always basically running your own event loop (infinite while True: loop). I think it'd be really awesome to utilize Python 3's asyncio package to write more event driven programs.
Writing code in asyncio is basically a different paradigm. It's not like Golang where you can write a function and not care if someone is calling it synchronously or starting a goroutine with it. With Python asyncio it seems for every good library out there someone has a create an asyncio version of it. For instance "requests" vs. "aiohttp".
So, while embracing Python 3 still wouldn't get me what I ultimately want, it's a first step in the right direction.
"asks" implements a requests-like interface over asyncio/trio.
There's a lot of money to be made doing migrations and keeping the beast itself alive.
I'm not complaining. I like Python 2.
Fun stuff...
And I think Audi is using it for their self driving cars...
https://discourse.ros.org/t/versioning-roadmap-moveit-1-0-re...
I predict Maya devs (and, by extension, game engine vendors. like Epic/Unreal, who version-lock to Maya's version) will stay on 2x well past 2020 by making their own platform-compat patches.
The "industry standard" platform is only just getting around to switching to Python 3: https://vfxplatform.com
This is the same with Python. Python 2 code will continue to work with python 2 interpreters, just as java 1 code will continue to work with java 1 JVMs and JDKs. But neither are being actively developed and Oracle will not even sell support for java 1, let alone release security updates.
Ubuntu 18.04 example:
$ python --version
Python 2.7.15+
$ python2 --version
Python 2.7.15+
$ python3 --version
Python 3.6.8
$ ls -l /usr/bin/python
lrwxrwxrwx 1 root root 9 Apr 16 2018 /usr/bin/python -> python2.7Typically version N+1 of a language compiles version N, N-1, all the way back to version 1. Usually, when this isn't the case, the incompatible areas of a language were either marked experimental, or obscure use cases that very few people use.
Today's Java compiler will compile source code written for Java 1.
The issue with Python 2 and Python 3 is extremely unusual in a language.
In all cases, the real problem is technical debt management: rather than attacking people who gave you something for free, ask whether it’s possible that the experience has more to do with procrastinating on upgrades until many large changes have to be made at once with inadequate test automation.
Incompatibilities between JDK 7 and JDK 6: https://www.oracle.com/technetwork/java/javase/compatibility...
Incompatibilities between JDK 8 and JDK 7: https://www.oracle.com/technetwork/java/javase/8-compatibili...
An exerpt from the Compatibility Guide for JDK 8 [1]:
>In general, the source compatibility policy is to avoid introducing source code incompatibilities. However, implementation of some Java SE 8 features required changes that could cause code that compiled with Java SE 7 to fail to compile with Java SE 8. See Incompatibilities between Java SE 8 and Java SE 7 and Incompatibilities between JDK 8 and JDK 7 for information.
There are also a plethora of deprecated and removed APIs throughout Java's history.
[1]:https://www.oracle.com/technetwork/java/javase/8-compatibili...
The incompatible areas that you explain amount to bug fixes and obscure areas of the language. They aren't anything like Python 2 vs 3.
Some of their “bad” designs were made decades ago, before it was even clear where the industry was going. Take Unicode: lots of bets were made on that, and arguably Windows got it way wrong. Sure their APIs are still “compatible” but what is that worth, when strings are an inconsistent mess that to this day require developer hacks? And that’s an OS people pay for, written by competent engineers as well.
If you don’t like your free software, please feel free not to use it anymore.
A few had 3.5, but I used 3.6.
What can be done to make this easier? I.e. why isn't python3 pre-installed by now? esp 3.6 or 3.7?
With that said, Apple will not be including Python (or the other scripting languages like Ruby or Perl) at all starting with Catalina.
Python 3.4.3 (default, Oct 14 2015, 20:28:29)
[GCC 4.8.4] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> x = map(lambda v: v + 1, [1, 2, 3])
>>> list(x)
[2, 3, 4]
>>> list(x)
[]I know many of the big ones are already on 3.x for a long time, but I wonder how long will it take to replace and upgrade all the small ones if it did not happen already...
BTW I don't know how necessary was to break compatibility between 2.7 and 3, the pain of this transition comes from that decision
It should be pointed out that Python 2 will continue to be used, and be usable. It just won't get new security upgrades or features. I do not know, but I doubt if there will be a feature or security flaw come up soon that "forces" existing python 2-only libraries to update
I've always thought Python 2 might be headed for the languages. Which is fine, it's great to have a programming equivalent of Latin.
Python is more like the English language... it's borrowed a bunch of stuff from many different languages and isn't even compatible with itself ;) (2 vs 3).
I mean, this is assuming with thing spoken languages are even at all similar to programming languages... which I don't think they are.
Hebrew was completely replaced with modern Hebrew and gets new changes constantly.
Python 2 sees new code all the time too. It's not dead! That's the whole point, it can simultaneously not see any new changes but still be used by those who value the absence of change.
Just to clarify further, modern Hebrew is mostly backwards compatible with biblical Hebrew, it's not an entirely new language. It was mostly updated in terms of vocabulary that was necessary for the modern era, plus some spelling/grammar changes.
Modern Hebrew speakers are able to read biblical Hebrew almost completely, though there are some differences (think of English speakers reading Shakespeare - though even less difference than that.)
I have a GPS watch and some old Python 2 script to export the data (found it online (unmaintained) a few years ago).
However, when Python 2 will be sunsetted I am a bit worried that Linux distributions will stop packaging the dependencies of said script, but I would like to keep using it until the watch breaks.
I would be interested to see if I can update to python 3.
One of the reasons for the 14 year deprecation windows (imho) is that the language maintainers didn't have control over much of the core platform libraries/ in everyday use. This created a lot of community angst against the upgrade path.
Overall, didn't have too many issues with the conversion but this stuff is going to hit some companies harder than others. I can tell that it's worth the migration, end this blood fest once and for all.
1. Native Python 3 Developers: It's great over here, what's the problem?
2. Python 2 Developers Migrating a Small App: It barely took me any time, you Python 2 developers are lazy and inhibiting progress!
3. Python 2 Developers Migrating a Large App: Ugh.
I've only used python for scripts and such and pandas for data-analysis at my last job.
We need to <specific goal> so we can <universal statement>. That statement is boilerplate manipulative garbage.. its ok to speak this way because you agree with the goal ?
In some sense chardet doesn't explicitly support Python 3.7, but it's packaged for it in Debian Stable. I wouldn't expect any problems.
more-itertools is listed in white simply because it doesn't use any granular version classifiers on PyPi. It only has a python_requires setting, which is authoritative.
It's plausible that every single package on that page supports Python 3.7 in practice.
Here is another issue which looks more recent and hopeful https://bugs.chromium.org/p/chromium/issues/detail?id=942720
Two distinct languages that happen to have the same name
No you don't! :D
http://charlesleifer.com/blog/new-features-planned-for-pytho...
I think it would probably be ok to start thinking about it in a year or two from the py2 sunset. It will be a minor update anyway, it's just a PR exercise that can be done whenever people think it would be more effective.
Latest version of the COBOL standard is COBOL 2014 (AKA the same year python 3.4 was released).
It seems a bit wrong to surprise a customer now.
I fully support them in this. Either you support it right, or you abandon/sunset it clearly.
If someone else wants to step up and do the work, they can always do that on their own.
By the way this is market opportunity that HN readers might consider. If you think you can make money supporting Python 2, then start working. Who knows it could be a gold mine.
The latest official COBOL spec is COBOL 2014 and there are at least two or three competing companies out there that will happily sell you up to date COBOL tools and compilers. So while COBOL may be old, there are still new 'exciting' things happening in the COBOL space.
Out of interest, does anyone know if new COBOL specs add any major language features or are they just low-profile security fixes and things.
EDIT: I see now that they mention that 2020 has been the plan since 2014, which seems reasonable.
That's more than 10 years ago that the intention to sunset python2 at a specific date was announced.
Anybody who, after 2008, started new, serious python2 projects that were supposed to have a life expectancy beyond a few years was rather "optimistic", and anybody who did after 2015 was almost callous, especially if you "played" with other people's money.
Should have been removed years ago.
It actually causes real issues, such as build systems that only support python2 but not python3.
The sooner python2 dies, the better. I need to use both python2 and python3 since otherwise quite several software projects refuse to compile from source. An example is Mozilla's mozjs.
http://www.linuxfromscratch.org/blfs/view/svn/general/js60.h...
Also increases your trust in the company creating Rust when they are too lazy to fix their whole build suite. Total clowns working at Mozilla here.
Was the cheap shot really necessary?
Mozilla is not in charge of Rust in any way.