Hacker News Reader

Comment by at_a_remove | Hacker News Reader

I'm in an awkward chain of dependencies. You see, in my industry there are very few players, and the current version of their software depends on much functionality and rather exacting specifications from products which are made using ArcGIS 10.2.1. Not 10.2. Not 10.2.2.

This is a very conservative hunk of software because this is not a "move fast and break things" industry. This is a "people die when we screw up" industry. So they haven't moved from ArcGIS 10.2.1 for our version. Change is coming but they have to be careful.

So ArcGIS 10.2.1 comes with a python install of 2.7.2 (I am pretty sure it is .2). And you are very strongly advised NOT TO UPGRADE THAT in big bold letters on the relevant help pages. So I need to use 2.7.2 to manipulate some stuff out of ArcGIS 10.2.1 to do "stuff."

And there I sit.

View on HN

I work for Esri, but I specifically work on ArcGIS Pro which has shipped with python 3.x for quite a while. (since before I joined the company anyway)

ArcGIS 10.2.1 comes with 2.7.5, [1] and is very, very old. It was released in Jan 2014 and hit EOL in 2019. [2]

All not-EOL versions of ArcGIS Server support python 3.x. But if you're using ArcMap, upgrading to a later version will not move you to python 3.x: you'll have to migrate to ArcGIS Pro for that. (note for the unaware: this is not a "well just buy the more expensive thing" post. ArcMap and ArcGIS Pro share the same license.) I recommend migrating to Pro for this and other reasons, but I understand that a lot of orgs don't have that on their roadmap.

Out of curiosity, is there a technical reason why your org isn't using a supported version, or is it an "ain't broke" situation? I'd ask migrating to ArcMap to Pro, but if you're on 10.2.1... yeah.

[1] https://support.esri.com/en/technical-article/000013224

[2] https://support.esri.com/en/Products/Desktop/arcgis-desktop/...

Hang on a second, you consider Jan 2014 to be "very, very old"? Commercial software often has dependencies on libraries going back decades.

A finished software product that gets regular updates is old by the time it turns 7, I'd agree on that.

Dependencies on libraries is a different story, there's only so many ways you can implement a functionality, and some of these happen to be decades old!

It can be "very, very old" in its context. If a new release comes out every year, support duration is 5 years and you're 2 years past EOL, I'd agree that the software can be called "very, very old".

And that's where things go off the rails. A release that is only supported for five years is a hobby project.

The kind of software projects that make the world go round continue past the lives of their original authors and can easily span decades. 5 years is just enough for the original shake-out.

Yes, ok, thanks, the Python foundation will immediately spend a lot of time and money to support releases for 2 decades. How could they not know!? /s

That out of the way, you're right that some niches require a lot longer cycles, but it's the big big biiiig advantage of FOSS. Downstream can maintain it for as long as they wish. As you said things got shaken out by the community for free basically, and if some serious software is so so serious that upgrading and retesting/certifying is somehow more expensive than trying to airgap an EOLed pile of libs (while at the same time it needs support) then the stakeholders can do it.

No sarcasm needed, Red Hat happily invests time and money in supporting it until 2024.

That might have been an easy eay to provide upstream releases too, had the Python maintainers not been intent on using deprecations as an instrument to get the community moving.

That strategy doesn't work very well however, as we've seen when TLS 1.2 was held off from Python 2.

I mean i am sure we would do that if any of the user company would want to pay the price for that. None of them want. You probably multiply the price by at least 10 (i would say add at least +1.5 to the multiplier for every year of support and that is probably highly conservative).

Come back when they are ready to pay for that.

> A release that is only supported for five years is a hobby project.

I believe what you are trying to say is that you chose the wrong tool for the job. It is really condescending to lash out on other projects like that just because they don't share the same needs as you. They owe you nothing. Python is free and open source, just fork the damn language spec and support it yourself.

The discussion is about ArcGIS, not Python.

The discussion is about a specific version of ArcGIS that only supports a version of python that reached its end of life, so I beg to differ. My point stands: if you had the need for a really long supported tech, in the order of decades, choosing a tech based on a dependency that had a clear life span of 5 years or so was a poor choice. If, still, you need that, you can support it yourself, by forking your own version, but python owes ArcGIS nothing.

>A release that is only supported for five years is a hobby project.

Sure, I'll let my Product Lead know that we're selling toys that only skiddies, not telecos NOCs use.

Totally agreed, but check upthread for a comment that literally says this tool is unsuitable for work where lives depend on it. And for telcos that's pretty much a given.

Having worked in a number of Telco NOCs, that's probably not the best example for the point you're trying to make.

> A release that is only supported for five years is a hobby project.

and how much are you paying for support?

It looks like probably around $3000/year.

https://www.esri.com/en-us/arcgis/products/arcgis-online/buy

> The kind of software projects that make the world go round continue past the lives of their original authors and can easily span decades

The name on the software might remain unchanged for decades. That doesn't mean that the software remains identical for that time.

Are you really accusing ArcGIS of being a "hobby project"?

Yes, the technical reason was outlined above. Our Big Software Vendor in the field depends on specifically the products of 10.2.1. They're extraordinarily clear on that.

So I assume that the geodatabases, the address locators, and such are all the sticking points in terms of the "products" of ArcMap. I suspect that sometimes their software dips into that specific version of arcpy, as well.

Until the vendor hops, we cannot hop. It's a problem that will be eventually fixed, but I bring it up because many people here at HN think of greenfield development often, and also in industries where tons of iteration are met with approval.

That sounds very familiar. My company deployed a patch so that their webmaps continues working after flash became end of life. We still use 10.2.1 for most things. Thankfully I have ArcGIS Pro on my laptop. I had to move heaven and earth to get it.

But that’s beyond conservative, Python3 isn’t new, it’s more than a decade old at this point.

I can understand that in some industries you don’t easily upgrade, because of testing and certifications. It’s however also possible to put of upgrades for so long that it becomes reckless. If you’re stuck on Python 2.7 in 2021, and your code could kill people, you’re putting people in danger by being reckless.

I’ve also seen people claiming a need for stable platform as a reason to not upgrade. Well, running Windows Server 2008 in 2021 you indeed have a “stable” platform, but it should come as no surprise when modern protocols and encryption isn’t working.

Yesterday in the Java thread I mentioned having done some Java 1.4 stuff in 2016.

The customer was running Red-Hat Enterprise 5.0, and it was going to stay that way as long as Red-Hat would be willing to support it.

> The customer was running Red-Hat Enterprise 5.0, and it was going to stay that way as long as Red-Hat would be willing to support it.

Making the migration when RH stop supporting it extremely complex and expensive, instead of iterating.

Oddly, not killing people is the reason for the very slow iterations.

> If you’re stuck on Python 2.7 in 2021, and your code could kill people, you’re putting people in danger by being reckless.

It's comments like this that make me very sad for the software industry as a whole.

No, he's not being reckless and he's not putting people in danger. The people who make breaking changes to languages, APIs, operating systems and libraries are the ones that put people in danger and that cause others to stop upgrading because they quite reasonably fear breaking changes that could have far reaching consequences.

The price of backwards incompatibility is poorly understood.

Breaking changes to languages, APIs, operating systems and libraries are not putting lives at risk. All of the above have expressed their compatibility guarantees in clear terms. No one advertises Python as suitable for life-critical applications. It's the responsibility of the person who choses to use it for life-critical applications to understand its change lifecycle and put that into their evaluation.

While I agree that in general we should care more about backwards compatibility, don't put the blame on the maintainers for bad choices in life-critical domains. Most domains are not life-criticial and it's entirely reasonable to make and meet a set of compatibility guarantees which applies to the domains you're working in.

> No one advertises Python as suitable for life-critical applications.

I think you just relegated python to the status of 'toy language'.

But that's the wrong attitude. Just like everything will sooner or later connected to the internet, which is why it matters that it is secure, with 'software is eating the world' as pretty much the mantra of HN you should realize that any piece of software, no matter how trivial has the potential to affect lives and sometimes in the most direct way with the ultimate price - of someone else's life - as the consequence of our collective failure.

And that's because of the very nature of software: it gets re-used all the time. One persons application is another persons library, one persons application is another persons service and so on.

Leaky abstractions are bad enough, leaky broken abstractions can sooner or later turn out to be dangerous.

Of course whoever deploys the software has the responsibility to check it for suitability. But in principle any language, OS or library has the potential to be applied in ways the original author did not foresee and if that potential can extend to life critical or life affecting purposes then that should come with a similar standard of care.

How many machine learning systems are written in Python? How many of those make decisions every day regarding people's lives, some of those life critical? Are you seriously suggesting that all those people made the wrong decision in picking the language the application is written in?

If so then you are most likely at odds about this with just about every developer out there who - rather than arguing with their manager about whether language 'x', 'y' or 'z' is suitable for the job will simply be told what language is on the menu and to do the job in that one based on availability of programmers and whatever is 'hip' at the moment. These decisions are not necessarily made in the most rational manner and whether you agree or disagree with that in principle it is the reality that we live in.

And so 'toy' languages such as Python, PHP, Perl, Visual Basic, FileMaker and so on are all going to be used for mission critical stuff, whether you add a disclaimer or not.

Better be aware of that before you throw your brainchild out into the world: you own it.

If the actual expectation of every piece of software published is that the author is responsible for every harebrained choice made with it, then nothing would ever get published. That simply isn't practical.

My responsibility is to meet whatever guarantees I make when I publish a tool, and to open source the code if I plan to stop maintaining it such that it no longer meets the original guarantees.

If you want to lay at my feet the consequences of everyone who chooses to use it, whether it's an individual dev or some executive mandating its use to their whole org, I think you need to reconsider your perspective.

The general attitude to software development is why we really should not be calling it software engineering. More like software plumbing. And even plumbers tend to carry liability insurance.

If you have a plumber fix your toilet, and multiple years later it breaks, and you go to them expecting their insurance to pay out for a replacement, how do you think that conversation will go?

If you hire an engineer to build you a bridge, and years later you are jealous that every other city has two-level bridges, do you imagine they will rebuild it for you for free?

I don't understand how software is any different.

I think our industry has too much sympathy for companies running obsolete software for decades because they can't be bothered to spend on keeping their software current and up to date. If it's mission critical and obsolete, you messed up as a company. If you allowed that to fester for a decade plus, you've been negligent.

Now a lot of python stuff isn't really that mission or safety critical. So, a hopelessly outdated red hat server that hasn't been updated in years running some software developed over a decade ago may be convenient to keep up and running but if it's mission critical you could have acted by now. The good news is, you haven't updated in years in any case so why start caring about that now? Either way, it's your problem; own it and deal with it or not.

The source and binaries are still there and will still work a decade down the road. A lot of banks do just fine running software written in languages that stopped being fashionable half a century ago, where the developers have long retired/died, suppliers have long disappeared. Running their stuff on (typically emulated by now) hardware and OS that has gone out of production decades before the end of last century. Same thing. If push comes to shove, you can pay people to fix things for you. Banks do this all the time. But expect to pay for the privilege. Or just fix it properly, finally.

Python 3 was always going to be backwards incompatible. That was the point. It was also why you where give a decade to migrate.

This is the same stupidity you see in hospitals, with medical equipment running Windows 7 or older. The companies knew when they sold the equipment that it have a service life beyond the EOL date of the OS, yet make no plans to provide updates. The customers where also given all the details, to ask about support lifecycle before buying, and apparently also don’t care.

The sad part about the software industry is how little some people care about providing their customers with the updates and support plan they should be entitled to.

Saying that it’s been more than a decade is not exactly fair. Yes the language was out then, but it took a while for even for basic practices like 2 and 3 compatible code bases to emerge, and for any of the most popular libraries to support 3. Even 5 years ago, maybe even 3 years ago the “is it compatible with py3 yet” page still had some non-trivial gaps.

So it’s not like it has been feasible for most projects to migrate for a full decade, it’s only been a few years.

Well, the good news is, Python 2.7.2 still works, as does pip 20.

If you haven't already, I'd recommend setting up a PyPI proxy and backing it up. I don't know what PyPI's policy is around supporting old packages or versions of the package manager are, but I have learned the hard way from working on other legacy software stacks that things have a way of disappearing from the Internet when you're not looking.

Well, Python 2.7.2 works, but I have never been able to get pip to work at all in my install. Instead I get a circular chain of dependencies ...

So if it isn't in the standard library, I haven't been using it. Pity, because lxml seems like it would solve some of my issues.

At my current company we’re in an awkward situation with python 2.7.1 and 2.7.5, I’ve found that a easy solution to circular dependencies caused by the new resolver is installing pip==20.2.4 and an older setuptools (I don’t have this version memorized) before doing anything.

That's a version of python that does not send SNI when opening TLS connections, so sites that don't have a dedicated IP address for the domain cannot support TLS with such clients, and most sites that do have a dedicated IP address won't support TLS "just because" (no SNI - get self signed dummy cert or closed connection).

So the first thing I would have to do with such Python is monkeypatch ssl.

I had sort-of traced down the problem to something involving SSL, but I am glad to see that someone else has confirmed it. Thank you.

Sadly, I have no idea how to monkeypatch SSL.

Same problem here..

"And there I sit."

There you chose to sit.

Reply on news.ycombinator.com