Hacker News Reader

Comment by bluejekyll | Hacker News Reader

I've always thought that the reason COBOL is still "popular" is that it became an entrenched technology many years ago, and the cost of replacement is high enough (and it works fine doing what it's doing) that no one wants to invest the time and money to replace it's usage.

This quote made me pause though, "COBOL remains the preferred choice for systems where application quality and operating cost remain important considerations". How much is COBOL being "chosen" vs. COBOL is basically required b/c of the existing codebase and systems that need to be integrated with?

The latter is my guess, and the choice seems to be really, "what's the best way to extend this existing system?" If that system is in Java, it's probably Java... C it's probably C... COBOL it's probably COBOL.

View on HN

I spent ~7 years or so at one of the large credit card companies. The adage "if it isn't broke, don't fix it" was very prevalent there, and anything that even hinted at touching the core payment processing code was scrutinized heavily and usually avoided entirely.

At the time, the cost of an outage was estimated in the $100K's per minute, and is likely significantly higher now.

When I moved away from FinServ, I found it surprising at first how cavalier some orgs are about ripping/replacing core services in the name of "modernization". Move fast & break things and all of that...

As someone who was recruited to a financial firm a year ago to execute a modernization effort, I think the difference is due to (simplified) two types of firms: Big ones with huge moats often protected by regulators and profitability, and ones that are insecure. There is a pretty big population of companies that are at huge risk of being disrupted in the best sense of that term, and that's why they need to modernize. Many of these outdated core systems don't have modern capabilities, such as being desktop apps that can't integrate with the web or provide/use APIs without heavy refactoring anyway.

Those big companies can be disrupted even faster if they break something though.

I'm not allowed to talk about it, but I know people in tier-2 support who were on an all night phone call with a big bank customer because if the computer wasn't up by morning the government would shut them down. (somewhat like the government did shut down some big banks in the 2008 crisis, but this was in the 1990s) At the start of the call the rep told them "We are pretty sure we know where the problem is and it isn't you, but we need you on the phone anyway just in case anyone has a question you are ready to answer." That is also why big companies have the million dollar support contracts: someone who can stay up all night and answer obscure questions.

Break things is a bad thing sometimes. I wish more of the move fast and break things companies would realize who annoying it is when they break something.

Some years ago, one of the UK's air traffic control systems needed to be uplifted off the deprecated hardware it ran on. Going anywhere near the code itself was a total no-no - even though there were known glitches, the risk of touching that code was just deemed too high. So they instead wrote an emulator for the old machine and ran the unaltered s/w that way. Never underestimate the power of legacy that works.

The sentiment ITT seems to be overly dismissive of the value of a system that can reliably run for decades. How many of the new Kafka systems being stood up today would you realistically expect to still be running in 2061? How reliably do you expect those systems to perform over that 40 year period?

The trade offs are quite apparent, but the value these systems provide is perfectly obvious too.

I’m not sure I buy the extensibility argument either. In my experience the COBOL typically runs on the core processing system, the bit of the bank that runs the central ledger. This is usually a small portion of the banks technology footprint, and the rest of their integrating systems can run anything they like. I’ve worked in quite a few banks, and in my experience .NET is a lot more ubiquitous than COBOL is.

> The sentiment ITT seems to be overly dismissive of the value of a system that can reliably run for decades.

In my experience it's a very narrow kind of reliability, where no-one cares whether the calculations are right as long as they're the same ones you've always run, and no-one's ever validated what or why is being calculated or sanity-checked it. Probably one of the vital constants in the system is 10x what it should be because someone made a typo it in 1982, but that's still considered "reliable".

> How many of the new Kafka systems being stood up today would you realistically expect to still be running in 2061? How reliably do you expect those systems to perform over that 40 year period?

If they were babied as much as the COBOL systems (no changes without months of testing, if data comes in in an incompatible format then we preprocess it...) then I'd expect them to last longer. They probably won't be, but if you decided to run the Kafka systems in a reliability-above-all-else way, I bet they'd do it better than the COBOL ones.

> If they were babied as much as the COBOL systems (no changes without months of testing, if data comes in in an incompatible format then we preprocess it...) then I'd expect them to last longer.

I'm skeptical on a pure hardware basis.

Your classic mainframe was a very expensive piece of kit and outfitted as such. Everything redundant, self-monitoring, and hot swappable. The hardware support and backwards compatibility runs into the decades.

Today we're renting a cloud service, that's made of pig-in-a-poke hardware. Reliable/high availability hardware is simulated with a bunch of disposable instances, at the cost of needing to design systems at the software level to sync and preserve data rather than having it built into the hardware.

I'd be pleasantly surprised if I can snapshot an AWS/Azure/GCP image today, and fire it up in 2041 with no changes. (Hell, we have no assurances they'll be offering native x86-64 hardware in 20 years)

> Today we're renting a cloud service, that's made of pig-in-a-poke hardware. Reliable/high availability hardware is simulated with a bunch of disposable instances, at the cost of needing to design systems at the software level to sync and preserve data rather than having it built into the hardware.

Right, but Kafka's already set up to do that. I've worked on Zookeeper clusters that have been continuously up for 10 years - not the individual hardware, but the overall system was continuously available for all that time.

> I'd be pleasantly surprised if I can snapshot an AWS/Azure/GCP image today, and fire it up in 2041 with no changes. (Hell, we have no assurances they'll be offering native x86-64 hardware in 20 years)

We'll be able to emulate them, and even if not, the Kafka stack is all JVM-based, so you don't need x86 to run it.

The Cobol code is mostly handling transactions that result in money going from one account to another. The people who own that money tend to scrutinize this very carefully! So after 40 years, pretty much any bug that occurs in practice has been squashed out of existence.

This doesn't mean that every function in the code works correctly for arbitrary inputs that appear to be allowed by the function interface (formally a bug!) but it does the right thing for the inputs that have historically been provided (in practice very stable and reliable).

That kind of reliability makes it hard to rewrite the code. There are cases for historically unused input combinations where you have no particular reason to believe the old code is bug free. So when your rewrite produces a different result in an obscure case, it is a long and painful business logic task to decide which version is right.

> The Cobol code is mostly handling transactions that result in money going from one account to another. The people who own that money tend to scrutinize this very carefully!

In my experience that's only true for a minority of them. The likes of e.g. VAR calculations were often straight-up wrong. And even for monetary transactions, you only have to look at the UK post office lawsuits where the code was simply wrong but rather than checking the code they spent years assuming that any discrepancies were the results of employee theft.

> This doesn't mean that every function in the code works correctly for arbitrary inputs that appear to be allowed by the function interface (formally a bug!) but it does the right thing for the inputs that have historically been provided (in practice very stable and reliable).

It's stable and reliable as long as nothing changes. But the world is always changing.

> That kind of reliability makes it hard to rewrite the code. There are cases for historically unused input combinations where you have no particular reason to believe the old code is bug free. So when your rewrite produces a different result in an obscure case, it is a long and painful business logic task to decide which version is right.

Right, but in practice you get an absurd double-standard. The old code is just as likely to be wrong as the new code, but no-one cares about the business logic, they only care about whether the result is the same as the old code.

Confluent managed Kafka offers an SLA with a downtime of 20 minutes a month… for mission critical real time services (rather than async background processing) that won’t cut it.

I manage some ColdFusion apps still going strong.

>the cost of replacement is high enough (and it works fine doing what it's doing) that no one wants to invest the time and money to replace it's usage

That's pretty much what it is. They're not hard to manage as they are and ... doing their thing.

Here's one I worked on over a decade ago:

https://www.faculty.uci.edu/

It's a little clunky, but still going strong!

Honestly, I've found ColdFusion is a great solution for a lot of web development problems. It tends to break down with very complex apps, but for a lot of simple-to-mid comexity apps CF is great!

I wonder if it's less COBOL itself and more the entire architectural and design-decision stack associated with traditional COBOL software. I'd expect you might see similar concepts with Ada, except those come with security-clearance requirements too.

For example, the entire "mountains of dependencies" mindset feels relatively new. There was a time before .node_modules, Composer and Ruby Gems.

If your codebase is legitimately decades old, your dependencies are either "something we built 100%" or "something we bought/fully licensed and retain 100% control over our copy" Even if you aren't worried about a left-pad incident breaking the entire payroll system, you probably have compliance/audit mandates that are much more easily satisified by saying "here is the master disc that is untouched since the Carter administration".

I'd also expect that the mainframe/batch-processing mindsets have wildly different philosophies on reliability and performance than most consumer services. You can't get away with returning a cutesy animated gorilla and telling people to retry a $7 million B2B payment, or developing elaborate GraphQL-ready chainable APIs when your connected device is a 20-year-old fixed-function terminal with a few dozen kilobytes of working memory.

Now, I'm sure you can build to those same constraints in Python or Node or C# if you want to, but the ecosystem is not really built to support you. There's a lot more move-fast-and-break-stuff.

I've always been conceptually interested in COBOL-- I have that "born too late" mentality; I missed when "you work in the same room as THE COMPUTER" was impressive and you had to wear a labcoat. For me, the problem is how you get started in the sense of "what's the toy problems to even start grasping syntax and build flow." What's the COBOL answer to the "demo to-do list" for web technologies or the "half broken Blackjack game that doesn't actually pay correct odds" everyone wrote in interpreted BASIC in the 1980s? Are there even readily usable toy problems that play well to its strengths?

Reply on news.ycombinator.com