Hacker News Reader

Comment by ptmcc | Hacker News Reader

> I can only imagine the fat paycheck a 20-year old mainframe programmer would get though, because your age in this case would be invaluable.

It's funny, people frequently assume this would be true but reality doesn't really bear this out. It's typically pretty average to even below average which contributes to the talent pipeline problem.

The other side of it is that its not the technical stuff like "knows COBOL" that is so immensely valuble. Any average dev can "learn COBOL" but that's not actually the valuable thing. The anecdotes of COBOL programmers coming out of retirement for 500k/yr contracts has little to do with COBOL, but their accumulated institutional knowledge of the giant ball of business logic encoded in that COBOL.

If these banks and other institutions actually did write fat paychecks to young mainframe programmers the demographic problem they're facing might not be so bad.

View on HN

I went to grad school with a guy who was in that situation. He worked at a bank and got laid off during the financial crisis. The plan was for the bank to port the old system over to Java or something like that, and they were "close" to the end of the project and they thought at that point the rewrite team was comfortable enough with COBOL that they could do the rest of the rewrite without him. Turns out that yes they understood COBOL just fine, but they desperately needed his institutional knowledge. He ended up agreeing to come back on a part-time basis at some obscene consulting rate for however long the transition took and in the meantime, he did grad school part-time to skill up.

I used to work at a large insurance company with a COBOL core system. I completely agree with this. They paid for people who knew all the undocumented idiosyncrasies and foot-guns in the code. Not just "Knowing COBOL". A decent programmer could read a COBOL program and follow it.

They wouldn't know that the reason the key export fell over when extracting data for your 10K filing because someone had decided/assumed/whatever that a certain record count would never go over 3000, so they hard coded the program to just error out if it went above that value.

When I was working in a finance company that used cobol, sure I could read cobol. I could probably write some cobol. But the extra crap around cobol, how mainframes work, DB2, JCL, etc. is where this gets complicated.

I feel like you and the parent are saying a company pays well if you master COBOL as opposed to just "learning" COBOL.

It has nothing to do with mastering COBOL. It has everything to do with mastering the banking systems.

Yeah, that's true of anything related to the finance sector IMHO regardless if it's COBOL or Java or Python.

The language you happen to be using in your system is almost inconsequential in the grans scheme of things, because it's like 1% of everything that's going on and that you need to know.

Training a GPT on all that Cobol could be invaluable at least as a “footgun catcher” for less experienced devs to act as a code reviewer.

I just shared this story earlier today with my team. This was Cobol insurance claims processing system, shortly before the Y2K issues.

The claims processing system I worked on used a number system that would only go to 4 digits, it would roll over to 0 if we ever processed more than 9999 claims overnight.

Our VP would not listen to us when we said we needed to change that. He said "Claims process fine every night! There is no problem." I had left before they got to that 10,000th claim (thankfully).

That said, anyone need an old COBOL developer?

Edit: now I’m the VP of Engineering. We do work on tech debt regularly.

Fortran, not COBOL, but I came across a similar thing when helping to update code for Y2K. An insurance company used a 2 digit field for year of birth, assuming that it would always be prefixed with '19'. I sort of forgave them because the code had originally been written in Fortran 2 sometime in the 60s, and who would ever assume that their code might still be in use 40 years later?

In the 1960s, there were still 70 year olds born in the 1890s, and 80 year olds born in the 1880s, so it still doesn't seem like a good decision even for something written back then.

In 1966 a 7.25MB drive cost $25,510.

In todays money that's $246,316.70.

So it would have cost 6 extra cents for every date entry.

> If these banks and other institutions actually did write fat paychecks to young mainframe programmers the demographic problem they're facing might not be so bad.

When I first started work I was involved in banking/insurance mainframe stuff, and it was honestly a pretty terrible working environment for reasons that spanned from the mundane (like dress codes) to the esoteric (like the horrible crufty legacy codebase). I would have put up with it if it paid well, but it didn't. In fact, it paid significantly worse than the job I'd had before which was installing physical cable plant (fiber optics and copper ethernet), the reason I took it was at least the office had air conditioning, but it certainly wasn't something I wanted to turn into a career.

As a mid-level windows sysadmin doing customer-facing phone support at a cloud provider, I made nearly 30% more than I had as a junior mainframe guy at a bank/insurance company. The difference in the skillsets and their commonality was massive, yet the pay was significantly worse for the mainframe work despite it being a rare skillset in a high value industry. I remembered when I was in this job that guys who worked on trading desk backend code were far better compensated while working in easier development environments on less esoteric codebases, the mainframe folks were the lowest paid of the developers at that company, at every level of seniority. The only person we worked with who was well compensated was an independent contractor who'd worked there previously for 30 years before retiring.

I had similar beginnings. Late 80s transforming literal reams of 360/70 assembly to “modern” cobol 85. Was still earning my cs degree at the time.

As you know, the pay wasn’t so hot for stuff like that back then. But it was an improvement over manually unloading flatbed trucks stacked with steel conduit, which was my prior line of work!

Today I think it’s much different. Some banks and insurance companies and the like still at least partially run on software as old as the Apollo missions. Maybe I’m wrong, but finding people still alive with that knowledge intact must be difficult and I imagine their compensation reflects that simple supply/demand formula. Shrug.

Yep. I have COBOL experience (on a mainframe even!), and I'd even be willing to do it again -- and I'm young by the standards of COBOL programmers, so somebody could even get 20 years out of me still.

But the wages being offered for those roles are abysmal relative to what's available for other skills, and modern life is applying a great deal of pressure to chase larger paychecks, sadly.

I can vouch that this is true in some cases.

I worked for a multinational bank headquartered in the U.S., whose corporate team came up with a sexy program to snatch up recent Computer Science graduates. The program offered them a "fair" market rate for an entry-level jobs in industry, and let the graduates try two to three different teams and/or departments in their first year. At the end, if the company still liked them, the candidate got to pick their permanent team for a full-time position. This position, however, did not see a bump in pay, and by that time the program effectively filtered out candidates that weren't performing at an accelerated level.

The results were great for the business: young programmers that were often more proficient in COBOL than their new peers were costing them $60k-$70k a year. The senior, or "tenured," peers were in the $200k-$400k range in some cases.

Last time I checked DMV of SC pays $50K salary with no remote work option for COBOL position. And requires 2 years of experience. Granted, it’s a state job, I get it, but still.

It's probably because they technically have to make that job posting and let it sit before handing it off to a contract firm for $300k.

I think so too.

What is stopping somebody who'd be inclined to take a COBOL job from just starting a contract firm and taking the $300k from the government? Does the government check if they actually have employees who have 30 years of COBOL experience before handing out those contracts?

There’s a lot of compliance work involved in getting government contracts.

Also, there’s usually 2-3 layers of pimps (prime and subcontractors) who take a vig at each layer. Total comp for the contractor is usually rate divided by 3 or 4.

Nothing is stopping them; this is exactly what people do.

It's been years since I looked, but mainframe jobs at the big banks in Charlotte were a little less than that.

Whenever I hear about a skills shortage (in any field), I automatically append the phrase "at the salary we want to pay" and that completes the equation.

There’s the apocryphal tale of The Retired Engineer:

An engineer retires from his company, after a 40-year career. He left at a senior technical level.

Some time later, he’s contacted by his old company, begging him to help them fix a problem that has the current staff absolutely stymied. They have been at a standstill, for weeks.

He agrees, and shows up. He sits down at a workstation, examines the behavior, looks at some code, then says, after about five minutes: “Here’s your problem. If you just do this, it will fix it.”

He hands in an invoice for $10,000.

The beancounters flip their wig. “We can’t pay $10,000 for five minutes’ work! Itemize it, and tell us exactly why you think it’s worth it.”

He takes the invoice, turns it over, and writes on the back:

    1) Fixing the bug in five minutes. . . . . . . . . . . . . . . . . . . .
 . . . . . . . . . . . .$20
    2) Six years of college, and 40 years of experience, so I can fix the bug in five minutes. . . $9,980

A great tale. I usually hear a version about a halted production line, with the punchline invoice something like:

    1. Hit machine with hammer ....... $1

    2. Knowing where to hit it ... $9,999

The production line versions have the element of customer losing money by the minute, which suggests that the customer received the value, and is only confused/petty because it seemed so easy for the engineer to provide that value.

I had heard of this re Charles Steinmetz, but apparently it goes way back:

"The earliest instance located by QI ((Quote Investigator)) appeared in “The Journal of the Society of Estate Clerks of Works” of Winchester, England in 1908.":

https://quoteinvestigator.com/2017/03/06/tap/

Another version, IIRC true story, from a small European country. A legendary artist and book illustrator (real person) meets a client in a cafeteria. The client says, I want a drawing with this, this and this, and it should carry a feeling of something like this.

The artist takes out a drawing pad and says to the client, would you excuse me for 15 minutes, I need to draw in silence and solitude.

The client leaves and sits to a different table.

After 15 minutes, the artist hands him over the drawing and names a price.

So much? cries the client. This is quite expensive. You only drew for 15 minutes.

Well, says the artist. In order to draw this in 15 minutes, I have been practicing drawing for at least 5 hours every day for the last 50 years.

Absolutely love this story (and the mentioned artist as well, he was a huge inspiration for many).

In tech, it's only true if you know some dirty secrets. The more dirt you are exposed to, the more you keep your mouth shut, the more it pays. The thing is that teams are so large and compartmentalized that nobody knows the true horror of what's going on.

>It's funny, people frequently assume this would be true but reality doesn't really bear this out. It's typically pretty average to even below average which contributes to the talent pipeline problem.

This, I went into programming over 40 years ago. At the time I worked in a warehouse, I had to take a 5% pay cut (IIRC). But in the long run it was worth it. Most people there stayed in the warehouse/manufacturing due to the pay.

100% true. The value is not knowing COBOL the value is hard core banking or insurance knowledge.

You don't want to know how much you can ask when you are one of the few that can help a multinational migrate away from an old database system ;)

Indian companies are still training tons of new graduates in mainframe.

Reply on news.ycombinator.com