What if we could transpile COBOL into Elixir | Hacker News Reader

What if we could transpile COBOL into Elixir

dockyard.com

58 pointscgarvis63 comments

There are plenty of ways to port out of COBOL, none will work.

I once interviewed for a COBOL position. It was a typical large financial institution. It was a lead position that wanted a migration plan out and then would lead to a team for me. I have a lot of systems integration and information management knowledge. I have worked on planning multi-year projects and migrating out of large systems like this before. The interview went well, and afterwards they showed me around the place and what they were doing.

I sat down again in the office with the hiring director, and I enquired about how long he has been with the company and what he has been working on. Turns out he had been hired in five years ago and had already attempted this once before without success. I knew without a doubt why they didn't have success, and it had nothing to do with COBOL.

Financial companies are highly risk adverse. COBOL developers know this. COBOL developers know that if the shop isn't COBOL their job is at risk. So, COBOL developers will constantly introduce "what about this risk" issues, which then must be considered in committee; thus, the company is eternally paralyzed. The fact that the director couldn't make it anywhere in five years meant nothing was going to change.

As I recall, I checked in a few years later and they were still in the same place. I suspect it might have been a decent place if all I wanted in life was a paycheck. They were nice people.

Are these COBOL programs running on obsolete hardware, or are the systems virtualized these days?

I suspect the way forward for COBOL is to modernize the development tools around it, rather then rewrite the code itself.

Have a look at Micro Focus compiling COBOL to .net and java. They’ve been doing this for minimum 11 years now.

Obsolete hardware? Mainframes are still a thing.

Why? I could never understand why do simple financial formulae coded in COBOL need some colossal computers and still choke on laughable volumes of documents.

I believe my almost-decade-old smartphone has more than enough computing power and bandwidth to process all the unemployment applications even if all the Americans would file them on the same day.

If you can deal with the price tag, mainframes are very nice machines in a lot of ways (scalability, availabity, storage). And if you're a large-ish bank or other financial company, chances are the amount of data you need to process and the requirements on availability you need to meet mean you're going to need something mainframe-sized anyway.

Plus, IBM and assorted companies treat backward compatibility religiously, so a random update to the OS or the DBMS breaking your application is pretty much unheard of. That kind of reliability is worth a lot to some companies.

The price tag is rather harsh, though.

Mainframes aren't just about computing power. In a mainframe, almost any component or subsystem can fail with no downtown to the applications at all. That is worth the money and complexity to some people.

You don't understand mainframe systems well if you think they "choke on laughable volumes of documents".

The mainframe isn't the bottleneck in systems like this. It's all the stuff around it that wasn't designed to the same specifications.

Its not about the complexity of the solution its the scale, mainframes batch processing was the go to solution before the advent of modern hardware. Mainframes were build for resilience but nowadays we have embraced fail early, fail fast to circumvent those scenarios.

Because those simple financial calculations need to be done reliably, for many, many clients at once, without falling over. You can requisition redundant cloud instances equivalent to tens or hundreds of real-world x86 computers, or you can do it on a mainframe. My guess is the cost works out about the same either way.

(Insert image here of IBM System z as Omni-Man, pointing to two fighter jets in the distance marked "AWS" and "Azure" and saying "Look what they need to mimic just a fraction of our power!")

Costs probably dont work out the same, if it were true why is IBM offering its cloud solution on x86 hardware and not mainframe?

Why is their cloud on x86?, an easy argument is because mainframes don’t run x86 or ARM instruction set, https://en.wikipedia.org/wiki/Z/Architecture

How will people use it?

Why would anybody need actual x86 or ARM or any other specific low-level instructions set on a server? I doubt server side developers program in assembly often. As long as there are a performant JVM, a HTTP server and a database management system available, everything is possible and almost nobody (but system programmers and also those who pay for the spare parts) really have to care about whether it is x86 or what.

Of course you have to care about architecture. ARM has been in the making for a long time now, and I can only imagine how far behind z is in terms of software support. To succeed, IBM would have to get Linux, Postgres/MySQL, Nginx/Apache, Java/Python/NodeJS, and the rest of the whole modern x86 ecosystem to work on z—that project would never be done. Or they could build their own, different tools—but they kind of already have, and why would anyone switch to those? To catch up with the other cloud providers, IBM needs to start with what everyone's already using—x86—and use their capital to build an advantage from there.

Rock-solid reliability for large-scale transaction-heavy workloads.

Reading all these replies, my takeaway is that there’s a lack of understanding in the internal structure and architecture of a mainframe and how it works.

Not just code, but how things are built and connected.

> Financial companies are highly risk adverse.

The financial companies running COBOL are. Generally it's quite boring, staid retail banking or the like. If you're a developer who sits on a trading desk writing code which goes straight into production for a trader, things get much more risky.

Indeed. Financial companies are in a difficult position. COBOL was invented for them 60 years ago. Then, they were the primary users of computers. Now, they are just one users among many and no language is going to be invented for them specifically. Instead, you have thousand of enterprise software companies that are trying to market them all sort of complex solutions and keep the gravy trains coming. Financial companies would need to take the matter in their own hands and invent their own modern language but their culture doesn't allow it. Financial companies have extremely long term needs but are unable to put together the kind of long term project that would allow a cohesive language to emerge. Instead, they buy into ridiculous short term thinking like Agile. If maintaining COBOL code from 30 years ago is costly, imagine what today's business software 30 years from now. I have seen critical information only a few years old being lost among insurance companies. Consequential bugs that lead to people losing money. It's scary to think that those systems are guardian of our finances.

I'm curious what features this hypothetical modern language would have that would be particular suited for financial institutions. This thought has piqued my interest

maybe put a framework on top of COBOL call COBOL-x, there use that as your starting point.

kind of joking/not joking, marketing has gotta play a huge role in that transition.

The language need a fairly limited set of functions to support actuary calculations. You don't want things to get out of hand, with side effects, complex libraries, and so forth. Things need to be robust. Language like PL/SQL and SAS were fine but the systems are a pain to deal with and require technical knowledge that actuaries don't always have. Python is no emerging in the field by default because it allow quick prototyping but I would have concerns of the long term stability and maintainability of anything built out of it.

I worked for a Wall St. trading firm in the early 90's, as a greenfield developer doing apps on Sun Sparcstations in cfront based C++ using Motif.

Two things shocked me at the time.

1. The company had a LOT of systems running in COBOL, for which they had lost the source code to the vagaries of time.

2. One of the big projects at the time was because of the lack of source, and the devaluation of the Italian Lira, they needed a "splitter"/"unsplitter" to take particularly big bond trades in Lira that overflowed the (eg) `PIC 9(9)` data clause for the total price, split them up into smaller trades, run them through the existing clearance system, and re-aggregate them into 1 trade after clearing.

Ironically enough, the way to get rid of COBOL is more of it. Democratize it. Right now, you can "learn COBOL" but you'll never get access to a mainframe environment or the actual COBOL compiler or the obscure database used by hospitals and banks, because they're too expensive. So, make emulators so that non-COBOL programmers can play around in a COBOL world without going to work for a bank or hospital. The problem with COBOl is, it's all-or-nothing. You're either in the COBOL world or you are not. But if you add an onramp, Java/C++ programmers will maybe make COBOL versions of their libraries. There will be more people going to COBOL StackOverflow. There will be more cross-pollination between COVOl and non-COBOL devs. More people will form businesses trying to bride the gap between COBOl and Java. Right now, only COBOL devs are trying to get out of COBOL-land. And their vendors will sell them overpriced solution. The Java developers on the other shore can only watch with fascination, they can't help.

> What if we could transpile COBOL into Elixir

You've then successfully ported your code base from one language for which it's really difficult to find programmers to another language for which it's really difficult to to find programmers.

I guess your comment was in jest, but I can find plenty of developers for elixir that are willing and able to learn it in a month if I want to.

Do you think they’d be happy to maintain a transpiled COBOL code?

No, of course not, I was just getting at this specific point that there are few COBOL and Elixir developers. Which, while true, has little to do with the talent pool you can draw from.

Makes sense. I’ve done some fair share of Erlang myself some years ago.

My company (I'm not the hiring manager, just in the know) has been struggling to find any kind of programmer who either knows Elixir, or who is willing to learn.

I'd love to help them along because the workload is getting rough. Are there any tips you can share about finding them? Where do you post ads, what's the process like, remote or in person? Anything would help. Thank you!

Any open positions?

Not OP, but we've got some open Elixir positions.

https://grnh.se/b87ce54f2us

Happy to take a look at your company as someone who's been picking at Elixir/Erlang in my free time for a bit :)

Did you guys post on the Aug 1 "Who's Hiring" thread? I got more than 5 responses to that for an Elixir job.

Damn I got rejected for a few Elixir jobs because of high/interest low professional experience with the language on my part.

I wish this weren't true but I'm contributing to it. I write Elixir for hobby projects and would love write it professionally, but I'm too happy at my current job. One of these days.

My current company hasn't really had a harder time finding Elixir devs than my last company had finding Rails devs.

You can transpile anything to anything, but computers will not be able to rewrite the code to match idiomatic style in the new language. Rather, (taking the example from the article) you get COBOL style in Elixir syntax and everyone is unhappy. Much better options are to maintain a COBOL team that continuously updates the program during its lifetime so that knowledge of how it works is never lost, or (if that is no longer possible) do periodic rewrites every two decades or so.

If your first response to this is "but that would cost a fortune!", you are completely correct. The biggest illusion in software is that a "complete" program is now "done" and will need no more investment. Software is a machine for producing information, fundamentally not much different from a lathe or a textile loom. If your company depends on a piece of software to continue existing, it is now de facto a software company. Many institutions have not yet realized this and will run into existential problems within a few decades.

Several Clojure libraries would like to have a word with you ;)

Disclaimer: Neither a fanboi nor user, just read that potential users thought those libs were abandoned, since they havent been updated in years.

The issue typically isn't just the COBOL itself, but the surrounding architecture. This will take in the nuances of the underlying operating system, system language operations and batch tasks.

For example, the COBOL might potentially have been on an ICL VME system. This will undoubtedly have a lot of support code written in things like SCL (System Control Language), and potentially even 68k assembly.

There are companies which specialise in migrating COBOL code from legacy environments. This can involve migrating the COBOL code itself to a different variant and rigorously testing it, and then often introducing utilities which replace operating system functionality which was present on the original mainframe.

The alternative is to "salami slice" bits of functionality into modern services written afresh, and slowly work towards decommissioning the old setup, but often this has to be done at the same time as a migration to modern hardware, due to the scale of the systems involved and the time taken for these sorts of projects to complete.

Why would there be any 68k assembly floating around in this domain?

I feel a need to comment but I can't find a form of words that isn't condescending to the point of rudeness. So this will probably come off as rude...

Transpiling, to more or less any language, is the easy bit. By a very, very long way. The hard bit is everything else. I mean, do they really think they're the first people to think about using a transpiler over the last 30/40/50 years?

If they had said that they'd used their transpiler on a real production codebase and their transpiled code was now running in production in any volume, I'd be quite impressed. But they didn't say that because it's blatantly obvious from the tone of the article that that will never occur.

Now, I don't have any issue with people being ignorant of a subject that they don't understand. That's the norm for most people for most things. But I have to say that this level of naivety doesn't reflect well on their company. They clearly made no effort to understand the problem that they are purporting to solve. Is that really how they function as a company?

The Japanese have a philosophy whereby they will rebuild a shrine every 20 years. The knowledge of what is necessary remains current, and understood. And the shrine continues in its purpose.

Perhaps we should consider how we can regularly rewrite different parts of our software ecosystems using new languages and approaches while preserving the essential knowledge and security aspects of what we have already built?

The important part isn't the language that the code is written in, it is the full understanding of the problem and thus the solution provided.

IMHO a more realistic idea is to treat the COBOL code as the "idea" or "essence" of the shrine which eternally remains the same, and only rebuild the "physical presence" of the shrine, which would be the environment the COBOL code runs in (e.g. modernize the hardware, the operating system, the debugging/development tools and virtual environment (== emulator) the COBOL code runs in, but don't rewrite the COBOL code itself).

But then you have solved nothing...

Maybe, but "nothing" is still much better than making things worse (first, a rewrite usually needs to be bug-by-bug compatible, so you have almost no leeway to "improve" the product, second, new code has new bugs that need to be found and fixed, don't underestimate the amount of work that has gone into a 50 years old code base that still does its job).

I don't think new languages and tools are necessary in all circumstances, I imagine part of the process of rebuilding every 20 years is to not forget how to build in the first place.

Being around for the frontend engineering revolution taught me what it means for an industry to forget how to build, and have to re-learn many already hard learned lessons.

But maybe I'm looking at it wrong, and it was that re-learning process that was the re-building after all? I don't know. All I do know is that you rarely get enough time in software to master your tools before everything changes, and you have to build the same old software requirements in brand new ways.

Basecamp does this.

Every ~6 years they completely rewrite Basecamp from the ground-up and sell it as a new product management tool. They are currently on rewrite #3 and rewrite #4 will be released next year.

https://signalvnoise.com/posts/3856-the-big-rewrite-revisite...

The only difference to your analogy is that they continue to support the old versions of Basecamp so that if a customer doesn't want to migrate to the current version, they don't have too.

Isn't that what the Frontend Ecosystem has been doing, but instead of 20 years it is more like 20 months?

You will have solved the easiest part. Now get started on replicating JCL, VSAM, CICS, IMS, RACF, and the rest of the ecosystem that COBOL code depends on :)

Back in the late 90s I had the bright idea of shipping our COBOL runtime as a browser plugin. This worked relatively well for a fraction of our customers since we had a transparent remote ISAM & RPC system the plugin could use (the idea being that the browser plugin meant you only had to update your Windows desktop clients with new versions of the runtime; they'd get the latest app on load) but the rest had a huge thicket of system-level dependencies which made that hard to consider, and that was for customers who'd already been able to migrate off of mini/mainframe systems to Unix.

At this point in time, the mistake is to hear “COBOL” and think about a programming language rather than environments where they've struggled to manage technical debt and staffing issues for decades. It's not that COBOL developers can't produce good code but rather than places which can modernize effectively did so 3+ decades ago. If some place makes the news now because of a COBOL-based system, COBOL is a symptom of a management failure.

Ah, a cobol browser plugin? Cool. And yeah, environments. It's like porting JS to the mainframe, but not providing the POSIX facade to allow it to make files, schedule batch jobs, kill processes, connect to the network, store data in a database, access rights, auth, etc.

Yes – AcuCOBOL had a COBOL development system which was byte-coded like Java (the technical founder had actually overlapped with Gosling in undergrad, I believe) so we already had the capability to copy the compiled program onto most platforms (Most Unix variants, VMS, Win16 and Win32, OS/2, DOS, OS/400, etc.) and launch it with the native runtime. There was also some ability to transparently redirect I/O — you could have any CALL which didn't wasn't in the local compiled code do an RPC call to a remote server, and redirect some or all ISAM operations to either a remote server or (with another license) a SQL database.

Once they had that infrastructure, it wasn't a huge amount of work to package the runtime as a plugin which was configured to enable the remote call & I/O options by default. Which is exactly restating your last point: having already done the hard 95% of the work, this part was straightforward.

Please no...

I've been unfortunate enough to witness first hand what +-5,0000,000 lines of Natural Adabas code transpiled to +-15,000,000 lines of Java code looks like, and it was not pretty.

This. Friend of mine worked at a company that did COBOL to Java compilation, with the intent of producing "maintainable" code, and he reported (after several years of work) that this just wasn't possible (for them).

I would like to know more… 8-)

This is from hearsay, since I was only there a few weeks before getting out, so I only spent some time on the floor and looked at the code for a bit:

It was a municipal management system that grew over time (decades).

At some stage a government standards body raised the legacy risk inherent in the Natural Adabas codebase.

Some (pretty impressive) 3rd party tool was used to transpile the codebase to a more "modern Java web application architecture".

The code wasn't idiomatic Java and used some extension libraries to supply missing capabilities. For example I don't recall ever seeing any other Java system that uses continuations.

I spent an entire day tracing one field from the front-end to the database. I documented the components, flows, logic and db structures and then asked one of the team leads to verify if I got it right. She confirmed that I did. I then asked what the code on question actually does, functionally.

She replied something to the effect of "Good heavens, I don't know either - you'll need to speak to one of the functional guys and then spend some time with the Natural code from before the transpilation".

She needed about 6 months to get a new Java dev to a level where they could do minor feature requests or bug fixes. Understandably her ability to retain developers was quite low - they tended to leave quickly after spending time with the system.

Business didn't seem to get why the velocity for new features was so slow.

They didn't have an architect, documentation was fragmentary.

Previous modernisation initiatives had already become obsolete as well: IIRC the App server was Glassfish and front-end (web) components were using some JBoss widget library that was already deprecated at the time.

Like I said, I left quickly.

I've had to port some old Visual Basic WinForms apps to C#, only a few hundred lines of code for each app, and even that was a huge pain in the butt. Transpiling anything of significant size and complexity will require a metric ton of cleanup and refactoring just to make sure it all works right and can be reasonably maintained by other developers.

I checked my bookmarks, and there have been success stories for porting cobol into other languages. Here's a discussion from 2019, from cobol into Java: https://news.ycombinator.com/item?id=19839563

Phase 1: automation of cobol to (ugly) Java

Phase 2: make the code more idiomatic for (cleaner) Java

Phase 3: move the infrastructure to AWS

In the discussion, someone mentioned "There are commercial COBOL compilers available that compile to Java bytecode."

Offhand, if someone were still on cobol, I'm not sure if they'd trust the comparatively new Elixir, but I can also see leapfrogging being a thing. (And I know it's based on Erlang, which pre-dates Java.)

This reminded me of the old lisp2cobol[0] joke script.

[0] https://stuff.mit.edu/afs/sipb.mit.edu/project/lisp2cobol/l2...

No one running COBOL will use it because those legacy applications "just work". Debugging and making certain the resulted transpiled code does as well will take a monumental effort (read: money). That is why those codebases haven't been ported because, based on similar efforts, transpilers are only partially easier than a manual rewrite.

That answering the what if question. The project itself is interesting.

Why should we? When we have this: http://www.coboloncogs.org

Sorry, no https availlable.

Anybody who is developing a COBOL compiler should know that there's a test suite for the language. See here (https://www.itl.nist.gov/div897/ctg/cobol_form.htm)

Also, I haven't seen any mention of any of the bits of the COBOL ecosystem here? No DB2, IDMS, IMS DB/DC, VSAM, CICS, REXX, JCL? These are a big part of why COBOL applications are sticky.

COBOL code is surprisingly readable when compared to "modern" JavaScript-like languages. The site linked above shows a very good example of this.

Then you could transpile COBOL into Elixir...

Reply on news.ycombinator.com