The Future of Big Iron: An Interview with IBM’s Christian Jacobi
morethanmoore.substack.com
morethanmoore.substack.com
at the heart of the Mainframe success is this:
> I’d say high-availability and resiliency means many things, but in particular, two things. It means you have to catch any error that happens in the system - either because a transistor breaks down due to wear over the lifetime, or you get particle injections, or whatever can happen. You detect the stuff and then you have mechanisms to recover. You can't just add this on top after the design is done, you have to be really thinking about it from the get-go.
and then he goes into details how that is achieved. the article nicely goes into some details.
oh and combine the 99.9999999% availability "nine nines" with insane throughput. as in real time phone wiretapping throughput, or real time mass financial transactions, of course.
or a web server for an online image service.
or "your personal web server in a mouse click", sharing 10.000 such virtual machines on a single physical machine. which has a shared read only /ist partition mounted into all guests. not containers, no, virtual machines, in ca 2006...
"don't trust a computer you can lift"
If it were actually cheaper, IBM wouldn’t be selling these machines so well.
However, he didn't elaborate or give any examples. If I were the interviewer, I would have followed it with: "Oh?! Can you provide some examples for the readers who believe that you only sell to captive audiences?"
If I had the R&D budget for an on-prem VM hosting box, I’d seriously consider their smaller Express 4, which is not much pricier than a similarly capable x86 machine.
Be sure to allocate me a bunch of shares for giving you the idea.
That is why Unisys ClearPath MCP is still a thing, tracing back to its Burroughs 1961 heritage, security above all.
It’s amazing how Microsoft convinced so many companies to shoot their feet.
This is an expanding market, due to the needs for more and larger and faster data processing uses for new tech, new markets, new transactions, new capabilities.
I used to work for Citrix, which is "software that turns Windows into a mainframe OS". Basically, you get remote thin terminals the same as you would with an IBM mainframe, but instead of showing you green text you get a Windows desktop.
Citrix used to sell this as a "cost saving" solution that inevitably would cost 2-3x the same as traditional desktops.
The real benefit for both IBM mainframes and Citrix is: latency.
You can't avoid the speed of light, but centralising data and compute into "one box" or as close as you can get it (one rack, one data centre, etc...) provides enormous benefits to most kinds of applications.
If you have some complex business workflow that needs to talk to dozens of tables in multiple logical databases, then having all of that unfold in a single mainframe will be faster than if it has to bounce around a network in a "modern" architecture.
In real enterprise environments (i.e.: not a FAANG) any traffic that has to traverse between servers will typically use 10 Gbps NICs at best (not 100 Gbps!), have no topology optimisation of any kind, and flow through at a minimum one load balancer, one firewall, one router, and multiple switches.
Within a mainframe you might have low double-digit microsecond latencies between processes or LPARs, across an enterprise network between services and independent servers its not unusual to get well over one millisecond -- one hundred times slower.
This is why mainframes are still king for many orgs: They're the ultimate solution for dealing with speed-of-light delays.
PS: I've seen multiple attempts to convert mainframe solutions to modern "racks of boxes" and it was hilarious to watch the architects be totally mystified as to why everything was running like slow treacle when on paper the total compute throughput was an order of magnitude higher than the original mainframe had. They neglected latency in their performance modelling, that's why!
Mainframe manufacturers talk about "huge IO throughputs" but a rack of x86 kit with ordinary SSD SAN storage will have extra zeroes on the aggregate throughput. Similarly, on a bandwidth/dollar basis, Intel-compatible generic server boxes are vastly cheaper than any mainframe. Unless you're buying the very largest mainframes ($billions!), then for the same price a single Intel box will practically always win if you spend the same budget. E.g.: just pack it full of NVMe SSDs and enjoy ~100GB/s cached read throughput on top of ~20GB/s writes to remote "persistent" storage.
The "architecture" here is all about the latency. Sure, you can "scale" a data centre full of thousands of boxes far past the maximums of any single mainframe, but then the latency necessarily goes up because of physics, not to mention the practicalities of large-scale Ethernet networking.
The closest you can get to the properties of a mainframe is to put everything into one rack and use RDMA with Infiniband.
Or PCIe... I really would like to try building that.
But I'd say it's much closer to Ethernet than USB. You have controllers (routers), switches and nodes... USB doesn't, not like this.
The features of the platform are the real technical edge. You need to use those features to get the benefits.
I’ve moved big mainframe apps to Unix or windows systems. There’s no magic… you just need to refactor around the constraints of the target system, which are different than the mainframe.
There is much less need for most business functions to sit on a mainframe.
However the mainframe offers some availability features in hardware and z/VM, which you need to compensate for in software and system architecture, if failure is not an option, business-wise.
and if your organisation can build such a fail-operational system and software solution, then there is no reason today to stay on the mainframe. it's indeed more a convenience these days than anything else.
Traveling at c, if a signal travels 300 mm (30 cm; 12") that is one nanosecond. And data signals do not travel over fibre or copper at c, but slower. Plus add network device processing latency. Now double all of that to get the response back to you.
When everything is with-in the distance of one rack, you save a whole lot of nanoseconds just by not having to go as far.
Then add in the switching, routing, firewall, and load balancer overheads. Don't forget the buffering, kernel-to-user-mode transitions, "work" such as packet inspection, etc...
The net result is at least 50 microseconds in the best networks I've ever seen, such as what AWS has between modern VM SKUs in the same VPC in the same zone. Typical numbers are more like 150-300 microseconds within a data centre.[1]
If anything ping-pongs between data centres, then add +1 milliseconds per hop.
Don't forget the occasional 3-way TCP handshake plus the TLS handshake plus the HTTP overheads!
I've seen PaaS services talking to each other with ~15 millisecond (not micro!) latencies.
[1] It's possible to get down to single digit microseconds with Infiniband, but only with software written specifically for this using a specialised SDK.
I understand your point about big iron, but where does something like Citrix reduce latency?
My estimation would be: Compared to a desktop Citrix adds distance and layers between the user and the compute/storage/etc., and the competition for resources on the Citrix server would tend to increase latency compared to the mostly idle desktop.
Insert Citrix, and the turd app is 3ms away in the data center and works.
I used to run a 100k user VDI environment. The cost was easily 4x from a hardware POV, but I had 6 guys running it, and it was always consistent.
Note that Citrix or any similar Windows "terminal services" or "virtual desktop" product fills the same niche as ordinary web applications, except that Win32 GUI apps are supported instead of requiring a rewrite to HTML. The entire point is that existing apps can be hosted with the same kind of properties as a web app, minus the rewrite.
Both suffered from the same issue, which is actually very common but nobody seems to know: power efficiency throttling of CPU speeds.
The irony was that the new compute platform had such a huge capacity compared to the old mainframe (20x or more) that the CPUs were only about 1% utilised. The default setting on all such servers is to turn cores off or put them into low-power modes as slow as 400 MHz. This murders performance and especially slows down the network because of the added latency of cores having to wake up from deep sleep when a packet arrives.
It was one of those situations where running a busy-loop script on each server would speed up the application because it keeps everything “awake”.
The telco doubled their capacity as an attempt to fix the issues but this took them to 0.5% utilisation and things got worse.
The health insurer also overcomplicated their network, building a ~100 server cluster as if it was the public cloud. They had nested VLANs, address translation, software defined networking, firewalls between everything, etc… Latency was predictably atrocious and the whole thing ran like it was in slow motion. They too had the CPU throttling issue until I told them about it but the network was so bad it didn’t fix the overall issue.
alas the last bottle of that was served in 1972
the current incarnation is z/VM, which started 2000. it is rumored to have one or two new features.
The heart of the mainframe's continued existence is as you say, combined with risk aversion to changing software that currently works, and a lack of people who understands how it even works to know how to change it.
But the heart of the mainframe's failure to expand is IBM's reluctance to join the modern world in pricing and availability.
There is no simple way to become a mainframe dev because there is no simple way to get access to modern zOS. There is an entry fee. And it's hard to be excited about anything they do to the silicon when you know that you don't just buy the system, you're charged for consumption as if it were a cloud device. So, nobody with a wallet wants to run anything other than the essentials there, hence the platform never grows.
Eric Bier Demonstrates Cedar
This interpretive production was created from archival footage of Eric Bier, PARC research scientist, demonstrating the Cedar integrated environment and programming language on January 24, 2019. Cedar was an evolution of the Mesa environment/language, developed at PARC’s Computer Science Laboratory originally for the Xerox Alto. Mesa was modular and strongly-typed, and influenced the later Modula family of languages. Cedar/Mesa ran on the D-machine successors to the Alto (such as the Dorado) and added features including garbage collection, and was later ported to Sun workstations. Cedar/Mesa’s integrated environment featured a graphical window system and a text editor, Tioga, which could be used for both programming and document preparation, allowing for fonts, styles, and graphics to be embedded in code files. The editor and all its commands were also available everywhere, including on the command console and in text fields. The demo itself is running through a Mac laptop remotely logged into Bier’s Sun workstation at PARC using X Windows. Bier demonstrates the Cedar development environment, Tioga editor, editing commands using three mouse buttons, sophisticated text search features, the command line, and the Gargoyle graphics editor, which was developed as part of Bier’s UC Berkeley Ph.D. dissertation. Bier is joined by Nick Briggs, Chris Jacobi, and Paul McJones.
IBM seems to be milking it in order to support their other businesses. I think if spun off, prices could be lowered and mainframes would start growing again.
I think with this AI bubble, if mainframes were not as expensive, they would have see nice growth.
* https://www.youtube.com/watch?v=-JkJO3rNZ10
* https://en.wikipedia.org/wiki/IBM_Telum
* https://chipsandcheese.com/p/telum-ii-at-hot-chips-2024-main...
(we need a bit of silly in our lives)
My wife ran a billing system that was a Motorola based AS/400. It was purchased in 1989/90. An ibm CE would drop by and swap parts every now and again when the computer phoned home.
They retired it in 2013, when a key part for the printer was no longer available.
I think they had all moved on or disappeared by 2013 though.
The as/400 has always been based on an IBM CPU, but it did use Motorola cpu's in the IO cards.
CEREAL
Oh yeah, you want a seriously righteous hack,
you score one of those Gibsons man. You know,
supercomputers they use to like, do physics,
and look for oil and stuff?
PHREAK
Ain't no way, man, security's too tight. The
big iron?
DADE
Maybe. But, if I were gonna hack some heavy
metal, I'd, uh, work my way back through some
low security, and try the back door.