Why Do Big Irons Still Exist?

As a systems enthusiast and someone who has watched as computers got small and then big and then small and then big again, I believe the fundamental answer is based in state machine theory. Specifically around how data becomes "entangled" with other data. That is the essence of what makes transactions hard.

I first ran into this looking at scaling file systems. Unlike RAID where all of the blocks in a stripe are related mathematically, a "file" as a sequence of octets is defined not only by the mutations that happen to it, but the order in which those mutations take place. So "append 1, 2, 3", back up one, append 4, 5" leaves 1, 2, 4, 5 if applied in sequence but leaves 1, 2, 3, 4 if the last two steps are swapped. Thus both operations and the order of the operations are important. To hold the state of a complex sequence stable, you generally have to have it all in memory ready to complete (commit) and then rapidly verify its stable, and then commit it.

Clusters of smaller systems have a hard problem with this. That said, I would love to play with some of Google's spanner systems to see how well they handle the OLTP workload with respect to cost/size/power. The paper suggests that there is a credible path there as flocks of distributed systems get cheaper and more easily connected.

http://ezinearticles.com/?Advantages-and-Disadvantages-of-Ma...

Hard to toss out a trouble- and hacker-free system that has handled everything thrown at it for 30+ years, can run any workload, maintains backward compatibility, and supports new stuff. Channel I/O is also frigging awesome:

https://en.wikipedia.org/wiki/I/O_channel

Note: I wrote in Ganssle's Embedded Muse that real-time could benefit from a beefy CPU and low-end one for I/O interrupts. He agreed and one of us found a SOC that did something like that with quite some results. :)

Virtualization, pay for what you use, hardware accelerators, I/O offloading... all these "new" things have been in mainframes since the 70's. Unlike the modern stuff, the people coding them focus on making them boring, predictable, and reliable. Plus your old software is future-proof and you can do new stuff. So, risk-adverse businesses think they're worth the HUGE amount they spend on them.

That said, there is a negative reason many companies stay: lock-in. The older companies invested decades worth of money in mainframe-specific software and libraries/tools from companies that no longer exist. Porting all that over to modern architectures would cost way more than a mainframe plus have risk of catastrophic failure. So, they just pay the bill each year and accept any improvements they get.

Little known fact: the original high-throughput NoSQL document database was written by IBM and is still around.

https://en.wikipedia.org/wiki/IBM_Information_Management_Sys...

At my last company we had this problem. We had a simple application for tracking inventory that morphed in to a CRM with tons of data from emails, notes, etc. Database grew to be terabytes in size. Using fusion-io cards and replicating to read only databases was required to keep us going and it worked. We had to keep buying bigger iron. A database server with 2TB of RAM!

At my new company, Stackify (http://www.stackify.com), I made sure we gave each client their own database so we knew we could scale out from the beginning. Now we manage over 1000 SQL databases and that has its own set of challenges. We use SQL Azure so that makes it all pretty simple thanks to their new elastic pool features. Versioning SQL schemas and using tools from red gate have helped us keep our sanity.

I always thought it was 'big iron' as in something big and made of 'iron'. The idea of 'Big Irons' makes me think of ironing shirts with some super-sized, barely lift-able iron rather than something the size of my Philips iron. I imagine the steam from a 'big iron' could be quite fearsome.

Having made the link I now have a sensible name for the server room where I work. We didn't put a sign on that door because it would be helpful for thieves. 'Ironing Room' as in what hotels have might be more befitting even though a server and a firewall does not make 'big iron'.

Either way 'big irons' is now added to my lexicon to go along with other deliberate misspellings including 'nucular' and 'skelington'.

Super-linear disk cost back when disks were already atrociously slow compared to the rest of the machine have largely gone away with SSD's hitting huge capacities and tech like NVMe, solid state RAM modules, and Intel's upcoming Optane tech ensuring that more than ever, scaling horizontally can be put off way more than used to be possible.

If you look at scale-out vs scale-up for any applications that were disk limited, disk performance is now ridiculous - > 1GB/s and IOP's measured in 100's of thousands. I'm expecting a bit of a comeback for HA over HP. More than likely, your app can be served well by a single big machine that is well within the linear scaling regime, and you need several for durability and geo-availability.

The question is still interesting, why does Big Iron still exist in High Performance Computing? I'm not completely up to speed, but I think the reason has a lot to do with specialized network interconnects, such as three-dimensional toroidal interconnects [1] ... These specialized interconnects differentiate "Big Iron" from commodity clusters. Another differentiating feature relates to memory, such as very large memory capacities and unique memory hierarchies using NVM, SSDs, etc. A third possibility is very large CPU socket capacities, going beyond the standard dual-socket or quad-socket configurations. This type of technology can certainly play a role in databases and it sheds some light on the "database appliance" trend (integrated hardware/software solutions).

[1] https://en.wikipedia.org/wiki/Torus_interconnect

Given that even Linux is now used in HPC scenarios, are the "true mainframe" OSes really still so different, either in architecture or in average sysadmin experience? Or did the microcomputer OSes effectively converge to have all the same features?

Then Sun pushed out SPARC boxes with 3 mouse buttons (don't even LOOK at those let alone touch those sayeth the sysadmin)

Then we moved everything server side to the cloud, and virtualize everything in our dev environments.

In the future, I predict that things will come around full circle: quantum computing will bring us back to the Big Iron days of "don't even look at it, don't even touch it" but given cooling requirements of QCs these days you won't ever see it.

"All of this has happened before, and it will all happen again."

MS SQL Server DOES NOT run on Solaris on SPARC. The author should have done his/her research before making that claim.