Hacker News Reader

Comment by freeli | Hacker News Reader

A certain nuclear power plant had a Windows NT 4.0 machine running as late as 2007. The reason is interesting.

The machine's purpose was to report status of the control rods that mitigate nuclear reactions. Basically, "are the rods inserted, and if so, how many / how far?". I want to emphasize that this was reporting only, NOT control.

The original software was written back in the 80's, when the plant was originally commissioned, for AmigaOS. Of course, it's hard to buy Amigas anymore, and the original one died long ago (nobody remembers when).

So in the mid '90s, the utility purchased an AmigaOS emulator that ran on Windows NT 4.0, which was current at the time. The emulator (IIRC) was developed by a firm in the UK. The firm went out of business sometime in the late '90s. The control rod monitoring software ran under this emulator on top of NT4.

Windows NT 4.0 was the last OS to allow the emulation software direct access to the physical hardware that produced the status signal. Later versions of Windows abstracted the hardware access away, and the monitoring software broke. Because the emulation company had gone belly up, there was no way to fix the incompatibility.

So the utility had a choice: get new hardware/software certified (by NRC?), or keep doing what they were doing with the software (and hardware) that they had. They chose the latter.

So this is how, in 2007, during a tour of the facility, I stumbled across a Pentium 1 system running an AmigaOS emulator on Windows NT 4.0 that was responsible for displaying the status of the control rods of a nuclear power plant.

Spare hardware for this setup was purchased off of eBay and stocked on an adjacent shelf.

View on HN

To me the funny part of this story is that they used to show you this warning as part of the EULA when installing Windows NT 4, which I remember joking about:

NOTE ON JAVA SUPPORT. THE PRODUCT MAY CONTAIN SUPPORT FOR PROGRAMS WRITTEN IN JAVA. JAVA TECHNOLOGY IS NOT FAULT TOLERANT AND IS NOT DESIGNED, MANUFACTURED, OR INTENDED FOR USE OR RESALE AS ONLINE CONTROL EQUIPMENT IN HAZARDOUS ENVIRONMENTS REQUIRING FAIL-SAFE PERFORMANCE, SUCH AS IN THE OPERATION OF NUCLEAR FACILITIES, AIRCRAFT NAVIGATION OR COMMUNICATION SYSTEMS, AIR TRAFFIC CONTROL, DIRECT LIFE SUPPORT MACHINES, OR WEAPONS SYSTEMS, IN WHICH THE FAILURE OF JAVA TECHNOLOGY COULD LEAD DIRECTLY TO DEATH, PERSONAL INJURY, OR SEVERE PHYSICAL OR ENVIRONMENTAL DAMAGE. Sun Microsystems, Inc. has contractually obligated Microsoft to make this disclaimer.

Also:

It would take a whole lot more context to make this somehow comforting. :D

> It would take a whole lot more context to make this somehow comforting. :D

This is this fascinating 6 part documentary about the Chernobyl incident explaining how it was caused by bad control rods. But the main point is that control rods prevent the facility from going boom, so be glad it’s not the AmigaOS emulator on a NT machine handling it.

LOL at “Chernobyl accident was caused by bad control rods”. This seems to imply other control mechanisms worked, when in fact plant engineers could not even monitor the plant operations. They had malfunctioning Geiger counters. Also - plant engineers had no idea how much water was flowing through the nuclear core cooler. Monitors for the temperature of before mentioned water did not exist even on a whiteboard.

A friend works for an airline as a flight simulator tech. Their entire software stack, including the compiler and OS, is FAA-certified.

Then their ancient Honeywell(?) mainframes reached end-of-life they scouted for compatible hardware, of which there was none. The cost of certifying new software, plus the time involved, was astronomical. So, after consulting with the FAA, they paid a hardware company to clone the ancient mainframes in modern silicon. The FAA signed off on it, and they had all-new computers - much smaller than the originals - running the old stack.

There's a mechanical tank simulator at the Swiss Military Museum that was built in the 1970s and used for actual service before being retired and becoming a museum piece:

https://www.youtube.com/watch?v=AcQifPHcMLE

The simulator has a simulated cockpit for the user, which controls an armature with a camera attached that is piloted around a miniature battlefield environment and provides a video feed to the cockpit. A "foot" on the armature senses terrain elevation changes, which are then supplied as movement feedback to the simulated tank cockpit. All of this was controlled by a 1970s mainframe, for which replacement parts were unavailable so it was replaced, in the museum exhibit, with a Raspberry Pi.

It looks like hardware emulation was NOT used; rather, they translated the original program from paper printouts to a modern programming language: https://www.raspberrypi.com/news/simulate-driving-a-1970s-ta...

It's _very_ hard to believe some company to simply "clone the mainframe" into chip. Mind sharing any link to this effort?

Custom chips aren't as expensive as you think. About a few million dollars for the design and a hundred thousand chips, way out of reach for a hobbyist, but accessible to large enough companies. Just because it's opaque to us software people doesn't mean it's not a real industry you can buy things from.

And all that expensive engineering is still less costly AND faster (!!) than a new software certification.

Cannot refrain myself from asking why ...

Situation: Software-1 on Platform-1, both certified.

Time-evolution: P1 is deprecated, replaced by uncertified P2, but S1 remains certified.. just nowhere certified to run yet.

Solution: There's another software S2 which originally vouched for P1, itself still certified, which can still be used to certify P2.

Counterfactual?: If S1 were deprecated in favor of new S3.. there'd be no plan to certify it!

The problem: None of this actually makes any sense! But we're trying to fake due diligence. Everyone knows the hardware/platforms kinda need to be certified with respect to each other anyway, but if we did it that way it would all be even more expensive an time-consuming.

Doesn't seem crazy to me. Which one do you suppose needs more proof of correctness: a cake recipe, or the oven you bake the cake in? The recipe has to be correct or the cake won't work, but the oven just has to hold a temperature.

Ovens vary greatly. There is no consistency in how even the heat is in the space, the airflow through the space, how much and how fast the temperature varies around the set temperature, etc. These things all do affect how things bake up and often people have to tweak recipes for their specific oven.

In a case where something is safety critical the computer is super important as very subtle errors could cause catastrophic consequences. This is why you get into things like running software on 3+ computers concurrently depending on how fail-safe something has to be.

That "just" is quite the load-bearing seam ...

Particularly when the task here is "just" to measure some rod positions in a constrained 3D space ...

When Xerox established PARC, they asked the assembled scientists what computer they wanted. The majority view was a DEC PDP-10 KA10, with the BBN memory management unit that let it run Tenex (the ancestor of DEC TOPS-20). Xerox couldn't really buy a competitor's mainframe, so they built MAXC (maximum access computer), which was a complete emulation of the Tenex machines.

>MAXC (maximum access computer), which was a complete emulation of the Tenex machines.

Compuserve also had a software dependency on the PDP-10 and Decsystem 20, and when those machines were no longer available they bought a company making clones so they could continue to manufacture them for themselves. The company they bought, Microsolutions, was Mark Cuban's first startup.

making clones of mainframes (IBM's) had been a big area of intellectual property litigation, but also facing monopolization investigations, IBM had to allow them. They were referred to as "plug compatibles".

The clone hardware manufacturer was Systems Concepts. Microsolutions was a software play -- I think they had something to do with trying to port 1022/1032.

very apt that xerox parc was full of tenex engineers!

IBM released the PC XT/370[1], built around a pair of 68000s and an 8087 with custom microcode to emulate System/370 instructions, in 1983.

It's not hard to imagine a similar system fabricated around an open or licensed processor core today being well within the budget of a major airline today.

Software emulation would no doubt be less expensive, but there could have been regulatory reasons for building plug-compatible hardware.

[1] https://en.wikipedia.org/wiki/PC-based_IBM_mainframe-compati...

It could be an FPGA. Given enough gates, and the know-how to program them, you can make an FPGA emulate anything. Speed and efficiency could be better or worse, but if you're targeting old hardware, better is likely.

I mean...there's some precedent. The XKL TOAD ("Ten On A Desk") implements the PDP-10 instruction set, and will happily run TOPS-10/TOPS-20. LCM had one doing just that.

https://www.computerhistory.org/collections/catalog/10277383... https://www.twenex.org/?network

It's tempting to write a PDP-10 emulator for the Atari ST called TOAST.

FWIW I'm developing "tiny-titan", a stand-in for Cambridge's Titan mainframe that talks to Heinz Lemke's 1972 PIXIE program over an emulated version of Neil Wiseman's PDP-7 <=> Titan link. It doesn't emulate Titan itself, just the conversation PIXIE had with it, so the unmodified 1972 PDP-7 code dials home and uploads its drawings.

https://en.wikipedia.org/wiki/Titan_(1963_computer)

>Titan was the prototype of the Atlas 2 computer developed by Ferranti and the University of Cambridge Mathematical Laboratory in Cambridge, England. It was designed starting in 1963, and in operation from 1964 to 1973.

https://news.ycombinator.com/item?id=49407938

https://news.ycombinator.com/item?id=40617894

Flight of the PIXIE - Yuja Wang:

https://www.youtube.com/watch?v=jDrqR9XssJI

UNIVERSITY MATHEMATICAL LABORATORY, CAMBRIDGE: Cambridge Supervisor - Planning Document 10 Software for the Titan/PDP-7 link:

https://cucps.soc.srcf.net/titan/supplan/pd10.htm

Here is a photo of Heinz Lemke operating PIXIE with the light pen and 340 display on the PDP-7 (also seen in the video above):

https://www.facebook.com/groups/779220482206901/posts/629772...

Fascinating stuff; thank you for sharing. Can I follow your work somewhere??

Thanks! I'm in the process of bailing from Medium, and I'm moving my stuff to a reimplemention of HyperTIES, the hypermedia browser I developed for NeWS in 1987-1990 at the University of Maryland Human Computer Interaction Lab, and I'm dogfooding it to publish all my other stuff:

https://hyperties.org

Ostensibly the site is about HyperTIES itself, but I'm publishing other stuff there too like a PDP-7 emulator!

https://hyperties.org/databases/pixie/pixie-live/

Here's a part of the demo I was giving at EduCom in October 1988 at the Sun booth on the trade show floor, across from the NeXT booth, where Steve Jobs and Bill Joy came by. Because he had shaved his beard, I didn't realize it was Bill Joy when I was showing him a demo of his head popping up! He took it pretty well though, better than Jobs.

https://hyperties.org/databases/playground/sun-founders-big-...

Here's a paper looking back at HyperTIES:

https://hyperties.org/databases/hyperties/a-look-back/

XKL is still in business (https://xkl.com/)

as far as i know the control plane on their equipment is still a pdp-10 compatible

I'd happily run ITS on that.

Meh, old architectures are relatively simple and you can get 95% of those via fpga softcores (some freely available, some paid) and run them on fpga. The rest can be implemented by a proper ee team.

>Then their ancient Honeywell(?) mainframes...

https://en.wikipedia.org/wiki/GE-600_series from the 36-bit era which turned out to be a sweet-spot for Lisp; also "famous" for running Multics (old joke flipped: "which was many of whatever unix is one of")

GE sold it's division to Honeywell, and from there to Group Bull (French) and then to NEC (Japan)

Also immortalized as the name of the "gecos" field in the UNIX passwd(5) file, described in the v7 man page[1] as "GCOS job number, box number, optional GCOS user-id" (GCOS = GECOS after the Honeywell acquisition).

[1] https://man.freebsd.org/cgi/man.cgi?query=passwd&apropos=0&s...

Moltix would have been a better name than OpenClaw.

That's interesting. That means there are fewer checks for hardware than software?

Hardware is more amenable to static analysis than running programs which may receive an arbitrary number of inputs and express an arbitrary number of possible intermediate states.

I am interested in how firmware is treated, since perhaps in the case of these old machines it's small enough to be analyzable or at least cloned bit-for-bit.

Software "engineers" like to abstract things. This doesn't pay well e with reliability.

You have inspired a blog post. World, be warned!

Having worked on nuclear plants (as a reactor operator) around that time this doesn't surprise me in the least.

Thats far more advanced than the systems I worked with, one of which reported rod position via resistance measurement on a brushed cylinder (one for angular and one for depth).

Cleaning and calibrating those was a constant maintenance item every time the reactor was shut down.

This is a great post. How does someone write software that needs to run for ~50 years where the hardware will need to be replaced with non-equivalent, newer hardware? If I were facing this issue today, I might start with an OS that has excellent emulation. Example: Can I run 32-bit MS Windows 95 via emulation on a variety of current 64-bit OSes, like MS Windows, Linux, AIX, HP-UX, etc. If yes, then we can assume(?) this emulation will remain relatively stable even if we upgrade our hardware later. Maybe I am overthinking the whole problem: Can VMs do exactly what I want today? Will VMs running ancient OSes, such as 32-bit MS Windows 95, continue to be stable/viable in the future? I am unsure.

Make sure that you have as thorough a specification of what the system is supposed to do as you can.

Then define the version control system and the build process, specifying the dependencies, and so on.

Think about any opaque blobs in the system and try to eliminate them so that you have plain text source code so that no tools are needed to read the code.

Make sure that the build process runs entirely locally and never fetches anything from outside.

Simplify everything, use only tools and languages that are well understood and supported.

The real problems are not strictly technical but social: how do you prevent loss of the code, the tools, the specification, how do you maintain the expertise needed to maintain it. How do you ensure that all those things that are obvious to you now are written down in all their gory detail so that your great grandchildren will not apply their new and different preconceived ideas to the system?

Document all this on paper as well as electronic storage, make sure that version ids are recorded on every page as well as being available to the user of the machine or program.

In the industry in which I worked for the last thirty years of my career it was not uncommon to have things come back for repair after fifty years use and to be able to consult the original drawings and bill of materials so that exact replacement parts could be made.

What is this industry, if I may ask?

Electricity distribution, mainly transformers from 10 kVA to 500 MVA and more.

It takes a long time to design and a lot of material to make a big transformer and they last for decades. They are critical infrastructure so when one fails you want it replaced or repaired quickly but they are too expensive (millions of dollars/pounds/euros) to keep a spare unit idle for decades and the lead time for a new large power transformer might be six months or even longer. So repair makes sense, especially if it can be done on site because moving a big metal box full of oil, steel, and copper that weighs hundreds of tonnes is a major logistical exercise.

This isn't something that can be done by a company that 'moves fast and breaks things' and disappears next year.

Hat tip for the first hand details. I sound like a broken record at this point: Posts like make are the secret sauce of HN. It is the main reason why I come back time and time again.

There must be some extremely salty machinists that get bizarre part requests after breakage for some transformer from 1950. That could make a very cool YouTube video series: Following these machinists and repair people as their repair very old transformers.

> How does someone write software that needs to run for ~50 years where the hardware will need to be replaced with non-equivalent, newer hardware?

Some ideas:

Write it in a popular language/ecosystem, stick rigidly to well-defined APIs, use commodity hardware, flatten out any malignant cleverness, maintain documentation on why every part does the thing it does, and make the source code readily available.

This is based on working with some very old systems, and each point above is the opposite of something that made life harder.

Unfortunately, “popular language/ecosystem” includes JavaScript and PHP.

I’d go for “universal and timeless”, which has meant C for several decades now. There is, I’d concede, a reasonable chance that Rust obtains that label in our lifetimes.

Yeah agreed. Popular is a good first approximation, but to be more precise I think you want tools where the incentives in the community reward stability over features that require breakage, which would include languages and runtimes like Erlang and Clojure.

Neither is popular when compared to for example Python or PHP, but they both have cultures that care about stability in the core runtime and the libraries. Most folks probably don’t know that about Erlang and maybe think it’s some new whiz-bang thing, but Erlang is incredibly stable and well supported by a dedicated team that has been doing it for years and years at this point. I don’t know if Erlang will be more or less popular in 20 years but it 100% will be around and be receiving updates, I would put money on it.

My C programs from a few decades ago, before namespaces were invented, no longer compile. However, the old executables still execute. The old C compilers are available but not supported; ChatGPT warns to expect trouble. Perhaps maybe there are compatibility switches that work correctly. Using them would require changing the makefiles.

And worse for posterity, Python

The Amiga was a popular ecosystem.

And as a result it's very well emulated, making it very likely that Amiga code can still be run in 50 years' time.

Today. That was not true a few years ago.

I remember Amiga emulators (notice the plural) going all the way back to when i was in high school - and that was around 1999-2001 or so.

That was about the time frame of the commercial Windows NT emulator mentioned up thread. There was a market for them.

Today’s landscape is shaped by the pandemic. In between, there were many years of impracticality.

Perl was popular

Wasn't there some idea that went like "if something has been around for x years, it will likely be around for x more years"?

Maybe follow that. It's 2026 right now. What way of writing code would have worked 50 years ago (1976) that still works today?

Well, C, SQL and Lisp were all around 50 years ago and still exist today.

In terms of hardware, 50 years ago there was intel 8080. apparently code for that can still work via emulation today.

So if you wrote C for low level, Lisp for high level, and compiled the software to run in an Intel 8080 emulator, that combo would likely still work in 50 years.

C is very different C. There is C where programmer rushes to call fork(), just because he likes the idea of entering room once and exiting room twice. And this program has got excuse for his behavior. He has fork() in POSIX standard. So programming is using fork() here and there even if not strictly needed. Using fork()/exec() instead of more portable posix_spawn(). Using fork()/accept() instead of multithreading.

Then it becomes impossible to introduce database connection pool. Well, probably possible, but in single-process multithread server this is BSc grade job, and in multi-process server this is PhD grade job. libmysqlserver and others maintain some context near to the socket, in opaque way. Hard to pass context between processes. At least, hard to return used connection back to pool in main process. Proxy is also not an easy walk.

On client side although Windows NT had POSIX layer, and XP/2003 still had Interix SFU. But Windows Vista broke SFU. But then Vista has got SUA. But SUA was not binary compatible, required recompilation. And as of Windows 10 there was surely no SUA. Was it dropped in Windows 8? Windows 10 has got WSL eventually, but there was a gap between SUA and WSL. And deploying Windows application with SFU, SUA or WSL part is not easy walk. Writing installer is MSc grade job.

Then comes iPhone OS and Android. They are POSIX OSes. Kind of. But spawning external processes is prohibited or not desirable. This is what was blocking LaTeX adoption on mobiles.

Well I think if you were doing the 1976 way you might not even have multi processing or multi threading. Just a single process with a single thread. And so you don't have a database connection pool, you just have a single database connection and do one query at a time maybe.

That may not work for network connected software with lots of requests per second. But such software did not exist too much in 1976 either as there was not very much networking going on.

There were absolutely multiuser computer systems processing transactions from thousands of users in 1976.

As for networking, both the ARPANET and Ethernet predate 1976, though neither were ubiqutitous and most high-volume networking would have been mainframes and related hardware connected via leased lines.

Lindy effect

run for ~50 years where the hardware will need to be replaced

If it needs to run for fifty years, specify hardware that has a long support cycle such as Mil-Spec.

That’s why the Z80 was around for so long and why so many companies in the 80’s and 90’s still deigned around it. (1)

If you let programmers drive system design, winding up with an “Amiga” in your certified design is a more probable outcome…

(1) there’s a moderate chance that new hardware that can run Z80 code will be around fifty years from now.

> how does someone write software that needs to run for ~50 years where the hardware will need to be replaced with non-equivalent, newer hardware?

Make it open-source, or don't buy without source.

It's interesting to think about this in the context of parent poster's story about the Amiga emulator in the nuclear power plant.

Or, more broadly, in the context of any situation where the software is closely coupled to a specific operating system and/or hardware.

Having the source code would have helped tremendously, but might not have been sufficient because of the hardware story and the regulatory approval factor.

You don't need source anymore. Have Claude Code look at the binary and write a new program.

I wouldn't rely on that for mission critical systems.

Even if VMs do not, whole-system emulators surely will.

But Windows 95 would be a poor choice, as you'd be stuck emulating a bunch of very specific aspects of both Windows (non-Unicode APIs, 16-bit application support, etc.) and the PC archictecture (BIOS, A20 gate, etc.) that exist only to maintain backwards compatibility.

If Windows is a requirement, you'd be better off building against something like the subset of Win32 APIs that has been stable since Windows Server 2003 and ideally testing against Wine, contributing patches as necessary.

If it were me, though, and it was something that required a general-purpose, non-realtime OS, and wasn't so performance-sensitive as to require deep integration with platform-specific APIs, my first thought is to target a Linux-compatible subset of FreeBSD, which should be enough to get you most of POSIX and then some, plus a full-featured GUI if necessary, portable between two open-source OSes that run on a wide variety of hardware.

I'd recommend not having it so dependent on the hardware and choose an OS that is relatively hardware agnostic (e.g. Linux). Also, ensure that the software is open source and if compiled, has an open source toolchain to do so.

Though people often decry it (due to the many, many footguns), writing something in Shell/BASH will make it trivial to move to newer machines.

DOS emulation is available more widely than Win32 emulation, and there is HX DOS Extender. I would try sticking to Win32 dlls runnable in HX DOS Extender. rundll32 or some other exe host. HX DOS Extender has no paging support and has problem loading multiple exe having no relocation info to the very same 4Mb base address they all usually pretend. DLL should have relocation.

I thought "but what is the problem to generate EXE with relocation?" and tried GCC GNAT (from 2017), and GCC said no. EXE and DLL formats are very much the same, but GCC is completely unable to generate relocatable EXE. A very suprising discovery. More recent toolkits became aware that adding relocation to EXE is not a rocket science, just do the same like in DLL. But these recent toolkits may call APIs that HX DOS Extender does not support yet.

> How does someone write software that needs to run for ~50 years

You in a lot of ways have to respect the ability to have done this in the first place, compared to todays slopware which manages to break within 1-2 mandatory OS update cycles.

This was one of the promises of Java. You write your application and then you get a virtual machine (JVM). As time goes on you only maintain the virtual machine while the application remains the same.

The funny thing happened and verified this promise. There were several architects working on IBM System Object Model (SOM). In 1996-1997 many good things ended, and SOM was terminated. But the work was so good that two architects did not stop and wrote the book: Putting Metaclasses to Work (1998). That book has got reference implementation in Java. So we have IBM SOM as closed source binary and we have Java source codes.

IBM SOM executes on Windows 10. I have checked Visual Age for C++ Direct2SOM extensions, OpenDoc for Windows. Very good. 32-bit i386, but otherwise runnable, usable.

And Java is hard to build, hard to execute on modern Java toolkit. Java seemingly had no List, and authors made their own List, but when Java also got its List, that breaks the build. When building problems are resolved, runtime problems come. Original Java was seemingly fine with Property key being object, but modern Java only wants strings. And Property-based dictionaries are all around. PMtW is about building VMT, and there is multiple class inheritance in PMtW model, not matching Java's comparably limited OOP. So PMtW is building dictionaries for methods to support what is described in book, and foundational Java class is malfunctioning.

I did not finish this road, don't know so what does it take after all to run Java code from 1998 on modern Java toolkit. So far looks pathetic compared to Win32 programming.

    > As time goes on you only maintain the virtual machine while the application remains the same.

"only" is doing some heavy lifting here. A modern VM is probably more complex than a kernel.

Write it in C or Fortran.

50 years ago, fortran iv had the "extended range of a do loop". You could jump out of a loop execute some code, then jump back in. Therefore 2 loops could execute the same out-of-loop code. It was useful; preprocessors like Flecs used it to emulate missing control structures.

ChatGPT says, "likely to need manual conversion". So no Fortran hasn't been stable. In another post I showed that C hasn't either.

Use Perl

>So this is how, in 2007, during a tour of the facility, I stumbled across a Pentium 1 system running an AmigaOS emulator on Windows NT 4.0 that was responsible for displaying the status of the control rods of a nuclear power plant.

Vernor Vinge's A Deepness in the Sky depicts a human society thousands of years in the future, in which pretty much all software has already been written; it's just a matter of finding it. So programmer-archaeologists search archives and run code on emulators in emulators in emulators as far back as needed. <https://web.archive.org/web/20231114211656/http://www.gareth...>

(Heck, recently I migrated a VM to its third hypervisor. It began as a physical machine a quarter century ago.)

I grew up in a little post soviet republic, in a small town with a big nuclear power plant.

Coincidently, that's where my father was working as a programmer until the shutdown of the plant.

Anyways, he was maintaining a bunch of systems related to collecting data from environment monitoring sensors around the plant. In mid 2000s or so he showed me some data collecting machines using what looked like DOS text UI that mentioned being written in Leningrad, 1989 or 1988.

The software was running in an emulator on Linux, which my father used to replace original x86-compatible machines runninh dos.

Pretty sure thr whole thing was running for quite a while since my family moved elsewhere.

Did they not need NRC recertification when they moved from a physical machine running AmigaOS to an AmigaOS emulator?

That is an advanced question best asked of the folks in charge of the plant. At the time I was just trying to figure out how to explain to the IT auditors why there was no antivirus software on this piece of crap.

Could you skirt around it and just remark "there are no viruses for AmigaOS"

(Probably not literally true. But functionally true in the sense that they were likely transmitted via infected floppy disks, of which there'd be virtually none left in the wild in 2007)

Hello, my name is Lamer Exterminator, pleased to make your acquaintance.

https://en.wikipedia.org/wiki/Lamer_Exterminator

It wasn't the AmigaOS running in the emulator that they were worried about, it was the Windows system running the emulator. (Not that the average IT auditor there could have understood the difference.)

How would a virus get in there?

If it were a newer version of NT I'd be concerned about USB media, but being NT 4.0 and not supporting USB it's imminently more capable of being air-gapped than later versions. (I recall a fun Ed Skoudis quote-- "At best, an air gap is a high-latency connection". Evidence Stuxnet.)

The auditor probably doesn't know nor care. "Every windows machine runs antivirus" is a checkbox item, zero thought involved.

Depends. A lot of these audits are quite prescriptive and don't leave much room for actually thinking about the problem. There is often some kind of mechanism for 'this is sufficiently segregated it doesn't matter that it's utterly out of date' but then usually some awkward rules develop that prevent some things from being put into that category. More subtly you can get whether the thing even exists as a thing that the audit cares about, and that often depends on the framing (embedded software is often invisible here but it needs to not look too much like a general-purpose OS even though it often is).

(Also, in my experience, what the auditors think the rules are and what is written down can often be divergent and even contradictory)

> Also, in my experience, what the auditors think the rules are and what is written down can often be divergent and even contradictory

Sounds interesting, could you elaborate?

This is pretty universal in my experience any time you have inspectors/auditors intended to enforce a set of rules. The most common case is that they just miss some rule: they can be hard to properly map from what you're seeing to the rules reliably, so different auditors will miss different things (so you can get a sign off from one and then another highlights an actual contradiction).

The next level is that the rules are kind of vague or unclear and so different auditors will have different interpretation of it. You can try to fight it but ultimately the auditor is who needs to sign off on it. This is also why switching auditors can be a right pain.

Then you get interpretations which are flat out contradicted by the rules. Sometimes this is necessary to make the rules work (there's some blanket rule that's impractical to impossible to apply literally, so the auditors contort some interpretation that's more workable), sometimes it's just incompetence, and when it's wrong enough to matter sometimes it's easier to just go along with it than try to escalate the issue with the auditor.

You can also get situations where the auditor has their own pet rules, which aren't written down anywhere, but they will become much more adversarial if they are not followed (sometimes they'll outright tell you what they are, sometimes it's a guessing game).

The Amiga thing is funny, but NT 4.0 was still receiving security updates until the end of 2006, so seeing it in 2007 wouldn't be much different than seeing Windows 10 today.

I'm curious what changed in Windows 2000 that would have broken the emulator, though, as direct hardware access in any version of Window NT requires a driver, and I don't recall any major breaking changes to the driver architecture between NT 4 and Vista.

This reminds me of the apocryphal story of the IBM System/360 running a 1410 emulator that ran an IBM 705 simulator that ran a business-critical application.

Not wholly apocryphal. The U.K.'s air traffic control ran, and possibly still is running, on an IBM 4381 substituting for an IBM 9020.

* https://news.ycombinator.com/item?id=49763972

They've reportedly already tried, and failed, to replace it with a more modern system.

Which is why we should target VMs directly. Then that VM can be brought forward to new platforms rather than emulating the entire machine, not that emulating the entire machine is bad. One could argue that the emulation stack you outline was made possible by putting in the work to make simple, fully specified and documented hardware.

NT4.0 in 2007? That wasn't even very late. I was replacing NT4 servers into the early 10s. Car dealerships were terrible for keeping that crap around.

Interestingly, there are a few places that will sell you a warrantied and new desktop or rackmount PC with a Windows NT 4.0 license. I imagine buyers of their systems are often in similar predicaments, not sure who else would buy these.

https://nixsys.com/products/nx370-t90

There are some things I don't understand about the OP's constraints (are you sure you can't run that Amiga VM on a newer version of Windows, or better yet, run that Amiga-based software on one of the currently supported Amiga VMs?). But they can probably pretty easily keep going as they're going with this software until some other, more significant thing in that plant becomes fully obsolete and the place is shut down.

"Windows NT 4.0 machine running as late as 2007"

lol. I know a ... factory, that had a BBC model B (with a rather complicated wiring loom) still doing a job around that time.

The IT supplier at the same factory went to a museum in Cambs. around late '90s, early '00s to ask if they could buy an exhibit because something had failed locally. The museum gave them the part.

That was just aerospace and nothing fancy like your nuke plant!

It is wild to me that you would use any GPOS for something like this instead of dedicated firmware for the task. This is evoking similar reactions to when I read articles about infrastructure being hacked remotely: Something must have taken a wrong turn with the architecture design for this to happen!

I find it deeply upsetting that anything important for a nuclear power plant is using an OS without protected memory.

You don't need protected memory if your program is the only thing running. Plenty of microcontrollers have no protected memory and get used in safety critical applications.

Protected against what?

Bit flips from cosmic radiation. Pretty common.

they probably mean https://en.wikipedia.org/wiki/Memory_protection actually

Just another program poking memory it’s not supposed to. Not even necessarily maliciously.

That AmigaDOS emulation will almost certainly never run another program. (-:

Far more concerning from a RISKS point of view is the spare parts being physically in the same place as the live machine.

> Just another program poking memory it’s not supposed to. Not even necessarily maliciously.

What other program? Most DOS versions were not multi-tasking beyond "terminate-and-stay-resident" (TSR) programs like mouse drivers.

Whatever application you ran on DOS was THE application your computer was running.

We're not talking about DOS here. This is AmigaOS, a fully preemptively multitasked OS from day one. No memory protection, but unless your code is running on bare metal like demos and games, there are other tasks running.

Top parent of this thread is talking about a package for AmigaOS being run on a bespoke emulator (as the Unusable Amiga Emulator of the 90s wasn’t really a thing that could be used for that)

If it's still running I wonder what they are going to do about the year 2038.

Set the date back to 1980 and sticky tape over the corner of the screen where the date is displayed.

I'm curious, how did the hardware present itself? It couldn't have been an Amiga Zorro card since it would've been impossible to get that into a PC. Did it connect over a serial or parallel port?

I recall it being some custom ISA card with connectors on the back. The emulation software had drivers that interfaced directly with the card.

Interesting! So they were able to port the hardware but not the software for whatever reason?

If it's what I suspect it might be, it was probably a generic IO card with maybe a bunch of 8255 PIOs on.

Was this status indicator simple vertical bars that were black and white and displayed on an old CRT Monitor? Almost looks like black and white terminal vertical bars (like Alsamixer)?

> A certain nuclear power plant had a Windows NT 4.0 machine running as late as 2007.

To put this in perspective, NT 4 was released in 1996, 11 years before 2007. That’s roughly the same as someone running Windows 10 today, which a lot of people still do. The Pentium 1 was only 14 years old at most. These parts of the system shouldn’t be surprising.

The AmigaOS and software from the 80s are of course the older more interesting bits, but for critical infrastructure and safety, shouldn’t we expect and design for our hardware and software to last a long time, and not try to keep up with tech fads every decade? I feel like the main problem isn’t hardware or software that’s old, but that there wasn’t a longevity plan.

It’s always been an interesting question how to write software that will last a hundred years. Maybe with AI this is the first time in history that planning to port & upgrade to new hardware every 5-10 years seems totally reasonable.

> That’s roughly the same as someone running Windows 10 today

While true the situation is different. Hardware and software moved a lot faster in the 90's and early 2000's. Everything has matured, is far more standardized, and moves more slowly now. Hardware from 11 years ago when Win10 was released isn't that different from today's hardware. Same goes for Win10 and Win11.

There might be a seed of truth there, but FWIW, I’m not very convinced by that. CPU speed development has slowed down a bit for sure, but software development is going faster than ever. GitHub and NPM and Python 3 and Rust all didn’t exist in 2007. CUDA was first released that year. LLMs weren’t being used. At the start of 2007, the latest Windows version was Windows 5, one more than Windows 4. Almost same as today. There isn’t a Windows 12 yet, while Windows 6 was released in 2007. So the pace of major Windows versions might be slightly slower. OTOH, the OS is much bigger than it used to be and it gets more frequent updates and point releases now than it did in 2007.

> At the start of 2007, the latest Windows version was Windows 5, one more than Windows 4

Windows 2000 (NT 5.0) was released in 1999. Vista was NT 6, and released in 2006.

Where do you find 2006? My googling says NT 6 was released in early 2007, so I meant that on Jan 1 2007, Win 5 was still current. https://simple.wikipedia.org/wiki/Windows_NT_6.x

It wouldn’t make a huge difference to me if 6 was released either 6 months earlier or 6 months later. In 2007, NT4 was only 1 major version behind for most people; adoption takes time. The broad timing of major versions being 4-5 years apart didn’t change until Windows 11, and we don’t know if that’s an outlier yet, but we do know that Microsoft has been changing the Windows release strategy, with more frequent updates. https://endoflife.date/windows

I wonder if there is push in the public sector for open source software/hardware for cases like this.

I completely get the decisions over time here, but it's unsettling having a relevant piece of software (reports are important too!) working on with parts from ebay, in 50 years time they might be gone.

I've dreamt of governments hiring devs with promises of full time OSS work. It _might_ get some people over the lower salaries of government work. But I overheard a state CIO once say "We configure, we don't create".

That's to say nothing about opinions of open source that rival early 2000s Microsoft.

For old ass critical software and hardware, the org can always have a backlog of hardware and contingency plans (paying someone to fix the hardware, paying someone to create a fully certified and modern solution..) for when they start running out of parts.

I wish there was some picts.

Nothing wrong with this.

If it ain't broke, don't 'fix' it.

Just make sure that its well isolated from the Internet!!

Up until a few years ago the Metrocard machines in New York City were NT4.

How on earth are these massive pieces of infrastructure relying on software they don't have the code to? i'm just utterly confused how these things happen.

This is crazy. Why on earth wouldn’t the respective government spend money to modernize a critical infrastructure like a nuclear power plant?

This is how we would end up with nuclear disasters, not because the technology is bad, but purely because of mismanagement and human negligence.

This is crazy. Why on earth wouldn’t the respective government spend money to modernize a critical infrastructure like a nuclear power plant?

Safety and reliability come from understanding the system. Something that's old but that you know everything about is far safer than something new. This risk is that the people who know move on, or the sources of replacement parts stop working, or that other things around the system change. Then the risk curve inverts and you find the old system more of a liability than a asset.

Almost always people choose to update things either too early or too late. Knowing when to do something is hard.

That’s pure negligence. It’s not like the people that have the know how disappear suddenly . It’s the responsibility of the management to make sure the knowledge transferred to new generation.

I guess that kind of thinking is what led to the collapse of ancient civilisations like why even bother to improve anything and let everything decay and die out

You can only say such things with hindsight. In the moment, nobody knows what the future will hold. Take the space program in the US for instance. In 1969 we landed a man on the Moon. We'd repeat that several times until 1972. At that point Nixon made a speech which implied various unpleasant things about space's future, but he'd be out of office, disgraced, a couple of years later, so how big a deal could it have been anyhow?

I think almost nobody from that era would have believed you if you told them that more than 50 years later not a single human would have traveled beyond low Earth orbit, and that we'd now be struggling to recreate what we did in 1969. Furthermore a significant chunk of the world doesn't even believe we landed on the Moon anymore, no doubt in part because of this apparent anachronism.

It's not like there was any sort of collective or even singular decision to just let things die out. I mean sure Nixon did his thing intentionally, but even as a President in the golden years of the US, he was still just one man.

> I think almost nobody from that era would have believed you if you told them that more than 50 years later not a single human would have traveled beyond low Earth orbit

Four people went around the moon earlier this year (leaving earth's orbit for the first time since '72)

Basically everything collapses as far as it is allowed to, and the collapse is slowed only by:

- profit, if a system generates it, people will keep the system running.

- dilligence, usually driven by personal idealism, and unrewarded.

- legal or financial penalties, or insurance costs.

- it has collapsed to a temporarily stable state for now.

For another comment I was looking at the Grenfell Tower fire in the UK around 2017 and the people who lived there had been raising fire risks for years and the landlord (management organization) and the council had been ignoring them, and ignoring the fire department. The companies which replaced the cladding on the tower with flammable cladding were choosing the cheaper flammable option and pointing fingers as well. And during the fire, the fire service had never dealt with such a big fire and didn't have a truck with a long ladder, didn't have extended-breathing gear, had radio problems, water pressure problems from the local water company (who deny that).

This kind of backstory is typical for disasters, and for IT outages - and I've taken to believing that if something "should work" but wasn't tested recently then you should expect that it doesn't work. This is a common saying in backups (you need to test restoring), but it seems to apply everywhere. Backup internet connections that don't have enough bandwidth for the company to keep running. Disaster recovery sites that share resources with the production site. 'Disaster recovery' that recovered into a remote site, but they couldn't do CAD work remotely over their slow connection so it was still disastrous. Multiple power feeds but they were cross-wired so one failing still functionall took down everything.

If knowledge transfer "should be happening" but it's not critical to a job, or it's not profitable, or it's not legally required and audited, then it isn't happening. Subject to the dilligent idealist employee mentioned above who are temporary in the long view. Management's involvement seems not to arrange the larger system to effectively shoulder responsibility, but rather find ways to avoid personal blame while cutting costs beyond the point where everything that needs to happen keeps happening.

It’s the responsibility of the management to make sure the knowledge transferred to new generation.

That's just lazy thinking, and probably a sign you've never been a manager. For all the responsibility to lie with management you'd have to be leading people who enthusiastically do what they're asked to do, raise problems, document things, follow processes, and make sure everything is handed over to the next person when they leave.

That's not how people are. They'll have times when they're unmotivated, underpaid, grumpy, over-worked so they miss things, etc. That's when the organisational tech debt starts piling up, and there's nothing a manager can do to stop it except manage it as best they can even though they're under the same pressure, with the same lack of motivation and crap pay as everyone else.

It's so easy to think you can fix it just by keeping on top of it and micromanaging where it starts to show up. It doesn't work that way.

> … underpaid … over-worked

These two (and many others) are well within management’s purview.

> … there's nothing a manager can do … [emphasis added]

Right, but Management (capitalized and plural) can. A manager should report, and management should fix - if not, it’s management’s fault.

The Management of a company is a group of managers. There's no real difference between the group and the collective.

The fact that you're willing to absolve an individual manager but still believe Management can fix the problems is a big part of the problem. The Management usually includes the last person in the chain who has the ultimate decision making power. If you assume that when I said 'a manager' I meant that person, it should be fairly obvious that even they don't have unlimited budget or unlimited time, and they can only try to fix problems they hear about. Within those constraints Management are still going to find fixing systemic problems hard.

It's very easy to look at something with hindsight and think the outcome should have been predicted and corrected. If you look at the real world you'll see that just isn't the case. Personally, I wish we had a lot more people applying some systems thinking approaches to at least consider the second- and third-order impacts of their decisions but I imagine that won't ever happen. It makes me sad because it's actually really interesting.

> This is how we would end up with nuclear disasters, not because the technology is bad, but purely because of mismanagement and human negligence.

I'm pretty sure constant upgrading would be more likely to cause disasters.

I can only imagine how much less reliable a modern JavaScript Electron app would be in this role...

Lindy's law:

For non-perishable things - i.e technologies, ideas, books, institutions - the expected remaining lifespan is roughly proportional to how long they have already survived

Time acts as a filter: what has already withstood a long stretch of disorder is more robust, so the longer it lasts, the longer it is expected to last.

The old nuclear software running on mainframes have passed the test of time. The new software, has yet to.

Nassim Taleb talks about this at length in his book, Antifragile.

If it's not broken, don't fix it.

Same reason why MTA in NYC still operates subway using 100 y/o signaling hardware on most of the lines.

In a way, your comment and the responses you get is a perfect allegory to inexperienced vs experienced engineering.

Their 100 y/o hardware is built from stuff that's still produced though. We still make relays (not identical ones but ones that perform the same function), wires, and mechanical levers. The system's behavior is not encoded in the relays but in the connections between them. If you had a CPU made of separate transistors you could replace burned out transistors with any future transistors but if you had one made of chips you'd need those exact chips that were out of production.

Replacing the relays in old telephone switches is a problem because they use so many weirdly specific types of relays - but those systems are only found in museums now. I believe track control uses fairly ordinary DPDT, etc., relay designs.

But similar reasoning is why Canadian trams are worse than other countries, and it's not "good experienced engineering". In this nearly hour long rant by NotJustBikes[1] the section I've linked is about the technology and how the transit authority will not move on from 100 year old track technology which makes everything worse for customers. There are junctions where tram drivers have to stop and look at the track to make sure the junction is correctly set because the single-point switching is less reliable than modern designs. Or the driver has to stop the tram and get out to use a stick to move the track and change the junction because they aren't electrified. And then pass through junctions as slowly as 10km/h because the old track design has higher risk of derailment than modern designs.

The designs for the electrified track switches were made by one company which was sold, re-sold, and then burned down and the designs lost, and there isn't anywhere else to buy them because nowhere else uses such old track design anymore, so not only is Toronto struggling to buy parts to continue electrifying the manual junctions, it's struggling to custom-make enough replacement parts to keep the aging electric switches working faster than they fail.

And they were using trolley poles to connect to the overhead power lines instead of Pantographs until 2017, 50-100 years after everywhere else, which meant a) they couldn't get enough power through them to run air conditioning, and b) they are more likely to fall off the power line and the tram stops and the driver has to get out and reconnect them, and c) their newer trams now have to have custom dual-power connections to go through older parts of the network, and d) the carbon shoes on the trolley poles wear out quicker which means the trams are dirtier and taken out of service for maintenance more often in the wet which is when people want to use trams more.

[1] https://youtu.be/HhQxNHrD6fA?t=2311

OK, but the context here is IT in mission critical infrastructure, like energy or banking. These industries will not jump on the latest technology and will drag their feet for as long as possible, and for good reasons.

Also, I happen to be familiar with NotJustBikes, watch regularly. I don't see this as the same kind of problem. What you explained is clearly BROKE. What we're talking about works just fine and migration to newer technology doesn't provide advantages outside easier maintenance. It can't be justifies by costs, either.

A friend discovered that one of the more modern pieces of equipment instrumenting those old signals was an ancient WYSE terminal from the late 90s that had been bricked over during a station project.

They moved a booth or something and the compartment it was stuffed in was missed during the cleanout. The thing was working for at least 20 years and may still be there!

I worked with a client that had a cash disbursement system working on an ancient dBase 16-bit application. They had a very difficult to change process that required multiple transactions because the system couldn't represent the dollar amounts appropriately.

> If it's not broken, don't fix it.

I understand that way of thinking for a small business computer system. Not for a critical infrastructure. Also if people actually thought about like that we would still be in stone age, like why invent something new or use a new technology? Keep on using rocks to crack nuts and kill animals

I think you can imagine the difference between “I did some work and made this thing that is clearly better, but we might have a few kinks to work out” and “I did some work, and if I did it perfectly, it’s just as good as the old thing, but we might have a few kinks to work out.”

I agree with you. First, "broken" is not a binary, it might not be broken but it might not work well either. Second, sooner or later everything breaks, and you better be ready for when that happens.

> Why on earth wouldn’t the respective government spend money to modernize a critical infrastructure like a nuclear power plant?

Same reason they don't modernize health care infrastructure. It's too expensive, people die if they get it wrong and they get voted out of office for their troubles.

Because what they have works well enough to be NRC-certified, and they have enough spare parts to comfortably outlast the rated lifetime of the reactor.

Relevant: https://www.joelonsoftware.com/2000/04/06/things-you-should-...

There is no reason a more modern system would be better in any aspect.

Now you know why "nuclear power may have been dangerous in the past, but with modern technology it's completely safe and Chernobyl will never happen again" is not a good argument.

Chernobyl didn’t happen because of technological blunder or even human negligence. It happened because poverty caused by communism in Soviet Union had to use old outdated (not the modern nuclear technology at that time) graphite tips in carbon rods because it was cheap and also kept it as a secret from the facility operators giving them a false sense of trust on the kill switch.

Chernobyl is the perfect example to not use outdated old technology. Had they used the modern western technology at that time to upgrade their nuclear reactors the kill switch would have prevented the whole disaster instead of increasing the neutrons make the core a nuclear bomb.

Next time educate yourself with facts before making a fool out of yourself

> old outdated

Wikipedia's pages on Chernobyl say that construction started in 1972 and the problem with the control rods causing an initial spike of power was discovered in 1983 ?

[1] https://en.wikipedia.org/wiki/Chernobyl_Nuclear_Power_Plant

[2] https://en.wikipedia.org/wiki/Chernobyl_disaster#Accident_se... (search 'Ignalina nuclear power plant')

Chernobyl was very much not "old", it was a more modern design than the PWRs which are built today. Chernobyl Unit 4 was also basically brand new, only 3 years old at the time of the disaster. The RBMK design dates from the 60s, whereas the PWR is from the 40s.

"Had they used the modern western technology" ... "Next time educate yourself with facts before making a fool out of yourself"

But you are aware that this thread is about a reactor in the west, that "updated" their certified hardware to a VM instead running on windows NT?

Everyone knows that all technology newer than Windows 2.0 will never be obsolete!

Frankly, if not for the issue of components aging I would trust 1970s PDP-11 or even a PC-XT running MS-DOS a lot more than a modern system. This vintage's hardware was extremely mature and simple to understand and operate. No Ghz buses, no multiple levels of caching, no speculative execution and long pipelines.

DOS was a single task operating system. Once you proved such a system could run for days, months and years, there's no reason for it not to function 50 years later other than component failure.

That must have been a crazy moment for you when you first saw it and a big sigh moment for whomever had to explain it to you the first time. Thanks for sharing this; it made me smile!

Reply on news.ycombinator.com