A fellow new hire and I were tasked with fixing a machine that ran in a clean room where semiconductor wafers were made. On weekends while the line was down, we would go in, crank up one section of the line and let waste wafers travel through the line where very rarely they might get stuck in a multi-lane machine that would etch the wafers with some sort of acid.
The machine had over two dozen asynchronous motors, actuators, pumps, sensors and so forth. All generating interrupts and I/O events that were sent to a computer that ran the whole line and controlled all the machines.
We couldn’t slow down the machine, it had to run at full speed. The program controlling the machine was thousands of lines of assembly language—everything was assembly language, including the homemade OS that ran the computer running the line. It took like an hour for us to bring up the line and two more hours to see the machine do something strange.
The computer running all this had no user interface other than a some front panel switches and some panel lights that would reveal 16 bits of it’s 128K of memory at a time. This was in the 1970s before Ethernet had been invented.
It felt a bit like those escape room events where you know there is a solution, but you don’t know if you will ever get out. Without my coworker cracking jokes about our plight, I’m not sure we would have ever triumphed over that stupid machine.