Hacker News Reader

Comment by swamp40 | Hacker News Reader

I thought you needed three, and you go with whichever two match?

No, it's not finding quorum, it's checking for bit flips generally. You just need dual redundant CPU's in lockstep.

"The FFB is at the heart of the Ingenuity helicopter’s avionics package. Two TI TMS570 Hercules microcontroller SoCs, originally aimed at automotive applications, operate as Ingenuity’s low-level flight controller (FC). Each TI microcontroller incorporates a dual-core, ARM Cortex-R5F processor operating in lockstep. The FFB also incorporates ECC-protected Flash memory and RAM. The dual-redundant lockstep processors and ECC memory provide some protection against radiation-induced soft errors such as SEUs (single event upsets)." https://www.eejournal.com/article/an-fpga-flies-on-mars/

I still don't understand how you can check for bit-flips without a quorum? What do you do when one CPU says there was a bit flip and the other one doesn't?

> What do you do when one CPU says there was a bit flip and the other one doesn't?

Reset and erase processor internal state that was corrupted. Startup times are typically one of the metrics controlled for in these systems.

Indeed, Ingenuity can do it in under 10ms iirc, which is highly imperceptible (to us) and it can do it mid-flight too.

Dual-redundant gets you error detection without correction whereas triple-redundant with majority voting can get you error detection with correction in the case that one fails. The error detection with dual-redundancy only works if one fails. If both fail and both fail in the same way you won't be able to detect it, but you basically accept the risk that it's an improbable event. The "correction" in the dual-redundant case is typically reset into some known state rather than trying to correct and continue operation.

To be clear, this setup is different than just 2 cpu cores in lockstep, with reset in case a fault is detected. It's also different from triple redundancy/voting where there is no reset, but continuous operation.

This is two cores times two, managed by an FPGA. If one SOC (one pair of cores running in lockstep) detects a fault, the flight controller is hot-swapped to the other SOC (also one pair of cores running in lockstep) while the first SOC is rebooted.

I also thought this. If you've just got two, how would you know which of them is the correct path?

For some applications, not necessary. You can simply detect the error, abort the operation, and retry it from the beginning. If the difference was due to a transient bit flip (due to radiation/etc), odds are high it will work the next time. Obviously that costs some recovery time, but if the application isn't overly time-critical, you can live with it. You also need to make sure your code can handle being interrupted and restarted at any point, even if it was half-way through mutating some data structure.

Reply on news.ycombinator.com