Hacker News Reader

Comment by numpad0 | Hacker News Reader

How are distributed realtime systems like a rocket is synchronized? Are there central timeservers and allocated transmission slots and timestamped query-response action messaging or is it just small individual realtime nodes making their own decisions?

I’ve been wondering about this for a year or so because I can’t seem to find one that matches my description, other than concept drawings on NIST 4D/RCS paper, but there got to be one in the world.

View on HN

There's typically a central time source sending a 1Hz pulse into the main computer(s). The sensors will use something like Time Triggered Ethernet, RS-485, or a MIL-1553 bus.

Yeah this is in line with my experience. However there are concerns about the accuracy of quartz clocks as temperatures fluctuate, this poses a problem for some orbital patterns.

Thanks to both answers - I've asked couple engineers of my generation and "that would be an interesting concept" was the best response I got. I have so much to learn!

For LEO missions, the source is now typically a GNSS receiver outputting a PPS line. Otherwise, you try to have a precise clock (TCXO, OCXO, and now CSAC are available) as a reference distributing the time to all the subsystems, and you correct the satellite time offset from the ground (similar to NTP).

A popular solution is PTP [0]. Once you have synchronized hw clocks you lay your software synchronization on top of that.

[0] https://en.m.wikipedia.org/wiki/Precision_Time_Protocol

Yes. This does require all of the network devices between the nodes to also implement PTP (otherwise you fall back to NTP-like behavior) but this isn't a problem if you're greenfielding the implementation.

fprime is what you're after; it's the embedded flight hardware that runs on the Mars 2020 drone.

https://nasa.github.io/fprime/Architecture/FPrimeArchitectur...

fprime is more of a set of libraries and daemon architectures, not really hardware time synchronization.

iirc all flight systems are in triplicate and all data is checked against the others, then a quarum is needed from the 3 to make a decision.

other rockets use rad hardened computers that make decisions on their own, but for cost and time, I believe SpaceX just triplicated all necessary systems and votes out and reboots any system that gets bit flips.

They actually use more than three now. The basic architecture is that computers are organized in "strings". Each string is doubled. If the two don't agree, the result is chucked, and the system reboots. Each of these "strings" then participates in a quorum. The result is extreme reliability, with relatively fast convergence.

Fascinating. Could you write more about this, or do you have suggestions for what to read to learn more?

Functional safety standards like ISO 61508. But I am not sure if you will like it.

Reply on news.ycombinator.com