Hacker News Reader

Comment by qwertox | Hacker News Reader

> Fortunately, JPL knows exactly what went wrong.

> and while the flight uncovered a timing vulnerability that will now have to be addressed, it also confirmed the robustness of the system in multiple ways.

This is so important. Imagine if it would have crashed and damaged communications, that would have been the end of it. This extra step in engineering effort really payed off and gave Ingenuity a second life, and NASA a really good opportunity to learn important lessons and do further debugging.

Also, the article at mars.nasa.gov is a delight to read, with all the information one would like to know being in there. Not to mention that there is even a video for us to see.

View on HN

Note that this robustness seems to refer to just the physical build, not that the code had a glitch yet somehow saved the day.

> Flight Six ended with Ingenuity safely on the ground because a number of subsystems – the rotor system, the actuators, and the power system – responded to increased demands to keep the helicopter flying. In a very real sense, Ingenuity muscled through the situation, and while the flight uncovered a timing vulnerability that will now have to be addressed, it also confirmed the robustness of the system in multiple ways.

The code being robust and self-correcting, rather than the increased demands coincidentally being within tolerance levels, would have been more interesting or laudable.

Afaict, JPL and NASA (and subcontractors) have a very strong historical knowledge base of building maximally-robust systems with limited redundancy hardware (in the sense of not having an unlimited mass budget for more hardware).

But then again, they've been working with semi-autonomous systems about as long as anybody.

https://newatlas.com/apollo-11-guidance-computer/59766/

The fact that the code disables optical nav at low altitudes is also one of the reasons it landed safely.

Yes, but in this case that was just good luck. As the article mentions, the low-altitude behavior is because of concerns regarding dust. It was not intended to provide robustness in the face of glitches in the optical navigation system.

Dust concerns were one reason, but avoiding glitches and ensuring the smoothness of position data is another.

> Then, once the vehicle estimates that the legs are within 1 meter of the ground, the algorithms stop using the navigation camera and altimeter for estimation, relying on the IMU in the same way as on takeoff. As with takeoff, this avoids dust obscuration, but it also serves another purpose -- by relying only on the IMU, we expect to have a very smooth and continuous estimate of our vertical velocity, which is important in order to avoid detecting touchdown prematurely.

https://mars.nasa.gov/technology/helicopter/status/298/what-...

I wonder if, had Ingenuity disabled the visual system upon noticing timestamp conflicts, the use of the IMU alone would have made the entire situation a more safe one.

> Note that this robustness seems to refer to just the physical build, not that the code had a glitch yet somehow saved the day.

I understnad the missing video frame is not just physical build issue. This is also a video of video pipeline and speed estimation implementation.

would have been more interesting or laudable.

I think it's interesting and laudable that the hardware coped. But yeah, on the face of it this sounds very much like brittle software. For 46 seconds the craft's performance suffered from a single frame being dropped. It sounds like either the calculations were smeared over way too much time, or (more likely) the code made too many assumptions that relied on every piece of the pipeline properly behaving, not only in the moment but also over the entire history of the flight.

I'm speaking from extreme ignorance obviously. But this reminds me of a million code reviews I've done where I've asked developers to make fewer assumptions about the state of the system. Often the response is something like "but how could that ever happen?" And my response is always "i have no idea, but shit happens."

I would love to see a postmortem that discussed the specifics of what went wrong in the software, and whether they can attribute the lack of robustness to system design flaws.

> This glitch caused a single image to be lost, but more importantly, it resulted in all later navigation images being delivered with inaccurate timestamps.

It was not a single lost frame which caused this issue. The issue was that the glitch then corrupted the timestamp accuracy of the frames that followed. I guess the dropped frame was just a symptom, maybe an initial memory corruption or something like that.

But it was not the fact that the frame dropped and that this missing piece of information then had the negative effects.

> The code being robust and self-correcting, rather than the increased demands coincidentally being within tolerance levels, would have been more interesting or laudable.

This is the good thing about the overall outcome of this incident: They now have a chance to learn from this.

It would be interesting to know from NASA if, had Ingenuity crashed and lost all COMMS, they would still know what was the cause of the issue. I mean the wrong timestamps of the images, not what caused them.

It would also be interesting to know what caused them, if it was a bug in the software, or a particle flipping a bit in the timestamping-code.

I had the same response while reading it, too. Like, wow, this is telling me all the technical things I wanted to know about the situation. This is one reason I pay for IEEE, they do a great job relaying the facts in summary articles from the main source.

For those on mobile: https://mars.nasa.gov/technology/helicopter/status/305/survi...

Reply on news.ycombinator.com