Computing glitch may have doomed Mars lander | Hacker News Reader

Computing glitch may have doomed Mars lander

nature.com

85 pointsokket121 comments

>>>...Then thrusters, designed to decelerate the craft for 30 seconds until it was metres off the ground, engaged for only around 3 seconds before they were commanded to switch off, because the lander's computer thought it was on the ground...

>>...The lander even switched on its suite of instruments, ready to record Mars’s weather and electrical field...

It's a shame. But, reading that, I can't help imagining the lander in the role of the Blue Whale from Hitchhiker's Guide to the Galaxy:

"I wonder what this big flat thing rushing toward me is? I think I'll call it 'The Ground'"

Given the history of ESA landers, the more appropriate quote might be "Oh no, not again!"

It really puts into perspective what good job NASA does at what many call a "boring" mission.

Considering NASA has a budget thats 3-4x that of ESA, I'd be a little less quick to judge if I were you.

> Considering NASA has a budget thats 3-4x that of ESA, I'd be a little less quick to judge if I were you.

I don't think GP was judging ESA. I think they were merely pointing out how impressive NASA's success rate has been lately.

Truth, its especially commendable that most of their projects keep soldiering on well past EOL.

Not to take away from that but I suspect that there are a lot of good reasons to underpromise and overdeliver. I also wonder to what degree budgeting has something to do with it. Budget for a shorter lifetime given that you can usually end up extending missions if the equipment is still operational.

Not to take away from the great work of NASA engineers but EOL is kind of a misleading term when it comes to space science missions. At lot of mission end times are based off of funding and the lowest level science goals of the mission. While those are being completed they gather a case for continuing the mission and usually get it. Of course the main cost is getting the stuff there in the first place so why not add a relatively small amount to make it last?

No, but given the short lifespan of these things due to radiation, literally other-wordly conditions, and the fact that you can never repair it, the fact that a rover on Mars can last a decade is a technical marvel.

You also have to pre-program the entire flight and find out half an hour later if your parachutes and retro-rockets deployed, or you made a relatively small crater.

Fair enough. How about the UK's Open University that was marginally more successful with Beagle 2 for £66 million.

It went riding on another orbiter, Mars Express, but at least reached the surface intact.

They're spending 1.4B on the two ExoMars missions (and that's not counting the Proton rockets, which Russia's contributing).

NASA's budget is larger, but they're doing more missions (and spending a lot on SLS and James Webb). I suspect blaming the ExoMars crash on budget is inaccurate.

It seems like ExoMars spent years shopping around between the US and Russia for launch, and major budget problems, which caused it to be redesigned more than once. I guess NASA doesn't usually have problems like that? https://en.wikipedia.org/wiki/ExoMars#History

NASA's Mars Polar Impacter, er, Lander, also shut off it's descent engine too early, believing it was on the ground, and slammed into Mars. This stuff is hard. The Wikipedia entry gives the details:

https://en.wikipedia.org/wiki/Mars_Polar_Lander#Landing_atte...

They don't have altimeter's/range finders?

I wonder what causes the confusion.

They do have radar altimeters, but radar altimeters face occasional issues. So they usually also bake in various failsafe measures. Sadly the failsafe measures tend to be the thing which dooms spacecraft, usually in odd ways unpredictable until an actual spacecraft is hurtling towards the ground.

That is too bad. I wonder if a laser in the front would be useless as the heat build up as it enters the atmosphere would make the "bounce-back" hard to receive.

I think a space program that has lost a Mars-bound spacecraft because different engineering teams could not decide on if they are to use metric or imperial units of measurement has nothing to gloat about.

It's not gloating, it's just pointing out that ESA has failed repeatedly where NASA has had repeated success. No need to take it personally.

At least it wasn't a unit-conversion problem. It would be really shameful for an _European_ lander to mix up an altitude reading between feet and meters.

But it's not the case, isn't it?

It may very well be.

Maybe they used an US-developed microcontroller library and screwed up the unit setting.

How realistic is a dry run via an earth simulation from high aptitude to make sure units are sane?

Given that Mar's atmosphere masses about 25 terratonnes compared to Earth's 5148 terratonnes, and Mar's atmospheric pressure is about 1% that of Earth's sea level, you'd need to do high altitude testing at about 30 miles up, which is about 5 Mount Everests.

Much simpler to test fire the engines in a lab somewhere to make sure they run, and test the software in a simulation of the landing. It'll probably come out in the next year or so that there was either just a flat error in the code, or that the simulation was incorrect in some way.

Surely in 2016 no one is using imperial units as a default for anything even remotely related to aerospace programming.

I work at NASA and sadly, we do still use Imperial units for many things. It's a travesty.

In military aviation mission planning, it's even worse. We have to support any damn-fool unit some random major might want to see on the printed reports. A length might need to display in feet, yards, meters, kilometers, statute miles, nautical miles, US flight level (hundreds of feet by barometer reading), AGL vs. MSL, etc. Sometimes, people mix customary, nautical, and SI units, such as by showing distances in km, altitudes in feet, and speed in knots (NM/h).

If I were ever to somehow become absolute tyrant over the entire Earth, my first edict would make open and notorious use of non-SI units a capital crime, then I'd go live in my imperial hermit shack on a remote mountainside (paradoxically, one with a >1 Gb/s network connection) until my minions finished mopping up the last of the blood.

I enjoy the literary irony here of you imposing a ban on Imperial units and yet you'd go live in an Imperial shack.

Does the software store unit information and calculates with units (like say 1 m / 1 s = 1 m/s) or does it directly operate on the numerical value (so just 1/1 = 1)?

I think the first way should be fast enough with today's processors (but IIRC the processors on the Rovers are slower).

.. I don't understand how you guys can get something up in space. That's amazing. It's either a remarkable feat of engineering or an gigantic 42 gallons barrel of liquid luck.

That seems exceedingly unlikely. No-one in Europe would even consider using anything other than metric units even for buying their groceries, let alone a critical engineering task.

That said, _if_ there were any US companies involved then perhaps there might have been complications. Many years ago I worked on guidance software for tunnelling machines and we had a US client who insisted that we reprogrammed the entire system for them to work in "centifeet", i.e. 100th of a foot or approx 3mm.

I well remember the chaos and arguments caused by adopting such a ridiculous chimera of a unit, and was struck by how resistant some people are to adopting something as universal and transferrable as SI units.

10ths (and hundredths of an inch) are completely standard in US surveying. For example: https://www.amazon.com/Lufkin-FE200D-Engineers-Fiberglass-Me...

Interesting to know - 100th of an inch is extremely small - certainly not visible with my eyesight! Seems odd to need that for surveying.

There are actually a fair few problems with SI units. The candela is fairly ridiculous, the kilogram needs to not depend on a physical object (it's also a little silly to have a prefixed base unit). The ampere is not practical to measure with its current definition, and apparently with the current definitions of Hz and radians, 1 Hz = 1/s = 1 radians/second. Other people take issue with the mole, as well.

SI is an improvement but it is not quite perfect. As long as we have a choice of imperfect systems, we have to pick which is most convenient.

> it's also a little silly to have a prefixed base unit

I somewhat agree with that - I had the exact same conversation with my kids very recently.

Re. physical object: well, whatever physical reference artefact is used, to all intents and purposes the physical stuff is water, with the basically sensible equivalence 1L = 1kg = 1000cc. As you point out, it's not perfect because of the somewhat arbitrary scales (kg, cc), and it gets worse when you bring temperature into it (I believe it's supposed to be measured at 4 degrees C, but I'm not totally sure of that).

You're right in that it is not perfect, but at least it is a fair attempt at coherence.

I believe it's supposed to be measured at 4 degrees C

Correct; it's because at that temperature water is at its densest: https://en.wikipedia.org/wiki/Water#States

re: the kilogram issue:

Redefining the Kilogram with the DIY Watt Balance: https://www.youtube.com/watch?v=ewQkE8t0xgQ

It's very strange because, if would have been a glitch that would cause the program to halt unexpectadly, the engines would have remained running. But from the data, it actually entered into "on the ground" state initiating its exploratory procedure.

But what kind of "on the ground" test condition could have made it to do that ?

The fact that it was "designed to decelerate the craft for 30 seconds until it was metres off the ground" but instead it ran only 3, raises many questions. Not to say that there are zillions of ways to actually test that you are on the ground or not..

> if would have been a glitch that would cause the program to halt unexpectadly, the engines would have remained running

I don't know about that. I've worked on systems where there is some hardware with thruster duration that software is writing. So if software crashes, the thrusters stop. (But I have never worked on a lander). If software is running the control loop, things would get very bad if the engines kept running at their previous output levels for 27 seconds. Thrusters are never aligned perfectly, who knows what kind of rate the lander had, there may be wind pushing it, etc. If a thruster blindly kept running, the lander may have spun up, flipped over, etc.

Ok, good point.

But the point that I wanted to make is that a software crash wouldn't explain why the lander behaved as if it was on the ground. So it's more probable that it reached the "on the ground" condition.

If that condition was adequately programmed, the chances that some sensor glitch could have triggered it are quite low. So the probability is higher that it was either badly programmed or maybe someone flipped a bit somewhere ..

Yeah I agree, it seems like that between an accelerometer and gyro that it should be fairly clear when you reach the ground condition. Maybe a switches that engages when a lander legs are pressed or something similar as well.

The lander didn't have sensors on its feet, so my guess is there may have been two conditions to check for "on the ground": either a vertical speed of 0m/s (gyroscope), or an altitude of 0m (radar).

Given that the parachute was deployed at the correct time and its deployment was governed by air speed (to be deployed around 1mach), the gyroscopes were likely working correctly. The radar detection wasn't available until after deployment of the parachute (more specifically, after ejecting the heat shield, which occurred at the same time), so if the machine's decision was based on faulty input, the doppler radar is likely to blame.

It is also possible that the sensors were working correctly, but the inputs were processed incorrectly. I'd expect that an issue like that could have been caught while testing the lander on earth...

edit: it's also unlikely the engines would have kept firing after computer malfunction: there were multiple (9 iirc) thrusters all around the craft and they were fired independently using pulse-width modulation. This approach was intended to both slow down the descent and to stabilize its course/orientation.

Don't most landers have a mechanical probe extending down from the landing gear to indicate contact with ground? I wonder if this lander was so equipped?

Unless they used a ground proximity radar which sent back false pings due to overheating caused by the too early discarded heat shield?

I'll wait for the final crash analysis and report, but I trust that the lessons learned here will significantly improve the chances of future missions, if at least by adding some more redundancies into the mix.

See my post above about a similar NASA incident. The ground contact in that case was indicated by sensors on the landing gear. They appear to have recorded a false positive when the gear were deployed. Different spacecraft use different methods, I don't know what technology Schiaparelli used.

Is software easier or harder than hardware?

Somehow people seem to assume software is easier, but then there are at least as many critical failures blamed on software as on hardware.

Maybe there are different curves: software is easy to get to 90% working but hard to get from there to 99.9%. Hardware is hard to get to 90% working but easy to get from there to 99.9%.

I don't know much in the way of electrical engineering, but I think the complexity possible is fairly close, but its likely easier to judge a hardware design as being sound since it ultimately has to tie back to reality. I think the complexity of software is that it's almost completely abstraction.

So the question reminds me of someone asking, "What's harder, physics or philosophy?"

> the lander's computer thought it was ...

> led the craft to believe it was...

Don't anthropomorphize computers, they hate that.

As an armchair space explorer, I would have imagined such a crucial part of the mission had serious redundancy.

I would have imagined such a crucial part of the mission had serious redundancy

Redundancy adds either weight if you're computing things on two systems at once, or time if you're recomputing using different code. I'd guess that working out the balance between risk of failure, weight and time is a very complicated problem.

Also redundancy doesn't solve a lot of potential problems. What if your redundant systems disagree? Which one is wrong?

It probably did, and whatever went wrong went wrong across the redundant systems, which is why they're thinking it's a bug.

It could be something straightforward that presented an unexpected input and made it behave in an equally unexpected fashion. I'd imagine they're currently replaying the sensor data to an instance on earth and seeing how it responds.

I think the anthropomorphizing in this case is fine. Someone familiar with programming easily takes "computer thought it was on the ground" as shorthand for "computer began executing subroutines designed to activate when the craft was on the ground."

The anthropomorphized version is more concise for someone familiar with and more understandable for someone unfamiliar with the inner details of computer systems.

I think it was a joke, seeing as SixSigma proceeded to anthropomorphize computers ("they hate that").

A bit of both.

I actually don't think anthropomorphization is helpful in discussions with the lay public but I'm not that serious about it.

Perhaps it is also a matter of how you fix it. Generally software bugs can be fixed by patching over the Internet, making them less scary, whereas hardware bugs are set in stone. (Stone is mostly silicon, right?)

Of course, this is a bit different for software we ship to other planets :P

By mass, stone is mostly oxygen. Silicon is in second place.

From my experience working across both industries: Software is harder than hardware (generally speaking).

The root cause is that hardware has clear purpose and is limited by the real world.

An aircraft from 2015 or from 2000 does the same thing: Get you from Point A to Point B, same world, same constraints, same goal.

A software that does something today. It doesn't give any clue about what it will be or do in 2-5-10 years.

IMO: Software projects are way more ambitious, more complex and evolving faster than hardware. What is expected from software is NOT expected (nor achievable) in hardware at all, that's one of the first thing I noticed.

---

Nonetheless, hardware can run software onboard. So you get hardware and software dev projects, both running along. That get some quirks of both world. IMO, the hardware [and real world] limitations will quickly limit the software.

Is software easier or harder than hardware?

That's hard to say in general. With current practices, I'd say software is harder to get right, but that is mainly due to a failure to spec correctly. Hardware is engineered with very tight controls: dimensions, shapes, and properties are all produced according to a specification that doesn't just specify its nominal values, but also the acceptable tolerance for any deviations.

No software that I know of is engineered to the same level of precision as hardware. The lack of specification-guided tooling (and validation for 3rd-party components) is what puts software development not even in the same league as hardware development.

> No software that I know of is engineered to the same level of precision as hardware.

Are you counting NASA-produced software? I'm not familiar with ESA's software pipeline but I imagine it's similar.

Sadly, I'm not. I truly wish I could say that NASA-produced software is within the range of "software that I know of", but it's not.

Computing Needs Time [1], which focuses on the handling of time, is a nice read on this topic from the software point of view.

[1] http://digitalassets.lib.berkeley.edu/techreports/ucb/text/E...

On that subject, I like this quote:

"Hardware eventually fails. Software eventually works." -- Michael Hartung

What makes you assume the root cause was the software ?

Perhaps the hardware altimeter was sending errors or the gyroscope was out ?

The comet lander Philae bounced and was lost because the hardware harpoons failed to secure it.

The issue is an ongoing battle where hardware functions get emulated in software to save costs which just makes the software more complex and error-prone.

Reminds me of the old joke. If it was a hardware failure, the hardware guys will spend months building models and testing them for stresses and strains and running simulations to find the point of failure. The software engineers will just ask if they can run it again, with breakpoints.

>Is software easier or harder than hardware?

Neither. Rather, it's the total lack of design accountability procedures in place for software. When a mechanical or electrical engineer messes up, there is a legal framework in place to hold the involved parties responsible for their actions. Depending on how bad the negligence was, this can include jail time. This motivates those designing the hardware aspect of a product to check and recheck their work many times over, and as a result you rarely read about hardware failures with this level of frequency.

In contrast, when a software system fails, everyone seems to just throw their hands up in the air and say how this is par for the course. There is currently no legal PE certification available for software, so we have seen disproportionately many catastrophic software failures.

In my experience, hardware is much more difficult to debug. But my hardware experience is limited to FPGAs, never a full ASIC team. There are nightmarish bugs in hardware associated with things like clock domain crossings, bad timing constraints, and signal integrity. Give me software bugs over that any day!

I know this is off-topic, but reading about it's mission has me wondering why (as far as I know) no agencies are sending devices that could return more sophisticated visuals. Even something simple like the 360 YouTube videos would be beyond words, and everyone could experience it. I've really enjoyed the panoramic images. I just wish they'd focus more on trying to get people there virtually since it's a fraction of the effort of getting them there physically, and it could be shared with everyone. Is there some kind of barrier to sending that much data through space? Or should the focus always be on looking for signs of life?

Satellites still use CPUs with speeds in the hundreds of MHz, because creating technology hardened enough to withstand travelling through space, or being used on another planet, is a very difficult challenge, and we just aren't at the point where we can make an Intel i7 (for example) work in space. This isn't even accounting for heat issues, and power issues.

Because of this, my guess is that the camera technology that we can safely get to Mars is significantly worse than what we expect in our day-to-day lives.

ok you answered my point...but is there not another way to shield consumer electronics ? wouldn't it be cheaper to find a good shielding system (magnetic ?) than to develop expensive custom hardware ?

Ever try upgrading the CPU on an iPhone? It's like that.

These projects started out 10-15 years ago. So upgrading them mid-stream is almost impossible.

They're stuck with the hardware available when they chose that part. Re-designing for new hardware means changing everything.

I never considered that. I suppose that does give me some hope that we'll see future projects with today's hardware eventually.

Sure, but by then we'll be lamenting the lack of TrueVR(™) support that would allow us to walk the surface of Mars.

http://asia.nikkei.com/Business/Companies/Canon-readies-mini... Is interesting to see.

But as I'm developing software for my Universitie's cubesat, it's incredibly interesting to read all that goes into Single Event Upsets and other hardware failures from being outside of Earth's protective atmosphere.

Is it an EMI issue? Slower speeds somehow stand the trip better? If there was enough money in it, I'd bet our top minds could find solutions to these problems.

Its answered downthread, but I'll tell you too and add some. The main issue is that the project started 10 years ago, and that is when the general specs were set for pretty much the whole bird. Minor upgrades are made, but you have many many teams working on many little things that have to be integrated, and that spec is 10 years old. I mean, our tech from 10 years ago is billions of times better than a Viking lander, but I get the sentiment.

For silicon devices, the VERY hard rad environment of inter-planetary space is pretty much instadeath. So you are forced to be 'dumb'. You can shield it, but that adds weight and heat issues. Redundancy is out, it just doubles/triples the weight, heat, and interfacing issues. You can use rad-hardened devices (best idea) but then you have to fab that silicon anew, and this is really expensive. Then you have to test all these things before you put them up there. So you need to put your device in a vacuum-rad chamber. Governments are not so hot on just having these rad sources laying about and letting a piece of silicon 'cook' next to a power station that is in use. So you do accelerated testing, aka, super radiation. I've been in those chambers before, it's scary. After the magic glow machines are turned off, you have to evac all the air as the O2 in the room has been turned into O3 and you can't breathe in there. The roaches in the corners are literally cooked and brittle. It's nuts. Unfortunately for your testing, this environment is more like the sun's corona than the Earth-Mars transit, so you have to take the testing with a grain of salt. Also your vac chamber is not nearly as good as real space. So everything is oxidized and hat messes up your analysis. You do the best you can, but we don't get many sample returns from outside the Earth's magnetic field to test against. NASA et al do a GREAT job, but they are dealing with very extreme jobs and we just don't have a lot of data about what we are dealing with. We can't do a lot of in situ tests.

Less speed = less heat which is surprisingly a large concern in space. The other factor is that the transistors on the chips are larger than the ones used on Earth. I don't remember the specifics but the reasoning was that less transistors would be damaged if a spot on the chip was hit by radiation and built in redundancies would be able to compensate. (Source: had a few EE classes where this kind of thing was brought up been awhile though)

Thinkpads are used on the ISS but the ISS itself probably uses far more reliable and slower hardware.

http://blog.lenovo.com/en/blog/thinkpad-laptop-nasa-youtube-...

The ISS is still close enough to the Earth for it to experience much less radiation than it would while travelling to Mars.

Correct, and the reason it is only thinkpads, is because that particular model went through extensive testing and verification that its heat output, etc, were ok for use up there. Also I don't think they are used for critical systems.

this is something I never got: why don't they just chuck a whole bunch of cheap consumer gadgets on these things ? Is it really that hard to attach an array (for redundancy) of gopros to constantly transmit video at 60fps (and sound, even if Mars doesn't really propogate sound) ?

Consumer grade electronics would likely not survive the radiation that is present in space outside of earth's atmosphere.

Mostly because of mass. I think the cost of these probes scales with mass and adding those cheap gadgets would displace other scientific instruments. A 60fps camera might be interesting for entry but then it's mostly useless on the surface. And it would take a veeeeeery long time to transmit video (even if you downsampled in spatial and temporal resolution it's just not worth it).

As pointed out elsewhere in this thread, the consumer grade devices cannot withstand the extremes of space (not just temp extremes, also high energy rays/particles we're shielded from here).

> why don't they just chuck a whole bunch of cheap consumer gadgets on these things

This [0] is an interesting read on some of the considerations for the Curiosity rover cameras.

[0] https://www.dpreview.com/articles/0353350380/curiosity-inter...

great link thanks!

Yes. Space radiation doesn't kill 1% of gopros, it kills 99% of them (both numbers made up, but you get the point). Then there's the power budget. A watt is cheap here, not so cheap on Mars, so even what counts as "low power" on the surface of earth isn't cheap on Mars. And of course, n-item arrays need correspondingly more power than single items.

(I realise it's not your main point, but broadcasting 60fps video across the solar system sounds like it might need a LOT of power, no matter of sensitive the receiver is.)

Electronics to be used in space must be hardened against:

- radiation: electrons, protons, heavy ions (MOSFETs are very vulnerable)

- cold temperatures (up to -235 degree celsius)

- high temperatures (up to +250 celsius, and there's no air-cooling in space)

The performance of electronics vary greatly depending on radiation amount and type, and temperature. You can absolutely not throw consumer electronics (GoPro) in deep space and expect it to function.

the focus of this mission was the orbiter, the lander was basically "ah, we can fit this too", that's why it had minimal capabilities.

I get it, and I'm definitely not knocking any endeavor as being trivial. I'm just curious, in general, why this isn't more of a focus. Possibly for selfish reasons.

The hardware has to be manufactured from the ground up for space. The silicon has to operate over a wider temperature range, the processor needs internal redundancy as well as external redundancy, and memory (caches included) has to include support for multi-bit error correction. Getting electronics to work reliably in space is no small feat.

And since the fabs that manufacture this kind of stuff are working at a relatively large technology node, it's non-trivial to design something like an i5 for space. Why? Well, Intel and friends have no interest setting up an entire fab process that will be utilized once in a blue moon. Even if you could use a more advanced node, the extremely high circuit area overhead required to realize the requirements I listed above will remain a limiting factor.

Plus the very real and difficult problem of any system being literally shaken apart under the launch forces. AFAIK, they like to pot all circuitry in epoxy to stop the solder joints breaking.

I did an internship at an oil drilling company, and the electronics on the tools were covered entirely with epoxy (or something similar) to minimize the impact of mechanical shocks. So yeah, I'd assume you're right.

I hope they didn't mix up metric and imperial units this time again. Or is it the Agile specter that haunts software made in Europe?

I'm wondering if they used formal verification and how they tested the software.

Given that it's an European project I'd be surprised if it was a metric vs imperial unit mixup.

UK says hi.

And soon "bye."

Not necessarily, the ESA isn't an EU-only institution.

The UK isn't going to leave ESA or CERN, not to mention that both of them have non-EU members IIRC.

For what it is worth - apart from road signs and speeds - I've not really seen imperial measurements used for much any more in the UK.

I would be amazed (and saddened) if someone had started a new, large-scale, serious, rigorous, engineering project in the past couple of decades (i.e. since the 90s) and used imperial units (I am sure there are lots of small-scale/personal/etc stuff done in imperial though - I dont count interplanetary space flight as small-scale!)

Imperial is pretty much dead in the UK (or maybe just London?) apart from roads and conversational/casual usage where its often easier to say the imperial equivalent than the metric (e.g. "pint of beer" is easier than saying "568 millilitres of beer", "about a foot" is easier than saying "about 30 centimetres" just because of fewer syllables if nothing else).

In other words, not dead at all. In fact, still in common usage.

If you think imperial measurements are dead in london you never had to deal with plumbing in your apartment where you have a mixture of imperial and metric pipes and threads ;) And the best is always finding a metric pipe with an inch thread pitch on it....

Also IIRC technically most engineering schools in the US also use metric these days, it doesn't mean stuff doesn't get screwed up, imperial on it's own is also annoying since you have both decimal (thous) and fractional units which I always found frustrating on it's own.

thruster_test(3000); // todo: add powered descent code

Everyone has a plan, until they get hit by a planet.

-Mars Tyson

They should have used Haskell.

Damn, beat me to it.

then they'd have 2 problems.

I assume this is a joke? Is Haskell suitable for hard real-time systems with things like lazy evaluation and a GC?

I think it's a joke. Haskell is manifestly not suitable for writing a Mars lander.

A verified DSL written in Haskell on the other hand ...

http://ivorylang.org/ivory-introduction.html

Reminds me of that robocop scene "You call this a glitch???"

This was my first thought actually. Faulty ground sensing has doomed many a craft.

I was about to suggest n-version programming redundancy, when it occurred to me: aren't we starting to reach a point where deep learning systems are good enough for unmanned spacecraft? How we would generate the learning data would be a question though.

You wastely overestimate the capabilities of the equipment we send into space. It is nowhere near powerful enough to power any meaningful deep learning system in an online fashion. Offline has either to high latency, or a problem getting data to seed the initial learning (and even then we have a problem actually running the network).

Well than and there just isn't a whole lot of training data to play with. It's doubtful that a simulator could ever model actual landing situations well enough that the whole thing will a) work on Mars and b) not overfit weird behaviour in the simulator. Faulty ground sensing has resulted in the loss of more than one mission, and deep learning isn't going to fix any of that.

Oh, and s/wastely/vastly/ ;)

this days, almost every failure can be related in some way to a "computing glitch".

Glitch is a strange word to blame everything on. It implies a temporary problem. This sounds more like a more permanent problem, as in it's permanently crashed into the ground.

Blasted clipping errors.

It seems no matter the problem there's always someone trying to blame IT.

Reply on news.ycombinator.com