Hacker News Reader

Comment by fdej | Hacker News Reader

If a computer crashes, you can identify the software or hardware flaw that caused the crash and fix it once and for all.

Improving human driving skills collectively is much more difficult. Even if one driver learns from their own mistake, other drivers and the next generation of drivers will still be just as prone as before to make that mistake. Better education can only do so much anyway, since people often don't think rationally and don't necessarily make safety the number one priority (e.g. drunk driving, trying to get to the destination as quickly as possible, or reckless driving for entertainment purposes).

View on HN

> If a computer crashes, you can identify the software or hardware flaw that caused the crash and fix it once and for all.

Right. Except that 40 years later computers still crash. The more complex a system the harder it becomes to debug and there is no complex software system without bugs. It's like rocks in a field, you'll never get the last one out even if you think you did.

And 100 years later humans still do stupid things in traffic (not to mention they also "crash" - fall asleep, get strokes, whatever). Why does a computer driver have to be flawless when human drivers are anything but?

It doesn't have to be flawless, just better than humans.

And that's not a simple thing: it will have to be better than humans in every situation, just being better in some but clearly worse in others will still end up looking bad for self driving cars. People don't deal in statistics, they simply look at the situation and say 'that would never happen to a human' and will keep on driving themselves.

So it's a very fine line to roll this sort of thing out, do it too early and you end up turning off your potential early adopters.

We build lots of computing systems that rarely have components crash and almost never (eg, not even once over decades) have the whole system go down. We know how to do it.

It's just not worth the effort in the vast majority of circumstances -- so you get computer crashes.

I know we know how to do it. It's just that the cost-benefit analysis usually falls to 'let it crash' rather than 'let it crash and put a supervisor on top of it'. Besides the fact that most developers don't care too much about their stuff crashing (for instance: one valid excuse for not digging in and solving a problem is 'can't reproduce', whereas those are the worst problems to allow to persist).

In the end it is all economics. But I really would like to see some real life stats on how these self driving cars perform over the longer term in less controlled situations and with an honest accounting of the root cause of any issues. Fat chance that will ever happen but that's the sort of rigor that would get me to switch from driving myself to being driven by a pile of software.

That's the good bit about Google being involved: at least they really 'grok' software and reliability engineering. As opposed to your average car manufacturer where software is a dirty word best outsourced to some contractor.

Regular software, yes, they crash all the time. But I would think these systems would be more safely handled. For example, nuclear energy, air traffic control, bank systems and automated train lines don't crash nearly as often, for example.

Being such an important feature (not crashing) it shouldn't be ignored. I am also skeptical but we have examples of reliable systems so I believe they can work.

> nuclear energy, air traffic control, bank systems and automated train lines don't crash nearly as often, for example.

Bank systems crash with great regularity. Source: worked for a bank, customer of a bank.

Industrial systems and anything related to flight is typically much better from a design point of view than your average bit of firmware. Redundancy is built in from the first day and all failure modes are tested in so much as feasible.

Also, and this is very important, such software is kept as simple as possible to reduce the surface area bugs can hide in.

Also! Don't forget how expensive this software is... This puts it out of reach for like 97% of the world.

Also, all of these are much simpler problems than fully-autonomous operation of a vehicle on a regular road surrounded by other users, which involves teaching the system how to deal with an enormously long list of edge cases.

I don't envy people tasked with balancing the need for self-driving car software to be able to handle a sufficiently large number of possibilities to consider eliminating the driver with the need for it to have a small surface area and sufficient degree of tractability to be able to minimise bugs.

When was the last time 40-year-old banking or safety-critical software crashed?

There's a flip side to that though. You may have a systematic vulnerability or bug that puts thousands of vehicles at risk. Bruce Schneier has spoken about this issue. An expert lockpick can still only pick one lock at a time. A super-skilled hacker or team can penetrate many devices in one shot.

Reply on news.ycombinator.com