Hacker News Reader

> If you have no clear idea how your driving software works, how are you going to make sure this doesn't happen?

A lot of people don't know how typing "www.google.com" into their browser works but DNS "just works" for them. There is software in nearly every part of our lives that could systemically go wrong in catastrophic ways - we do our best to plan and prevent them.

> People have common sense, variability and adaptive learning to prevent things like that. Moreover, our traffic system, our cars and our driver education have been co-evolving for safety for the last 100 years. (Heck, some of it started with horses and carriages.) The complexity and size of the system have been gradually scaled up for just as long, and it's designed and refined to be used by humans.

Isn't that incredible? The complexity goes up: we adapt. As self-driving cars are introduced we adapt and evolve and incorporate these changes in technology, the driving "environment", security, safety measures, and laws and regulations.

> Traffic system isn't a board game.

I didn't see any efforts at self-driving vehicles just say "ahhhh! this driving stuff is so easy!" Of course it comes with huge difficulties, an enormous amount of complexity, a baffling amount of data, a ton of edge cases, and now a wealth of tribal knowledge on the subject.

When considering these things think of the systemic benefits as well. There was a story on the Google self-driving cars in Mountain View not taking a turn when the passengers of the car expected it would. Turns out that the IR sensors on the vehicle saw that a jogger behind a wall of hedges was about to enter the intersection to cross while jogging. Systemically implementing that change across a fleet of cars would then save untold numbers of injuries or lives.

We adapt.

How about: I like driving so don't take it away. or perhaps: Just because we can, doesn't mean we should.

> Turns out that the IR sensors on the vehicle saw that a jogger behind a wall of hedges was about to enter the intersection to cross while jogging.

Some cars today already have IR sensors (night vision systems) installed in them, and include pedestrian detection monitors as well. Cars don't have to be full on self-driving to have a large increase in safety and accident prevention.

>> A lot of people don't know how typing "www.google.com" into their browser works but DNS "just works" for them.

On the other hand, noone was ever run over by a mistyped URL.

To guard against the risk of a bug/design flaw affecting all vehicles simultaneously, I would include an independent "dumb" override that was too dumb to drive but could stop all vehicles as soon as any of several TBD conditions occurred, regardless of what the primary "smart" system was telling the vehicle to do. It wouldn't be "smart" AI, but just something listening to both the AI's smart sensors and it's own, independent, simple sensors, and if ANY of them reported a listed condition, it would override the smart system and stop traffic.

Human intervention would be required to enable the vehicles to move again.

After some real-world experience, I would want to give the emergency stop system the ability to release the hold on one layer of vehicles at a time, allowing the smart AI to drive again but at a speed limited by the "dumb" system to nothing faster than human walking speed, enabling them to clear the area. If they cleared without further incident, the next layer would be released until the whole stoppage either cleared itself or was restopped because something was still wrong.

Self driving cars do not represent a correlated risk. The situation you describe is completely fanciful. There is no plausible circumstance in which a large number of cars would simultaneously crash.

A 0-day vulnerability in their OS, exploited ...

Here's a link to a 100-car pileup: https://www.washingtonpost.com/news/dr-gridlock/wp/2018/02/0...

These kinds of crashes generally have the some root cause: everybody is traveling too fast for conditions, and when one thing goes wrong, the end result is that you get a chain reaction of everyone accreting into a single crash. If self-driving cars are capable of making the same misjudgements, then these kinds of crashes absolutely can happen.

Sure, but you can't criticize SDC's by saying that they have the same failure modes as humans. The comment I was referring to was suggesting that they'd have some sort of mass failure mode, in which some substantial percentage of all operating SDC's in an area would crash simultaneously. That is clearly not going to happen.