Hacker News Reader

Comment by underbluewaters | Hacker News Reader

I think it's pretty clear that the bar for safety must be much higher (10x) than human-level for these to be accepted by consumers. Auto accidents are very common, and the first time someone has an accident with a car like this everyone they know will hear about it. It will be terrifying. If these are only as safe on average as a human driver then nearly everyone is going to have at the very least a 2nd-hand negative experience.

The emotional response to these accidents is not going to be entirely irrational either. If I have a minor accident with my traditional truck, I'm going to probably have a good understanding of what went wrong and how I can prevent future collisions. With an autonomous vehicle... software upgrade? I'd rather take responsibility for my own safety if that's the case.

View on HN

Which one will learn faster: each individual driver each time they have a minor accident (or a close-call)? Or the autonomous vehicle software updated from data collected from each and every accidents involving an autonomous vehicle? The result will probably be software-updates like Tesla cars are getting today.

I suspect that if the attitude towards autonomous vehicle accidents is similar to that of aircraft accidents, they will become extremely safe very quickly.

I suspect that if the attitude towards autonomous vehicle accidents is similar to that of aircraft accidents, they will become extremely safe very quickly.

On the other hand, if every autonomous vehicle accident is investigated to the same standard as every aircraft accident, it appears likely that the entire system will become so overloaded and cost-ineffective that it is completely impractical to operate before we get over the hurdle and achieve any consistently safer performance that might otherwise be available.

I agree with the risk of overload. Sanity needs to prevail.

But by similar attitude, I was mostly thinking about the rigorous and systematic efforts to find root causes of accidents, rather than the toxic attitude of attempting to deflect blame (and liability) that is so pervasive in other industries.

Safety starts with the culture: it's not an add-on or an afterthought. If Google starts to not fixing problems with its cars to avoid acknowledging a problem (and the liabilities that go with it), then we have a problem...

The point is not which is safer on a bottom line level, the point is consumers are terrified of the idea of surrendering control to a computer. Lots of people are terrified of flying for example even though it's much safer. Unless it's 10x safer people will be scared of automated cars and possibly even push for short-sighted laws.

Millions of people commute by subway everyday without suffering panic attacks. Tens of thousands (at the very least) ride Paris completely automated 14 subway line every day without suffering panic attacks, or any of the many other automated subways/monorails all around the world.

And yet a subset of them are probably terrified of flying. Because flying, in a way, is scary. Unlike birds we are not wired to fly. But that's fine: autonomous cars won't be flying. If train/subways/tramway/bus can be successful, so can autonomous cars.

The idea that humans learn well from accidents stands contrary to all available evidence.

What evidence? Certainly individual humans (anecdotally: like myself) do.

Look at insurance premiums. Premiums go up after an accident because it is more likely that you are a poor driver and likely to have another accident.

That doesn't mean that people don't become safer drivers after an accident! It just means that the variance between drivers is more significant than how much a single driver improves.

Insurance companies raise premiums after accidents because there is an excuse to do so. It has nothing do with your likelihood to have another accident, and everything to do with charging you more so that they can attract new customers with a lower price. Premiums go up even if the driver is found to be not at fault.

I've been involved in several reported accidents that were judged to be fully the fault of the other driver, and my rates have never increased.

There are differences across states as to whether and when it's allowed. That might explain why it hasn't happened to you.

Anecdote not evidence, but still interesting - I know someone who was no-fault in an accident, and her premiums still increased because the insurance company claimed that statistically anyone who has an accident is more likely to have another, even if it's not their fault.

Same. I've also had one at-fault accident and my premium went up in the sense that my "accident free discount" was removed for about 24 months. But I eventually got that back too.

They will drop you for being unlucky if they are too closely spaced together.

Relatedly, my home insurance company dropped me for being unlucky enough to get burglarized.

Yes, this is my point: rate increases after accidents are done by choice of the insurance company, not because their risk management requires them to do so, as the grandparent post implied:

> Premiums go up after an accident because it is more likely that you are a poor driver and likely to have another accident.

No, premiums go up in that situation when the insurance company thinks they can do so and not lose too many customers. As you point out, not every insurance company operates this way.

An insurance company only cares about managing risk and revenue across the entire pool. They plan to pay out a certain number of claims, so any given accident might simply be fulfilling the actuarial expectations and not altering their risk calculations at all.

It is no more in an insurance company's interest to raise rates arbitrarily than it is for any other company. If one company's premiums are significantly more expensive than other insurers, they'll lose customers – and if what you say was true, any insurance company that realized this could cease the practice and gain many new customers.

The whole point is that in the post-accident customer's mind it is NOT an arbitrary price increase--the customer feels that they "earned it" by getting into an accident.

But all the insurance company cares about is revenue vs. expenses across the entire pool. If they can charge one customer more, it allows them to charge another customer less--like, by advertising a low fee for new signups.

Premiums go up because information is revealed to the insurance company that you are the type of person who gets into accidents. You were always this kind of person, but the insurance company didn't know until it happened.

From this point on, you might become less accident-prone than your previous self, but you established that you're more likely to get into accidents than the average 0-accident driver.

And what evidences are you bringing to the party?

Of course I'd learn from each fender-bender, and I'm clearly a better driver than average anyways! These fancy autonomous vehicles are too dangerous for my family.

> I think it's pretty clear that the bar for safety must be much higher (10x) than human-level for these to be accepted by consumers.

Will it? Think about the recent Toyota brake pedals that didn't work or the GM faulty ignitions. Both of these caused a number of deaths, and the Toyota one was especially scary because it didn't even need accident-like events to cause a problem, your brakes would just stop working.

It'll hit the news (like Tesla battery fires vastly disproportionately reported on over other, much more common auto fires), but it's not clear at all that it will cause some massive panic or backlash.

About the Toyota's unintended acceleration (UA) incidents. From Wikipedia: "On February 8, 2011, the NHTSA, in collaboration with NASA, released its findings into the investigation on the Toyota drive-by-wire throttle system. After a 10-month search, NASA and NHTSA scientists found no electronic defect in Toyota vehicles.[27] Driver error or pedal misapplication was found responsible for most of the incidents.[28] The report ended stating, "Our conclusion is Toyota's problems were mechanical, not electrical." This included sticking accelerator pedals, and pedals caught under floor mats.[29]"

https://en.wikipedia.org/wiki/2009%E2%80%9311_Toyota_vehicle...

If this was true, then the self-driving car may have prevented the accidents. If it was really a software problem, then fix the software. It may be much more easy to fix the software rather than to fix human behaviors.

The NASA audit wasn't nearly as comprehensive as you may have expected: http://embeddedgurus.com/barr-code/2013/10/an-update-on-toyo...

NASA didn't find a software defect, this is not the same as NASA finding that there is no software defect.

Toyota absolutely built and sold dangerous vehicles with software and hardware defects. This was not only a matter of gas pedals sticking to floor mats.

Importantly in this context Toyota did not put customer safety first and decided to suppress the truth of the problem rather than own the solution. This is cause for concern as we move toward self-driving systems which have to be perceived to work in order for the product to remain economically viable.

I am already aware of the Barr Group team and I know their claim.

>NASA didn't find a software defect, this is not the same as NASA finding that there is no software defect.

Read my post carefully. I haven't claimed such a thing and I offered two possibilities.

>Toyota absolutely built and sold dangerous vehicles with software and hardware defects.

You have to be careful with this. The Barr Group team haven't proven the relationship between the accident, and the software defect which they claim to found of.

The important thing is that software, hardware, system, and the whole design for car safety can improve significantly better than the current situation, which is much more better than just relying on human drivings.

I find it unlikely that consumers are as safety conscious as you imagine them to be. Think about how many people currently text or do other things to distract themselves while driving. They willingly choose less safety in order to be able to do something else while in transit. Pushing this behavior to its natural conclusion is what will allow self driving cars to be accepted even if they are 10% less safe than normal drivers.

People text while driving because they think they are better at doing two things at once than they actually are. For the same reason, they will expect that they can out-perform a robot unless those robots are so overwhelmingly safe that conclusion can't be rationalized. No consumer will be thinking about the big picture.

I agree that the bar for safety should be ~10x human-level. But I also find it interesting, because that should be a moving target. Eg if we realize humans often rear end the car in front of them, and fix that with "auto braking", then we'll improve how safe human drivers are.

Thus, hopefully, both humans and self-driving cars should have improving records [although I don't have any data to back this up].

You've raised an important point here: the 'human vs self-driving car' is something of a false dichotomy.

The human driving experience will become more and more augmented by technology, with auto-braking, lane-assist, smart cruise control and (eventually) auto-navigation until we have almost-self-driving cars which are still legally required to have a qualified human in the driver's seat. Many of the these features already exist in high-end cars.

People will be much happier to hit the 'auto' button and climb into the back seat when they've already had several years of 'almost-auto' driving.

> I think it's pretty clear that the bar for safety must be much higher (10x) than human-level for these to be accepted by consumers.

I think the safety level just has to exceed that of an average teenage driver for widespread adoption by a certain segment of the car-buying population: parents.

EDIT: Teen drivers are 3x more likely per mile to be involved in an accident than adults - http://www.cdc.gov/MotorVehicleSafety/Teen_Drivers/teendrive...

I think in this case, insurance companies will help drive adoption. If an insurance company believes that accidents are say half as likely, you would have to imagine one of them would bite on changing premiums by a bit. Cold hard cash will facilitate adoption, assuming basic regulatory guardrails are in place.

"There were 272 instances in which the software detected an anomaly somewhere in the system that could have had possible safety implications; in these cases it immediately handed control of the vehicle to our test driver. We’ve recently been driving ~5300 autonomous miles between these events, which is a nearly 7-fold improvement since the start of the reporting period, when we logged only ~785 autonomous miles between them. We’re pleased."

785 miles between anomalous events that require manual control is bad enough. 5300 is worse! That honestly needs to be 0 events (impossible as that might be) or only events for which advanced notice can be provided to be market-acceptable.

It's the TSA screening problem all over again: 13-88 hours of mind-numbing normal operation, with a single incident of something potentially exploding that you have to catch.

You mean that 5300 is worse because people will pay less attention?

Well, would it be better if it had a random chance of (artificially) needing human intervention, at on average every 785 miles? (Not exactly, because that would make it too easy to predict, and therefore might not make the person attentive when a real problem happens)

That doesn't seem like it would be too hard to add, if it was better by keeping people on their toes.

But, I doubt the problems get monotonically worse when the average distance between events increases? I would expect at some point, increasing distance doesn't significantly reduce attention , so the safety would increase a bit. Maybe not more than for some shorter distance, but I think for some slightly different distance.

> I think it's pretty clear that the bar for safety must be much higher (10x) than human-level for these to be accepted by consumers.

That's a really low bar.

Take a look at Google Maps in San Diego when it rains. Accidents everywhere. A self-driving car won't get into stupid accidents because it's following too closely, driving too fast, hydroplaning, etc.

Toss in the people who are distracted, tired, or drugged out on medication, and your numbers are even worse.

We're looking at the end of humans driving cars within 10 years.

> I think it's pretty clear that the bar for safety must be much higher (10x) than human-level for these to be accepted by consumers.

You might be right about that, I just don't see why it should be the case. Humans are not very good at driving safely, it seems like an easy bar for robots to clear. Even if robots are "only" twice as safe, we should all jump at the opportunity to halve injuries and deaths from auto accidents.

I think it could be predictability of a self-driving car in an accident that is making people uncomfortable and accident rate does not capture that. For e.g., an accident might means hitting a barrier, a human realizes the terrible situation and will swerve away from the cliff behind the barrier. But what would a self-driving car do? Its sensor might be too damaged to realize the dire consequence.

Reply on news.ycombinator.com