Hacker News Reader

Comment by code4tee | Hacker News Reader

The “autopilot” concept isn’t going to work at scale with cars. It’s either autonomous or it’s not.

As these accidents demonstrate drivers can’t be relied upon to ‘monitor’ the autopilot and react if something goes wrong. It’s a deeply flawed design concept from an otherwise very innovative company.

View on HN

As these accidents demonstrate drivers can’t be relied upon to ‘monitor’ the autopilot and react if something goes wrong.

We don't know how many other accidents have been prevented by autopilot. A safety system should not be judged by whether it's perfect, but by whether having it is better than not having it.

The NHTSA did a study on exactly this issue by looking at accident rates in cars with autopilot hardware before and after autopilot was actually enabled via a software update. They found that accident rates went down by about 40% after autosteer was enabled.

This seems to suggest that either people are fine with monitoring and taking over when needed, or else autosteer was already so good in 2016 that it made fewer mistakes than humans.

https://static.nhtsa.gov/odi/inv/2016/INCLA-PE16007-7876.pdf

>else autosteer was already so good in 2016 that it made fewer mistakes than humans.

No no no no no! The NHTSA report says nothing about whether Tesla's system is safer than humans. In that study there are two buckets, one which is total Tesla miles in TACC-enabled cars and then after the update total Tesla miles in cars with TACC + Autosteer and they calculated on airbag deployments. Human driven miles are going to dominate both of those buckets and there's a reason the NHTSA report makes zero claims about Tesla's safety relative to human drivers. It's totally outside the scope of the study. Then add in that some researchers who are skeptical of the methodology and have been asking for the raw data from NHTSA/Tesla have yet to receive it.

I didn't say it was safer than humans. I said that the study results suggest that either humans are fine with taking over and driving manually when required or if humans are not taking over when they should then autosteer might just be safer than humans. I don't claim to know exactly why autosteer would reduce accident rates, and neither does the study, but there must be some explanation and the last bit of my comment was simply speculating as to what that explanation might be.

Or it could show that Autosteer was so unsafe that people became scared and stopped using it entirely, engaging in the much safer method of driving manually and lowering the accident rate of Teslas after Autosteer. Or it could show that drivers new to Teslas have initial higher accident rates because of the driving differences between traditional gas cars and EVs. Or it could show that Tesla changed the sensitivity of the airbags in the same update.[1] Or it could shows that autosteer is actually less safe than TACC alone but AEB and collision warnings more than make up the safety difference. The point is that we don't even know that autosteer is actually reducing accident rates. No matter if you're a fan of Tesla or not you should support the release of the data the NHTSA worked with so we can do an independent analysis.

[1] It can do this, see: https://electrek.co/2017/06/07/tesla-software-update-airbag-...

Great post and completely agree. You can NOT be kind of driving.

Plus who does not work with a limited group of users while you are testing. That is true for tech that does not kill you and more so when it can.

What Tesla is doing is honestly close to criminal. Waymo approach makes far more sense.

I disagree, some kind of driving still can be useful. Of course like anything there always be some risk involved. Even when the death do happen, you can still learn from it.

The problem is you can NOT be kind of driving. The Uber video of the driver is so telling. You can see he is reading his phone when the car hit the lady.

Google has indicated this for years. There really should NOT be safety drivers behind the wheel.

What Tesla is doing is almost criminal. Having a autopilot button when the car really can not do auto pilot is about as wrong as it gets.

I do not believe you can do SDC directly to the consumer while it is really in beta.

Obviously you CAN. Of course it wont be perfect but most of time it should be fine. In some situation its still better than a driver using phone while driving a car that has no self driving capability whatsover, which many people do that right now.

Waiting until the technology is perfect will take a long time.

Could not disagree more. When I say "CAN" I mean safely.

You touched a nerve on some people. And rightly so. Perception and context awareness is not instantaneous.

Here's a quick improvement on testing for the learning agent: watch the driver while he is driving, watch the road, AI predicts what do in the same situation, diff that with the driver; diff it offline if necessary. Bang! massive improvement in learning data, time and pinpointing blunders. If they are not already doing this, they should.

Tesla is doing this. Comparing driver vs autopilot, all information from the fleet, is sent back to Tesla and used as learning data. Thats one of the reasons why all the cars have free internet access.

So Tesla is using its drivers as guinea pigs in a high steaks game that could result in death?

A lot of people want to treat Tesla like they're just a tech company, while ignoring that the "A/B test" conducted here could literally get people killed (and has).

For example, in the most recent death some owners are claiming that an over-the-air software update caused the issue (lane centering), it made their cars drive differently. In tech regular updates are common, but normally the biggest consequence for getting one wrong is angry customers, not dead customers.

This is done everywhere. Testing tires or medicines or road surface treatments. We all risk our lives every day to some degree. Testing how best to preserve them with A/B testing is a long respected approach.

And is it fair to put the blame solely on a cherrypicked action by Tesla engineering? A tragic coincidence is a compilation of events.

Tires, medicines and road surface treatments are all tested in controlled environments before being released to the public, not tested on the public.

Conducting A/B tests with live human subjects and in a context where failures will kill people (including those not even associated with the experiment!) is the level of scientific rigor I expect from Josef Mengele.

Inb4 "don't buy a Tesla then" - this is not limited to Tesla customers, but extends to anyone in the vicinity.

I drive on an experimental highway surface daily. It's quieter, but uses an unorthodox drainage system (that at least feels less controlled in wet conditions). Only real use statistics will say for sure if it's safety-equivalent to standard surfaces. The state DOT is testing this on me and tens of thousands of other people daily at high speed. Sure, I suspect that they did smaller scale testing before deploying it, but so has Tesla. So tell me: what's the difference?

That road surface was first tested in non-public situations to verify basic safety and functionality concerns. If it's reached the public-stage of testing, that means safety concerns were addressed and they're now testing durability.

I read the parent comment "Tesla is doing this" as confirming that Tesla vehicles alway have Autopilot running, even when it's not controlling the car, and comparing what Autopilot thinks the driver should do with what the driver is actually doing.

If true, having that data would give Tesla an enormous amount of data on situations in which Autopilot might have failed if it was in control of the car, such as if Autopilot (when not engaged) didn't see an obstruction but the human driver swerved suddenly.

This is also an advantage of Tesla installing full Autopilot hardware on every car regardless of whether the Autopilot upgrade has been purchased.

What're your thoughts on cruise control?

Cruise control is very different. It doesn’t allow people to take their attention off the road. In the Tesla deaths it appears quite clear that the driver was not paying attention to the road.

Reply on news.ycombinator.com