Tesla on part-automated drive system slams into police car | Hacker News Reader

Tesla on part-automated drive system slams into police car

apnews.com

40 pointsShank44 comments

This is a well-known issue with the current publicly available version of the Tesla auto-pilot: it largely ignores stationary objects that partially obstruct the lane.

The next version, currently in closed beta, is supposedly better at tracking parked cars and obstructions.

However, from what I've seen from dashcam videos by beta testers, it's still quite bad. Vehicles "pop" in and out of the AI's perception regularly. Not as bad as before, but still quite scary.

Not just a problem with Tesla. My Audi SQ5 with the 'Advanced Drivers Assitance Pack' will happily drive in to a stationary car in front of me.

It's a problem with radars, they don't detect stationary objects well.

It's not that it doesn't detect them, it's that all lane-following systems are programmed to ignore them, because the radar doesn't have the resolution to distinguish between objects in the lane and objects outside the lane, such as trees or bridges.

It's why Tesla is moving towards an optical based system, and why Waymo uses lidar.

The vehicles popping in and out was improved pretty dramatically in fsd beta 9. Seems unfair not to mention that so I’m guessing you were unaware.

FSD beta 9 which 1) you have to pay 10k for and 2) only a few dozen people are eligible for. Seems like if this is a real problem they should fix in the product that everyone has or stop people from using it

You want them to push out untested, safety critical code to the general population? Don’t think it works that way.

Considering that the current production release runs into stopped emergency vehicles, seems like they already pushed out untested code to the general population. That's a pretty basic thing that should have obviously been part of their testing from the start. If it wasn't, I'm not feeling particularly good about the idea that their production releases are "tested".

Tesla is very clear about the limitations of autopilot, they require the driver to keep a hand on the wheel for a reason.

Would it be better if the car drove perfectly in all situations? Obviously yes. Are some people willing to pay for imperfect software that when paired with a human makes driving safer? Also yes.

This has happened a few times before.

Why does Autopilot not detect parked patrol vehicles? Are other parked vehicles also being hit?

Because the Tesla object detector is too unreliable and fires too often for parked vehicles that aren't there. So they tune it to have more false negatives (to miss objects) because most of the time you're passing by a row of parked vehicles and nothing is sticking out.

It's a sign of just how bad and dangerous the Tesla autopilot really is. It should never be on the road. Dozens of people have been murdered by it. The engineers involved should all be in jail.

I don't think this is the case. We aren't seeing as many stories around regular parked vehicles being it, it seems to be mostly patrol vehicles.

patrol vehicles are the ones stopped on the road partially blocking the lane, so, what's the opposite of survivorship bias?

Tens of thousands of people have been murdered by bad drivers. The entirety of the DMV should be in jail.

There have been a total of 9 Autopilot deaths since the system was introduced in 2015. That's with 1 million+ cars on the road with Autopilot and billions of miles. 9 is a very small number. And in pretty much every case the driver was found to be at fault.

Autopilot is just adaptive cruise control and lane centering. Almost every car manufacturer has these as options.

do other manufactures call it full self driving and let you easily trick it while you sleep in the back seat?

Everyone says this but it’s not actually true in practice. Most of the other manufacturers systems are meant more as an emergency type feature rather than convenience.

For example the Subaru system when it comes to a stop in traffic requires you to hit the accelerator to tell the car to start going again. Other systems have similar issues with usability that actually make it such that most people with the cars don’t use these features at all.

> For example the Subaru system when it comes to a stop in traffic requires you to hit the accelerator to tell the car to start going again

No it doesn't

I think that feature was only just added to 2021 model year Subarus. I have a 2019 and after coming to a complete stop it definitely requires the driver to press the accelerator to start moving again.

Subaru has the traffic jam assist on their adaptive cruise control. Traffic jam assist means it will handle stop and go traffic without input from the driver.

https://rosap.ntl.bts.gov/view/dot/55587/dot_55587_DS1.pdf

Autopilot is generally only operated in fair-weather and good visibility. Sensible drivers, under other conditions, would not handover responsibility to the machine. In other words, the driver chooses to give the cars the 'cakewalk'.

In addition, the Autopilot has a redundant backup -- namely the human operator. There could be thousands of unreported crashes/deaths but for the intervention of the human operator. Accordingly, dividing 9 by 'billions and billions' is not directly attributable to the skills of the Autopilot.

I would love to see the camera feed, my guess is the police lights look like overexposed blobs against a black background and the Tesla can't figure out what to classify it as, so it's just ignored.

ultrasonics arent long range enough, radar has to throw out stationary objects as "probably a part of the road" and somehow no one has technology that can tell whether the path is clear ahead of them, which surprisingly was not the first step to shipping self driving cars

I would love to see the camera feed.

Me too. The NTSB should demand it. The NTSB has statutory authority to override trade secret law in an accident investigation.

Subject to subsections (b) and (g), the Board shall make public any part of a transcript, any written depiction of visual information obtained from a video recorder, or any still image obtained from a video recorder the Board decides is relevant to the accident— (A) if the Board holds a public hearing on the accident, at the time of the hearing; or (B) if the Board does not hold a public hearing, at the time a majority of the other factual reports on the accident are placed in the public docket.

It is curious and I would love to be listening in on the engineers at Tesla that investigate this crash.

Is the Tesla system getting confused by the emergency lights? The article mentions they were on.

Cops deliberately place their vehicles to protect other stopped vehicles. This tends to result in patrol cars partially in lanes or otherwise increasing exposure.

No need to fear everyone: I'm sure Tesla will let us all know shortly that it was the drivers fault. And the cop probably could have pulled over more safely, too. Nothing to see here.

How frequent is this compared to when humans only control the vehicle?

This is the right question to ask, but it’s less sensational.

There are roughly 40 - 50 fatalities for folks working on side of road a year?

"The tow truck driver was loading the vehicle onto the truck when he was struck by 23-year-old Michelle F. Parker, who was driving a Dodge Journey through the closed portion of Debarr Road at 3:07 a.m., police wrote.

The tow truck driver was brought to the hospital, where police said he died. No one else was injured and police said the tow truck driver would be identified once his family has been notified.

Parker was arrested and is facing charges of manslaughter, driving under the influence and driving in violation of license limitation."

gives an example.

More here: https://www.respondersafety.com/news/struck-by-incidents/yea...

Not sure how many of these are tesla's.

This is not the right question to ask because it's an apples to oranges comparison for many, but most importantly two reasons.

1. Automated driving right now happens in a tiny fraction of environments that are self-selected to be safe, if you were to put as many automated as real cars into absolutely the same environment as real cars the rate of incidents would obviously be substantially higher

2. automated errors are correlated. One bug in the model likely extends to every self-driving car in the same situation. Inherent reason why you cannot trivially compare the dangers of automated systems to people.

1. You need data about self-driving cars in all conditions. This doesn’t mean that “error rate of self-driving vs error rate of human drivers” is the wrong comparison.

2. This is irrelevant. What matters is the error frequency and severity. Correlations will increase frequency, but if that frequency and severity is lower than that of human drivers, it’s still a win.

it's not irrelevant because the number of cars on the road is so low. If we had tens of millions instead of a few hundred or thousand of these on the road, and you roll out an update that makes the car drive over small kids in green t-shirts then you have a big problem in a very short time. Correlated error is a big problem when combined with tail risk, which you simply cannot see right now due to how limited the use of the technology is.

Error rates account for these concerns. Error rate isn’t “the total number of errors”, it’s “errors per mile driven by all self-driving cars” (hence “rate”). Yes, an update might spike the number of errors, but as long as the error rate (accounting for severity) remains lower than for human drivers it’s still a net benefit. And moreover, no one is going to release an update to every car at once.

yeah the autopilot disengaged 0.000001 ms before impact. so it’s the drivers fault

Here come the tesla fans quoting stats about how safe tesla is.

I don't care how safe it is, its nowhere near as safe or reliable as the sales and marketing are pushing. I wish it was criminal how misleading tesla has been regarding reliability, and regarding how close they are to full self driving.

I do wonder if the FTC will open a probe into Tesla's self-driving claims. I'd be willing to believe that part of the sky high TSLA stock price is a mistaken belief that Tesla is actually close to self-driving.

> I don't care how safe it is

I guess it's polite to point out when you're unwilling to change your mind based on evidence, but that's a hell of a take.

> Here come the tesla fans quoting stats about how safe tesla is.

I'm not a Tesla fan.

How many accidents have been caused by automated drive systems and how do their failure rates compare to normal human-driven accident rates?

for better or worse the risk of 'random' luck the system failed means Tesla stats should be 10x or better than human driver.

A marginal improvement is not going to cut it.

> for better or worse the risk of 'random' luck the system failed means Tesla stats should be 10x or better than human driver.

Based on what?

> A marginal improvement is not going to cut it.

A 3% improvement is over 100k people not injured. I don't agree at all with your stance.

Well, this is what is revealed by a quick Google search:

> On average, there are 9.1 self-driving car accidents per million miles driven, while the same rate is 4.1 crashes per million miles for regular vehicles.

(https://www.natlawreview.com/article/dangers-driverless-cars, linking to https://carsurance.net/blog/self-driving-car-statistics/.)

Mind that most human accidents happen in environments which are considered off-limits for automated driving systems at the current state technology as deployed. (E.g., narrow urban streets in bad weather conditions and difficult, varying lighting.)

> Well, this is what is revealed by a quick Google search:

Sure, it might be. I'm not the one making claims challenging anything's validity, though, so it's not on me to provide any evidence for or against anything.

If you're going to make a post after a _single accident_ saying something isn't safe enough, you should be expected to define how you came to that conclusion.

In this case, though, it's just a general dislike for Tesla.

edit

I'll also add, your post is obviously anti-Tesla as well. From Tesla's website:

> In the 1st quarter, we registered one accident for every 4.19 million miles driven in which drivers had Autopilot engaged. For those driving without Autopilot but with our active safety features, we registered one accident for every 2.05 million miles driven. For those driving without Autopilot and without our active safety features, we registered one accident for every 978 thousand miles driven. By comparison, NHTSA’s most recent data shows that in the United States there is an automobile crash every 484,000 miles.

Which can be corroborated here:

> https://cdan.nhtsa.gov/SASStoredProcess/guest

These numbers are not directly comparable. Autopilot is only allowed to run in (relatively) low-complexity driving situations, and that's further restricted by a driver's judgment as to whether or not they feel it's safe to engage Autopilot.

What we really need is stats on the rate of human-driver accidents caused during driving situations where Autopilot could potentially be used. Then we could compare that number to the rate of Autopilot-related accidents.

For example, if a human driver causes an accident on a narrow, unmarked, unlit road in the middle of a snowstorm, you can't throw that data point into the "human driver accidents" bucket when comparing to Autopilot-caused accidents, because Autopilot would never be used in that situation.

This really exemplifies the problem with the current state of the discussion: Here's the result of an open minded search, it's a statistical statement across all self-driving automotive systems, independent of vendors, but, as the results are not pleasing, it's clearly anti-Tesla.

How should we be able to discuss these things?

Reply on news.ycombinator.com