San Francisco raises Tesla 'self-driving' safety concerns as public test nears

This isn't true, and there's any number of YouTube videos demonstrating that. At the very least, it knows how to navigate highway interchanges, how to change lanes, how to avoid other cars that would otherwise collide with it, and to avoid/slow down for pedestrians and cyclists.

The last feature is perhaps overly sensitive in my personal experience. It used to slow down dramatically whenever it was passing a cyclist, that has improved in recent versions.

Sadly, only half-joking here.

The danger in this case is entirely manufactured. Level 2 driving assistance features are safe when they are not oversold.

Self driving cars are a nice luxury, especially in city driving, not something that radically improves our world. You get to read your phone instead of paying attention, and the trade-off is someone might get killed. It's like treating a rash with a drug that could give you a heart attack. That's a far cry from, "with this technology something that took days and $$$$ now takes hours and $" as was the case with all the older examples you listed.

If self driving cars were more like airplanes, I'd have a little more faith. Tesla's marketing BS doesn't inspire me with lots of faith.

On black boxes: https://health.clevelandclinic.org/what-does-it-mean-if-my-m...

That's a strawman. Just make the company fully liable for any injuries or deaths from accidents involving their cars. The market will figure it out from there. But that's not how it's being handled at all.

Life expectancy for a child. A white person reaching the age of twenty would have an average life expectancy into their 60s. Which is still lower than today, of course, but I'm not convinced that the difference supports a conclusion about differing attitudes toward risk.

> not honest mistakes in the code

It's not mistakes in the code, but oversight in the design that worries me.

I think one of the risks of the current paradigm is the inability of US government to effectively introduce new regulation legislature. The new method seems to be to put the onus on the industry to regulate themselves in lieu of having to create actual legislation, which can lead to perverse incentives. In the instances where this isn’t the case, we have to deal with regulatory capture or the revolving door of industry/government which creates its own skewed incentives.

I don't think there is any risk of stagnation. Maybe we should work on why we need cars in the first place. We can simultaneously work on public transport, or virtual presence, etc.

But Tesla offer their own insurance. Isn't that taking responsibility?

You are mixing up autopilot on highway and FSD. And even then the numbers are comparing very easy stretches of highway driving where people feel comfortable activating autopilot to all driving miles.

Watch the videos on youtube for FSD beta. On most of them it does something or some things very wrong every couple miles and is saved by the hypervigilant driver. There is no way FSD is safer than human drivers right now. I could believe what you say is true about Waymo. But they are being much more responsible and it seems like SF doesn't have a problem with them.

I think this might not be true! I looked at the numbers a few years ago, and Uber had a much higher fatality rate per mile driven then human US drivers. Waymo was better; I forget if it was better than human or not. I couldn't find comparable Tesla numbers --- only PR-style numbers that could be based on all sorts of weird definitions --- but I don't get the impression that they're at the head of the pack.

EDIT: The number to beat is around ~15 deaths per billion miles driven, on the road in the US.

https://en.wikipedia.org/wiki/Transportation_safety_in_the_U...

For me to accept Tesla FSD it should be several orders of magnitude better than a human driver. I don't want an 'average driver' driving me to work. The average driver is involved in 6 million car accidents per-year.

Interestingly even that's a bit spun. There was one recent accident, in August. And the NTSB going back discovered that there's a cluster of (I think) 11 others where the car behaved similarly and appeared to strike an undetected emergency vehicle without trying to avoid.

Now... that's interesting, and potentially represents a bug worth fixing. But it was also rapidly pointed out that (1) this is an extremely common kind of accident (human drivers hit these things all the time too) and (2) given the number of reported Tesla AP miles driven and assuming no other unreported collisions cases, Autopilot is actually about twice as safe as the average US driver vs. stopped emergency vehicles.

There’s no evidence this is true and what evidence exists is more suggestive of the opposite.

So far you've only heard this from Elon Musk, who is a serial liar and hype machine. Wait until you've heard it from a trustworthy third party with access to internal Tesla data.

Just from the total number of accidents, I don't see any way for FSD to be safer than human drivers. It's likely not to even be within two orders of magnitude. Human-caused fatalities are in the order of a billion miles driven, while Tesla seems to be hundreds of thousands or millions.

I knew a girl (in the sf bay area) who hated self driving cars. She said people should just learn to drive better. While at the same time, you go to the DMV and people are on their phones in the testing area getting the test answers. Anybody with a CA drivers license should be audited by an out of state driving agency

Most of the SF folks you are describing lean toward the pro-vaccine-mandate position. They should therefore also be for self driving cars.

They haven't released a detailed deep dive on their statistics, other than some definitions of what counts as a crash. This is from the company headed by the "funding secured" guy. Elon is clearly willing to lie if he believes that it will serve his interests or mission.

Even if we accept their safety stats at face value, Tesla is getting ready to launch a new feature - "Full Self Driving" - which behaves substantially different from autopilot and has a much different operating design domain and intended functionality. They've had close to 2k internal testers and O(100) external testers of "Full Self Driving" for about a year - nowhere near enough time and data to get the same level of confidence they have for autopilot. Add on to this that it will operate in environments that are substantially more accident prone than highway driving or clearly marked single lane roads, and you can see why regulators might be worried by a larger rollout.

This is the crux of the argument. However, it falls flat when you consider that many other simple "can reduce that number" measures have not been adopted – like fitting speed governors to cars limiting them to some speed limit, putting tracking beacons in cars to making them automatically stop at red lights, or putting in devices in cars to limit drunk driving.

I don't see a reason why a much more complicated technology should leapfrog these simpler ones and be put into production so quickly, potentially risking innocent lives in the process. In fact, I believe the reason why it is being rushed so quickly is arguments like yours coupled with a low understanding of the technology involved. People are viscerally more willing to accept the idea that an "artificial intelligence" is going to drive the car, rather than the idea that their car is a multi-ton hunk of metal that needs to be limited and monitored carefully.

> I honestly think the biggest issue is probably the legal a s regulatory overhead of responding to the technology rather than the technology itself, but since it’s so hard to quantify I don’t often bring this up. But the biggest passion I have seen if from those same engineering minds, who truly understand the complexity, but also want to see it happen. Their timeframes are just long. I’ve heard 2045 and 2050.

Those timeframes seem normal to me. Rollout dates should not be decided by the whims of corporate shareholders. Let's put simple speed limiters on cars first, everyone can understand that technology – and see how that goes!

My understanding is that those companies had trained test drivers, but Telsa is pushing this out as a beta feature update and allowing the consumers be their test drivers?

It's not about emotions; it's about statistics.

The problem that needs to be overcome is not the emotions of people losing loved ones, but the difficulty of accurately estimating safety by test-driving autonomous cars.

The following is from a Rand Corporation study titled: "Driving to Safety"; subtitle "How Many Miles of Driving Would It Take to Demonstrate Autonomous Vehicle Reliability?"

Key Findings

Autonomous vehicles would have to be driven hundreds of millions of miles and sometimes hundreds of billions of miles to demonstrate their reliability in terms of fatalities and injuries.

Under even aggressive testing assumptions, existing fleets would take tens and sometimes hundreds of years to drive these miles — an impossible proposition if the aim is to demonstrate their performance prior to releasing them on the roads for consumer use.

Therefore, at least for fatalities and injuries, test-driving alone cannot provide sufficient evidence for demonstrating autonomous vehicle safety.

Developers of this technology and third-party testers will need to develop innovative methods of demonstrating safety and reliability.

Even with these methods, it may not be possible to establish with certainty the safety of autonomous vehicles. Uncertainty will remain.

In parallel to developing new testing methods, it is imperative to develop adaptive regulations that are designed from the outset to evolve with the technology so that society can better harness the benefits and manage the risks of these rapidly evolving and potentially transformative technologies.

https://www.rand.org/pubs/research_reports/RR1478.html

Bottom line, Tesla's (and everybody else's) "safety record" isn't and it will take a lot more time to know whether self-driving cars are as safe as human-driven cars. The stats you quote are not sufficiently informative.

This is like the old shitty political saying “Something must be done! And this is something, therefore this must be done!” It sucks that driving is so dangerous. That doesn’t mean letting tesla do whatever they want will help improve things. As others have commented, the data on tesla’s safety is apples-to-oranges. It doesn’t mean much of anything.

On a side note, legally if a Full Self-Driving system kills someone ... who gets sued? The manufacturer? Is Tesla going to tank every single lawsuit from its Full Self-Driving system injuring/killing someone?

Another aspect that is important but easy to miss - deaths caused by an autonomous vehicle are learning experiences - once a couple have happened there will be an engineering response and they will stop happening.

Compare that to ordinary drivers, where each death is just a pointless waste of a life. There is an argument in favour of putting these cars on the road even if they were more dangerous than humans. The faster we push through the painful learning experience the sooner the payoffs of superhuman driving will start being felt in the death statistics.

Each of those companies, and Waymo, etc., all agreed to accept liability for any damages from their efforts.

Tesla is sticking with "the driver is responsible, not us". So yeah, that's why there's a discrepancy.

Currently AP's "self-driving" features cannot save a life, they have killed people.

To understand that better, examine AP in finer detail:

Teslas have safety features that can intervene when the current situation seems to be leading to an accident. Let's call those correction. Those can (and have) save lives.

Tesla also has features that are supposed to be more convenient for the driver. The claim is they take a load off for the driver. Call those the convenience features.

-

The convenience features have not evolved to the point where you are ever allowed to be less vigilant than you would be otherwise! By definition if AP is driving, you too must be fully attentive and "driving" hence the increasingly intrusive detection which has now evolved to detecting wheel input every 15 seconds...

Now realize that while for the sake of examination I'm describing convenience vs correction, these are all systems in one car. You can get the correction features without FSD, but you can't get the inverse.

Because of this, the convenience features alone cannot avoid an accident that the avoidance systems would not have.

-

This is tricky to think about, but consider a driver who would have drifted off the road without AP. As soon as the driver deviates too far from what AP would have done and starts touching a lane marker, the lane detection systems realize there's an issue and correct.

However, say there's a tricky lane marking where the correction system wrongly believes the driver input is wrong. They get an alert and an attempt at correcting steering inputs.

But because the driver is already inputting a direction with full intention, this wrong input is already actively being overridden by the human who plotted their turn and can clearly sense that they should fight the suggested input before they end up running off the road.

With the convenience features enabled, the driver's input will come after the wrong action has already been taken!

If the avoidance systems knew that the convenience feature was about to send you into a jersey barrier... the convenience feature wouldn't be "allowed" to send you into a jersey barrier!

-

I want to see studies that focus on the safety of driving with full driving aids but without convenience features. Claims that you are less tired from using them show that it's an inherently unsafe practice since you should be exactly as vigilant as normal.

Because of what's explained above, the expectation would be that there is no drop in safety if your car only corrects when there is a hazard... but there is a drop in safety when you are falsely lulled into a sense of safety.

To me WIP self-driving should be monitored by trained people. It shouldn't be a box you get to tick in your infotainment screen.

If Tesla wants to carefully pick who's allowed to run these betas more power to them, but the number of idiotic videos I can pull up right now of people intentionally testing FSD updates in situations they know are hairy and risking other people's safety is insane.

That is just setting up self-driving to be dead on arrival.

But when Tesla cars were then compared to similarly priced cars; that record dropped through the floor.

"Tesla autopilot saves lives" is a bold claim.

I really don't get how regulators are OK with the yoke steering in a car! To me it says that Tesla really doesn't prioritize safety.

More than anything else it’s marketing hype, and I agree calling it full self driving when it has documented flaws is disingenuous. But it’s not vaporware.

Please. You can go online and buy a FSD beta car right now. Countless videos on YouTube of the product working or being improved upon release-to-release.

There have been 2K testers for a year w/o any accidents, I would much rather my life be "at risk" by Autopilot than a drunk/tired/distracted human driver.

Nobody disputes that it can drive fine most of the time on easy roads.

> I don't think we understand the human factors involved

Do we? Because human factors responsible for literally every traffic accident in history, and those don't seem as scary to you. Basically: it's fear. It's just fear. And it's reaching a peak right now (and being deliberately pushed, frankly) just as Tesla is reaching a public rollout.

And why? Because there's another fear at work here too. If this rolls out and doesn't kill people. If it turns out that it actually is safe, then a lot of people across a lot of industries stand to lose a lot of money. Stories like this are, fundamentally, just marketing press hits. The public FSD beta rollout in some sense marks the last chance to kill Tesla's autonomy product before... it wins.

The idea of even the most modern intelligence system navigating through an environment as elaborate and dynamic as New York City or downtown Chicago should give anyone pause. I can see them being more useful for long distance journeys such as hauling a trailer across rural desert areas where all the vehicle needs to do is stay in its own lane, maintain a set speed and brake if something obvious gets in front of it.

If you have no problem with humans driving by hoping they find the right pixels on their retinas, why would you worry about AI doing the same? If anything, AI has a perfect reaction, is never tired or sleepy or unattentive. It just needs to be taught properly.

What you are describing is actually probably the end game - but I think musk and others have correctly identified that if you wait for infrastructure to be ready then you will be waiting a long time, and if you want to see self driving cars anytime soon you have to assume current infra.