Hacker News Reader

Comment by adamjcook | Hacker News Reader

In my book, Tesla gets a bad rap for providing an unvalidated, should-be safety-critical system to run-of-the-mill consumers without an accompanying Safety Management System.

The fact that they profit handsomely off this structurally dangerous wrongdoing is just the cherry on top.

And, without robustly maintaining a systems safety lifecycle (which, by necessity, must incorporate a Safety Management System)... no technical progress is quantifiable by anyone, including Tesla.

Tesla effectively throws a system over-the-wall and throws it all on the human driver and on the public.

View on HN

People keep using their Necessary Captial Words to say what we have is balderdash. Another post guffawes that there isn't Operational Design Domain.

I agree that where we are is balderdash, and dishonest, unclear about itself in extreme & lacking. But I detest this My Paradigm Is Required phrasing. Say what you think please! Browbeating the topic with particular/spefific engineering dogmatisms is unhelpful, unclear: it leans on authority, while also not having assertable claims anyone else could contest. This kind of hollow criticism degrades.

Nothing you said is wrong, I just really, truly don’t care. FSD makes fewer mistakes on routes I drive than it did a year ago, so whatever they’re doing seems to be working. Complaining that people like me shouldn’t be allowed to purchase safety critical software just deepens my resolve to keep using it.

> Nothing you said is wrong, I just really, truly don’t care. ... Complaining that people like me shouldn’t be allowed to purchase safety critical software just deepens my resolve to keep using it.

You do understand that people are concerned about setting these cars free on public roads, right, where they can kill unwilling participants? I don't think the concern here is about your freedom to choke on billionaire boots.

Regulators (NHTSA in the US specifically, but other countries as well) continue to allow it.

Elon is a bit of a monster (personal opinion), but regulators have the final say. When they force FSD to be pulled, then there is weight behind the argument, but this hasn’t happened and that sends signal.

You already share the road with inattentive drivers and drunks, so the risk acceptance/appetite benchmark has been set. FSD is arguably better than both cohorts, considering number of deaths caused.

As always, nuance. More people will die in the next ~10 minutes from traffic deaths than have ever been attributed to Tesla’s autopilot or FSD (33 total, as of this comment).

https://www.tesladeaths.com/

(Again, not a billionaire simp, just a rationalist; booo on Elon, but props to Tesla engineers in the aggregate; personally, I hope he gets blown out the door and JB Straubel takes over as CEO)

> You already share the road with inattentive drivers and drunks...

The second is a crime and I believe the first is a misdemeanor. Getting caught in either scenario repeatedly will cause you to lose your license.

So we may share the road with dangerous drivers, but we don't accept it. So it isn't grounds to accept more danger.

Really this line of argument is always wrong. The presence of danger should make you less comfortable accepting additional danger, not more. It's not like they cancel out, they sum. (One might say that mature and accessible self driving technology would take these drivers off the road, but the situation today is immature technology and high end vehicles.)

As a self described rationalist, I think you should take another look at that - to me, it reads like you're saying that because it doesn't feel like we're taking on additional marginal risk in comparison to the risks we've already taken on, we don't need to worry about how we're actually doing so, so I was caught a bit off guard when you said you were a rationalist.

Also, is it essentially the same "driver" in all those Teslas? If one driver was responsible for 33 deaths over a couple of years, they'd have lost their license long ago!

> FSD is arguably better than both cohorts, considering number of deaths caused.

I'm skeptical of any digital technology use case where the analogies/comparisons tend to be:

a) to non-digital things, when there are perfectly decent digital comparisons to be made (e.g., autopilot in the airline industry)

b) to conspicuous non-digital things, here drunk or inattentive humans

c) more or less a $small-human-scale-X improvement on the performance/efficiency/safety of the non-digital thing, often implying a future $unspecified-X improvement that, say, Moore's law would suggest (not saying you're doing the latter here, btw)

Those last two especially. I'm just imagining someone hawking a newfangled realtime audio system, claiming that it outperforms hand-punching a player-piano score. It's a silly example for sure; but on a regular basis on HN I read how Bitcoin is no worse than fiat in terms of global energy use, how crypto scam rates are roughly equivalent to wildcat banks in the old west (and hey, we eventually improved on those, so...), and how many more humans cause car deaths than these intractable systems which are "arguably" better than drunks.

> Regulators continue to allow it, and their opinion > internet randos.

This is true.

> When they force FSD to be pulled, then there is weight behind the argument.

Well, I suppose that it is pretty hard to dispute this, but it should be recognized that the NHTSA (the theoretical regulator of vehicle and highway safety in the US) is extremely weak and virtually non-existent.

The NHTSA lacks anything close to the skill sets necessary to independently, proactively and robustly scrutinize even rudimentary mechanical issues (which has been confirmed by several USDOT OIG reports over the years).

With opaque, complex automated systems and software... the NHTSA stands no chance.

The NHTSA lacks the internal skill sets to understand any of the comments that I have made elsewhere on this post.

Again, you are not wrong per se, but again, it should be recognized that the NHTSA is concerned primarily with establishing plausible deniability to protect the agency and with headlines rather than protecting the public with solid regulatory processes and oversight.

(Coincidentally enough, yet another USDOT OIG report was buried in a Friday afternoon release: https://www.autoblog.com/2023/06/02/nhtsa-fails-to-meet-inte.... I kid you not, every four years or so the USDOT OIG releases another critical report on the NHTSA that focused on issues not rectified in the previous report. It is like Groundhog Day.)

> You already share the road with inattentive drivers and drunks, so the risk acceptance benchmark has been set.

This is true.

Because the US public does not demand change and because roadway deaths are high, but distributed across time and space... the NHTSA remains weak and overall US transportation policy remains dreadfully poor.

> FSD is arguably better than both cohorts, considering number of deaths caused.

Unquantifiable.

There is no way to accurately and independently quantify the downstream safety impact of FSD Beta.

Sure, perhaps the NHTSA believes that (because they must given their structural issues), but we should recognize why such assumptions are flawed.

> More people will die in the next few minutes from traffic deaths than have ever been attributed to Tesla’s autopilot or FSD (33 total, as of this comment).

There is the possibility for "indirect" incidents caused by FSD Beta where the FSD Beta-active vehicle is never physically impacted.

We cannot assume that those do not exist.

And we also cannot assume that the media is able to pick up on every Tesla vehicle-related incident - even as well-followed as Tesla, the company, is.

In fact, other than the automaker's word, in many cases, safety investigators like those from the NTSB cannot independently and forensically establish specific root causes.

Yet they allow human beings on public roads killing unwilling participants. There seems to be a two tier system at play - unaided humans killing thousands upon thousands to the extent it’s normal vs humans using a tool incorrectly and killing … a handful?

My experience with FSD is it’s a terrible autonomous system and anyone who uses it as such is a fool, and a fool with a car is dangerous no matter what. However the joint probability of my driving awareness and skill and the cars combined is greater than mine alone. I’ve had it suddenly brake when a car I didn’t notice was drifting into my lane and had it not I would have been in an accident. Likewise it made mistakes and I took control.

I personally don’t care if it ever is able to take me from point A to point B without my attention or assistance. I value its ability to navigate with my assistance especially on long trips, reducing my overall fatigue and taking me through confusing sections of urban interstate without errors - when I always make a wrong turn. The fact it’s 360 aware and I’m not and it’s indefatigable and I’m not is valuable.

In the last year it’s become remarkably more capable. I don’t know if they can continue this rate of improvement but if they can it’s about as good as I would expect from todays technology on a consumer car. That’s a decent bar for me. I think it’s also something valuable on the roads - as a driver assistance tool. The folks who turn it on and get in the backseat would do something just as bone headed without FSD. Rather I notice enormous number of Tesla cars on the road not being drive by total idiots, and presumably quite a few using FSD without issue. And, as I assert above, I believe the joint probability of the aware driver with FSD having an accident is lower than either alone.

I don’t care what hyperbole a bipolar nut job spouts, but I do appreciate him setting an unreasonable goal and failing halfway there while the rest of the world seems content with stagnating. Tesla created the EV movement in the mainstream, SpaceX created the space revival we are experiencing.

Fwiw, I think the choking on billionaire boots comment is not a particularly high value contribution.

    I do appreciate him setting an unreasonable goal and failing halfway there while the rest of the world seems content with stagnating.

This is underrated. I write this a someone who is half appalled and half amazed by Musk. How much that Tesla has already achieved in their self-driving efforts is incredible. Leave aside Musk's vision, it also means they must have an absolutely stellar engineering team working on this problem. Creating and maintaining this team is a huge feat by itself.

> right, where they can kill unwilling participants?

I feel the same way about teenagers, grandmas, and grandpas, and yet here we are.

Which is why many countries have a 18 year limit for driving, plus in my country there are people pushing for mandatory regular driving tests for old people.

If FSD is statistically safer than the 18 year old (or new driver of any age), is it ethical to knowingly cause more death by forcing the new driver to drive instead of allowing them to use FSD?

You seem to forget that Grandpas have rights, for example to live their life, and. being able to get from A to B is part of that.

FSD does not have rights.

Once grandpa's vision, reaction time, etc. declines to the point he becomes unsafe he should be losing his license anyways. That's the law today, though it is not enforced as strictly as it should be. In my model grandpa maintains the ability to travel by using FSD and everyone is safer, in your model he loses his license and is stuck at home or more people die.

Those concerns are entirely unwarranted, boarding on hysteria. FSD has an adequate real world safety record.

FSD has no safety record as Tesla does not release their datasets for analysis by third partys, research institutes, or government regulators. In fact, they have deliberately misclassified FSD to not be a autonomous vehicle system, despite repeatedly indicating that it is intended to be and currently is a autonomous vehicle, to avoid mandatory California DMV reporting requirements [1]. The only "safety record" available is published by Tesla themselves with no access to the underlying data which is the entity with the greatest financial conflict of interest. You might as well ask Ford how safe the Pinto was or VW how clean their diesel was. There is literally no reason to believe those safety reports, in fact, you should probably anti-believe them as this is the standard pattern of manufacturers of unsafe/inadequate systems.

[1] https://www.dmv.ca.gov/portal/vehicle-industry-services/auto...

If Tesla had an adequate safety record, they would back their claims by offering liability insurance when self driving is enabled.

The fact that Tesla does not put their money where their mouth is says all you need to know.

Wake me up when someone is willing to take on liability. Until then automated driving does not exist.

FSD isn’t to the point where it can safely operate without a driver monitoring it. You’re arguing against allowing it on the roads with a safety driver on the basis that FSD isn’t ready to be operated without a safety driver. That makes zero sense.

FSD with a safety driver does not pose a threat to public safety. That’s not coming from me, it’s coming from the NHTSA.

What does FSD stand for?

I would have thought that something called Full Self Driving would be capable of fully driving by itself.

Sarcasm aside, that’s how most people use it.

The NHTSA hasn’t regulated it, but that doesn’t mean it’s an endorsement. It just means that they haven’t found anything within their legislative mandate that they can regulate here. US agencies are largely reactive, and I’m not sure I trust them with my life to get this right.

Most people use FSD as a driver assist because the software will permanently ban you from the beta program if you don’t keep your hands on the wheel.

No one who has used the software for more than a minute has any misconceptions about FSD.

> No one who has used the software for more than a minute has any misconceptions about FSD.

But people who have heard the name might. Honestly, it's a ridiculous name. Just "Self-driving" might be arguable. What are they going to call it when it's really fully self driving? "Literally Fully Self Driving Totally Serious"

I have no problem allowing it on the road. I do have a problem calling something that does not drive itself “self driving”. Much less FULL self driving.

Although, I guess my bigger issue is actually with the government that takes no action about false advertising.

"Full" can mean a lot of things. You chose to interpret it as L5, but to me it clearly compares itself to AP and other lesser ADAS systems which has a much smaller ODD. FSD has a nearly full ODD (it doesn't do parking and reversing yet). I understand you might get the impression that Musk means L5, and clearly that is the long term goal, but that doesn't mean FSD has to be L5. Anyway, arguing about a name is pointless. There are literally thousands of videos online where someone who's about to purchase a product for $15,000 can see exactly what they will get. The "fear" that grown adults will purchase something for $15,000 purely based on a vague name without reading any disclaimers or read/watch a single review is disingenuous and purely false concern.

"Adequate" is an interesting choice of a word when describing something's safety record.

Adequate to many is “Kills fewer people than the average human driver.”

And yet, the average driver kills 0 people. It’s in the far margins where deaths occur - a fraction of a fraction of a percent.

This is in comparison to the arguably singular entity “Tesla Autopilot” which has already killed several.

Tesla autopilot replaces a lot more than one human driver, so this is a mathematically nonsensical argument.

So does a bus driver. Or a pilot. Or an engineer.

And we hold them responsible for the humans in their care.

We don't. 42K people are killed in the US every year in automobile accidents. Yet humans are allowed to keep driving and killing people despite their massive flaws.

Individual humans have their license revoked and sometimes even face manslaughter charges if they cause too much death on the road. FSD is like one mind in control of an entire fleet. It's mistakes are amplified relative to a lone human

The problem is, you don't hear the counterargument from the folks for whom it didn't work, because they're dead.

Pretty sure you do - any indication that even regular auto-steer+ACC is engaged during a fatal incident is cause to sound the alarm bells and put out a headline for "Tesla Autopilot killed this person!". The first fatal accident involving FSD Beta is going to be a multi-week long charade of media attention and pressure on regulators to exclaim "despite this being 5x safer, you must gate this until it's literally infallible".

first?

https://www.tesladeaths.com/

> > The first fatal accident involving FSD Beta

> first?

According to your own citation, there are no deaths where FSD Beta has been alleged to be involved in any way.

So a Tesla driving off a cliff, is OK because it wasn't using the "FSD Beta" or whatever that idiot is marketing as the latest buzz?

Shouldn't the public be already outraging from the existing deaths? what's the difference... There will not be any change with an FSD death. As there wasn't with the previous deaths. Most people just don't care.

Is it okay, you ask. Yes! Yes, it is okay to be factually accurate. It's not okay to be factually inaccurate in order to reject evidence which contradicts your position.

> So a Tesla driving off a cliff

This is a lie. There is no evidence that any Tesla vehicle has ever driven itself off a cliff.

Perhaps you are thinking about media reports from a few months ago where a human drove their vehicle off a cliff? If so, it is concerning that you misremembered and/or misrepresented this news event in this way. This news event was of a human driving a vehicle off a cliff. That vehicle happened to be a Tesla — noteworthy only because experts were saying that the chassis did an astoundingly good job of protecting its occupants.

No use of driving aids was ever alleged.

> Shouldn't the public be already outraging from the existing deaths?

Yes. A million people are killed by vehicles driven by humans every year. A million people dead. Every year. This is an outrage.

As for deaths where use of Tesla Autopilot is indicated or alleged, I find them no more outrageous than deaths involving any manufacturer's brand name for lane centering/adaptive cruise control. These features do not lower the driver's responsibility to keep the vehicle under control. And they do not diminish the accountability of the driver in the event of a collision.

Isn't that an empty criteria? Doesn't the system disengage before an accident by design?

Tesla says “ we count any crash in which Autopilot was deactivated within 5 seconds before impact, and we count all crashes in which the incident alert indicated an airbag or other active restraint deployed” https://www.tesla.com/VehicleSafetyReport

But even without that, there are no allegations of such FSD beta related fatal or incapacitating incidents, who would say it even if the beta put the human in harm’s way 10, 15 seconds earlier. The black box in the car includes both the dash cam camera and whether auto steer is activated, so any fatal crash would still leave such evidence of Beta being active.

Every day thousands upon thousands of new teenagers start driving. One day, maybe today, the self driving technology will be safer than those teenagers and it will be a moral crime to put them in control of the vehicle. The same applies for the elderly.

Reply on news.ycombinator.com