Hacker News Reader

Comment by rco8786 | Hacker News Reader

I don't know anything about this specific case but this exact thing will be a major barrier for autonomous vehicles to overcome. Instead of benchmarking AVs against the current, human-caused number of vehicular deaths - people will naturally benchmark AVs to 0 and treat every instance of death as a failure of AVs, even if the reality is that AVs end up 99% safer than human controlled vehicles.

View on HN

As it should be. “Safer on average” is not a defense for failing something as basic as stopping at a stop sign. The bar for not running one and killing someone is not unreasonably high.

I agree; that’s why I think we need a moratorium on human drivers while we figure this all out.

This also solves the rising oil and gas prices issue as well!

We do tend to impose them on individual drivers who make egregious and consequential failures, that’s the point. If every car running the same software is functionally the same ‘driver’ this class of failure absolutely should trigger a moratorium.

I think this is a fantastic distinction to make, and one I haven’t seen before.

>We do tend to impose them on individual drivers who make egregious and consequential failures, that’s the point.

Do we? It's a common saying that if you want to kill someone with no penalties, have them run over with a car.

Then there are examples like these all over the place. Not to mention distracted driving and DUIs don't even get your license suspended in many or most jurisdictions.

https://www.seattletimes.com/seattle-news/law-justice/teen-w...

>The Audi, which Jones had bought one month before the crash, was the third car he had totaled in a crash within 11 months. Speed was a factor in all three collisions, but police did not cite Jones in the first two crashes, prosecutors said.

Also see the 80-year-old woman in SF who killed an entire family at a bus stop, while speeding, while going the wrong way on a one way street, and was sentenced to probation and community service and banned from driving for three years.

What is this Tesla (or its non-driver) going to get?

more retail investors buying the stock

Humans fail to stop at stop signs all the time. Their failure to stop kill people as well.

And we take away people's ability to drive when they kill people.

Often we do not. The Freakonomics podcast had an episode titled The Perfect Crime where they basically say if you want to kill somebody, hit them with your car because that’s often not punished beyond a ticket.

This trial just happened where there was no punishment:

https://www.macombgov.org/node/55/news/former-warren-police-...

>According to evidence presented from law enforcement at trial, on September 30, 2024, Burke was operating his patrol vehicle at a high rate of speed without emergency lights or siren activated when it crashed with another vehicle. Burke's partner was injured in the crash, and 34-year-old Cedric Hayden Jr. and 33-year-old Dejuan Pettis were killed.

Then there was this one a couple years ago:

https://en.wikipedia.org/wiki/Killing_of_Jaahnavi_Kandula

https://www.theriolindanews.com/2025/04/22/529861/license-to...

https://calmatters.org/investigation/2025/04/license-to-kill...

Not sure why you are getting downvoted here though yes, the system is not perfect, in some states more so than in others. it is a fine line between giving someone who deserves it a 2nd chance vs. someone with 17 priors. but guessing you would also agree that allowing "self-driving" tech to kill people on the road without any repercussions isn't all that ideal either?

It’s in the US. The “no punishment” might be related to the offender being a policeman. Amazing it went to trial, to be honest.

You can't compare autonomous systems that drive hundreds of thousands of vehicles to a single person. You have to compare it to hundreds of thousands of people.

American drivers kill ~100 people every day behind the wheel of a car. If autonomy were 50% safer and was deployed at such scale that it replaced 50% of all hours driven it would save 25 lives every day. It would also kill 25 people every day.

Sadly, many people will clutch their pearls at inevitable headlines like Bloodthirsty Elon Sacrifices 25 Innocents on the Altar of Profit Every Day! and, their emotions sufficiently primed, they will happily pull the plug on autonomy, sending 25 people per day to their deaths and feeling great about themselves as they do it.

Ok, in my country, you actually get your license removed and jail time if you kill someone (usually 2 years, up to 10 if you're drinking).

Let's say Tesla kill 50% less than a human driver, I'd agree to give Tesla's board 50% of the sentence for each person autopilot kills. As an incentive to make the cars kill less.

Let's reformulate it as a railroad problem. In the proposed hypothetical, if you effectively ban self driving cars until they're literally flawless, 50 people per day will be killed by human drivers every day. If you pull the lever and allow imperfect-but-better-than-human self-driving, 25 people per day will be killed by by autonomous vehicles.

As a reference point, non-suicide gun-related deaths total ~45 people per day.

What moral principle could possibly justify not pulling the lever?

But, even as everyone talk about morality as an ideal, no one really cares about it (else the world would be very, very different). What most people care about is feeling that things are 'fair', at least the appearance of fairness. I guarantee you that if a company accepts that it's board (or executives) get jailed if their car makes a mistake that kills someone, like any random driver, people will accept it.

It should be a critical part of the evaluation. If you set the bar higher than “safer on average” then you’re going to delay the switch, and every month of delay will cost more lives. “Safer on average” is exactly when you should start using the new thing, even if it’s uncomfortable and even if it’s a low bar.

No it shouldn’t. Low risk behaviors should equal low risk outcomes. If it’s safer on average yet causes low risk behaviors to suddenly become unpredictably dangerous, folks will very reasonably push back

Sorry, I’m not following. What are the low risk behaviors you’re referring to and what makes them suddenly become unpredictably dangerous?

Walking / driving across an intersection where the cross traffic has a stop sign?

That's unpredictably dangerous today, with human drivers.

It should be a defense for FSD to exist though, right? Net more alive people.

No, because you can punish bad drivers, remove them from the driving pool, put them in jail, or in some way achieve justice (this may not always happen, but at least it's posible). There is no way for FSD to have any accountability, and so no justice can be served. "Sorry, your son was just unlucky that he was killed by the AI, but we can't do anything about it" is an intolerable breakdown of justice. Same problem we have with qualified immunity.

You can update software though, you can’t update a negligent driver.

But you can take their license, fine them, jail them.

Corporations just pay a fine and move on. I don’t want that to be the standard for road accidents and frankly outside of this techie niche no one does.

"Sorry, the driver could have turned on FSD, but decided to drive [drunk/tired/distracted] and killed your son. Thankfully we can ban them from driving for a while."

This could play out a million times at the population level. If you can decrease the chances of needing justice in the first place, shouldn't that be considered?

What you're talking about is who to blame, not how to keep more people alive.

“Who to blame” turns out to be one of the most important facets of humanity, and it turns out we structure our entire society around it: assigning liability and pursuing justice. That’s not something that can be handwaved away to technocratically chase a metric (traffic deaths).

A person needs to die for us to assign liability and pursue justice.

If fewer people die, there is no liability to assign.

It sounds like you're arguing for a human being to die to preserve a human being to blame. Which yeah, sounds just like us! "This is the way we've always done it!"

If fewer people die, there is only less liability to assign. Remember that the original argument was "FSD needs to have 0 fatalities before we adopt it". And it's not valid to argue that some traffic fatalities are unavoidable; I know people who work at the DOT and "Vision Zero"[0] is a thing, and already implemented in some European countries/cities. Without FSD. So FSD needs to clear this bar.

[0] https://visionzeronetwork.org/about/what-is-vision-zero/

> It sounds like you're arguing for a human being to die to preserve a human being to blame.

God I hate technocracy. There’s a reason we elect politicians and not technologists who think everything is simple and we just aren’t making the technically correct policy decisions.

“I’m sorry ma’am, we won’t be investigating anyone for your daughter’s death at the hands of the Sexy Deathmobile 5000 because technically no one is liable. But just think, there would have been so many other children killed this year if we didn’t allow these Sexy Deathmobiles to take over the roads! I hear there’s even a software update that will reduce toddler deaths by 5% at the expense of a 10% rise in dog and raccoon deaths, but it’s a choice our technocratic overlords are ecstatic to implement!”

You’re also arguing against a false alternative, which is the status quo. I’d rather have our money put towards safe streets and reducing demand for toddler-bulldozers (cars with raised grills) than a FSD pipe dream that would only enrich a few billionaires and the lucky SWEs who happen to pick the correct company to work for.

Safer than average is totally defensible, because every year the margin of what is "average" changes.

I'm just pulling numbers out of my ass here, but if say, one person is killed by an automobile every 100m miles, if the next year the average is 0.999 per 100m miles, that's a good thing provided the trend continues. We should not step over a dime to pick up a penny.

I do think there's a point where introducing different failure modes is harmful. Say they end up causing more pedestrian crashes and more head-on & angle crashes (such as the one in this article), but less loss-of-control and fixed-object crashes. That would make it safer for occupants but less safe for non-occupants of the self-driving vehicle. Since the occupant is the ones engaging self-driving mode they should be the one carrying the risk, so IMO even if it reduces number of deaths overall it should not be allowed

Imho part of isuse here is that you cannot "partial AV" it.

You cannot have a car that will EVER need you to intervene or else it does not work because of awful human nature. The amount of tesla uber drivers ive had literally browsing tiktok, watching youtube UNTIL the car beeps that it is a green light detected, then they immediately hit the accelerator before looking up is ridiculous. You cannot have a half ass "autopilot" driving experience where you are expected to zone out and then be expected to take control within a millisecond during uncertainty. It does not help that FSD, AP, aAP, bAP, the model, whether the car is visual only all apply differently

Another huge problem is that error rate it not error shape. We have an enormous body of knowledge and process for guessing what might cause problems for human drivers, how to deter the problems, and how to mitigate the damage.

For example, you'll have a pretty good idea how our fellow humans will react to rumble-strips if I describe them or play a recording. I don't need to consult a datasheet or design an AI test suite with a zillion simulations and a bunch of closed-course tests.

> I don't need to consult a datasheet or design an AI test suite with a zillion simulations and a bunch of closed-course tests.

Traffic engineers have been doing this for decades though?

What percentage of road accidents involve broken down old cars, drugs and alcohol, irresponsible and risky behavior?

Because most of us don’t have that problem, and insurance stats suggest we’re a hell of a lot safer drivers than a hypothetical aggregate including stolen car joyrides by teens and 50 year old teetotalers in a well maintained, modern car.

Teslas do very badly when benchmarked against human caused deaths in similar (luxury) vehicles.

Safer than the highest level numbers isn't really a win though for autonomous driving. All human caused deaths includes drunk drivers, inexperienced drivers, and elderly drivers. An autonomous system should never be any of those things, so comparing them against is disingenuous at best and outright manipulative at worst.

That's extremely silly. drunk, inexperienced, and elderly drivers are all human drivers. You can't just exclude them from your baseline because they are not in perfect driving form.

Let's also exclude young drivers, stupid drivers, distracted drivers, drivers who never took driver's education classes, short drivers, barefoot drivers, glasses-wearing drivers who forgot their glasses, and one handed drivers.

The insane variation of human drivers is exactly why AVs should ultimately bring vehicular deaths down..which is the goal.

You absolutely can and should exclude the known bad drivers from the stats because they are essentially outliers. If you don't, it's very possible that widespread adoption of AVs actually makes the average driver more dangerous.

Reply on news.ycombinator.com