Hacker News Reader

Comment by boulos | Hacker News Reader

Disclosure: I work at Waymo, but not on the Safety Research team.

The Ars article linked to the Waymo blog post [1], but the underlying paper is at [2] via waymo.com/safety . A lot of folks are assuming this wasn't corrected for location or surface streets, but all of the articles do attempt to mention that. (it's easier to miss in the Ars coverage, but it's there). The paper is naturally more thorough on this, but there's a simple diagram in the blog post, too.

[1] https://waymo.com/blog/2023/12/waymo-significantly-outperfor...

[2] https://assets.ctfassets.net/e6t5diu0txbw/54ngcIlGK4EZnUapYv...

View on HN

This is the first time I’ve seen an AV PR push that is remotely apples-to-apples. Props to y’all for that.

Do you know why Waymo didn’t include the dog they killed in this report? https://techcrunch.com/2023/06/06/a-waymo-self-driving-car-k...

I believe this study was about rider-only miles. The dog incident had a safety driver monitoring.

Sure but the news outlet headline is ‘only three minor incidents’ when in fact they killed a dog. It’s an act of not including all the information in a report about safety.

That's an issue with the headline; not Waymo. The Waymo blog post is clear they are comparing their robo-only-driver with human-only-drivers.

The Waymo blog post goes so far as to talk about human under-reporting of crashes, so it's only fair that Waymo call out their under-reporting of fatalities. It's suspect to claim the AI is better than the human when a notable fatality happened with a human behind the wheel of the exact same vehicle. Waymos have also hit people / bikers but magically the safety driver was always at fault.

There's good reasons to count robo-only miles. The other measure is confounded in both directions:

- Safety drivers prevent incidents

- Safety drivers cause incidents

Since the human+robo isn't the intended deployment mode, the only reason to count its numbers would be if there wasn't statistical power any other way. And that's no longer true.

A robo-only study is fine, but it’s misleading to hide that the human-supervised treatment has injuries and fatalities.

I’m also unsure how a safety driver affects the robo driving unless it’s an entirely separate program or the safety driver is pressing the “damn the dogs full speed ahead” button.

In my perhaps naive view a safety driver failing to act and a robo driver should be identical.

e.g. one of the most significant Waymo incidents in autonomous mode was a human driver falling asleep, bumping the gas pedal, and disengaging autonomy.

Comparing robo-only miles has the advantage of not considering incidents like this, and also not considering incidents where the safety driver prevented harm. It also prevents accusations of "you're claiming human Waymo drivers caused these accidents that you've omitted and are lying" because then they're counting 100% of miles driven without humans in the car.

Waymo calls out the under-reporting of human accidents and yet that’s exactly what they’re doing by ignoring their safety driver incidents that include a fatality.

What safety driver accident included a fatality? You're not obscuring talking about the dog with "a fatality", are you? Egads.

From looking at past Waymo data, you get similar numbers (a few more incidents, but a lot more miles driven) with safety drivers included. They're just noisier and more subject to dispute.

Exactly, the numbers are subject to dispute, so this is a press release and the whole dataset is relevant including the dog that was needlessly killed. Though I’m sure the e/acc crowd don’t count it as a “real” death even though the dog really died.

I've already pointed out why I believe the "robo only" numbers are more useful, and that the broader dataset says mainly the same thing.

I'm sorry, but a dog running out in front of a car is not a "traffic fatality" by any conventional sense of the world, and it's certainly not an at-fault accident for the driver.

At this point, I believe you are trolling.

I wasn’t contesting your opinion but rather the veracity of Waymo’s “study” which is really a press release. I and others have seen Waymos in collisions (e.g. in MTV) where Waymo chase vehicles showed up and acted as private security in order to shield the scene / liability. And then there’s Levandowski’s hack to the car and resulting crash that might have ended self-driving taxis today had there been reporting requirements https://www.salon.com/2018/10/16/googles-self-driving-cars-i...

At this point I believe you are trolling and disinterested in discussion that might be critical of Waymo’s brand and safety.

Keep in mind that you're complaining about excluding some incidents with humans drivers, but it's also excluding a whole lot of miles. That is, the denominator gets smaller.

We've previously had accident rates vs. total miles disclosed. Now, we have a new, better number (for reasons stated above). Of course, it's always good to look at a variety of measures.

> discussion that might be critical of Waymo’s brand and safety.

When someone uses the word "fatality" to obscure that they're talking about a not-at-fault accident where a dog runs in front of a vehicle and the dog dies, it convinces me that the person I'm talking to is not an honest counterparty in discussion.

I don't understand your point. Would it be better if it happened to a kid?

Points simplified:

- Most people reading "fatality" assume that it means a person died. This was not true of the incident. Repeatedly using the word "fatality" here is intentionally misleading. I have not heard the word "fatality" used in accident reporting to describe the death of an animal before.

- This was not an at-fault accident. It was not possible to avoid hitting the dog. Talking a lot about this incident doesn't provide very much information about Waymo's safety. (Of course, humans get in accidents that would be impossible for them to avoid, too-- all else being equal such accidents should be included on both sides of the analysis).

- The analysis didn't include this incident, because Waymo made a very rational choice to calculate using no-safety-driver miles and incidents. This isn't malice; it doesn't change the numbers much (because incidents and miles both go up when you include safety driver miles) and the numbers are higher quality and less confounded without including safety driver miles.

Waymo claims the dog was fully occluded by a parked car before it ran into the street, giving the vehicle and a safety driver at the wheel no time to react.

Not sure why you're focusing on this event. They're talking about miles with no safety driver.

Reply on news.ycombinator.com