Hacker News Reader

Comment by sonofhans | Hacker News Reader

“Safer” == “Safer than all other human drivers in the same city.” By their own admission, this is not a straightforward comparison. If they could do the math for the same routes, times of day, and conditions … maybe I’d believe it. Otherwise, this data is trivial to cherrypick, and they have every reason to present it as well as possible.

I believe Waymos are pretty safe, and that’s a great thing. “Safer than humans (for selected rides inside this area)” is still very good, but it’s not at all “Safer than humans (period).”

View on HN

In essentially all cases where a Waymo and a human-driven car have collided, the human driver has clearly been at fault. This seems definitive and not susceptible to cherry picking.

That could just be, and seems to be in some cases at least, because Waymo doesn't behave like a human would, and people gets tripped up.

I don't doubt Waymos are very safe, but I always irk at these comparisons. Majority of human accidents are due to gross negligence and/or driving under some influence or serious fatigue. A system incapable of alcohol etc. is better than that? Well that is a substantially lower bar than you can possibly imagine. Add to that that all systems have constraints on how and where they are able to go. Combined even Tesla can be made to look good.

Depending on the context and question it might still be the question to pose. But people often make the leap to assume that a typical Waymo is x better than a typical human driver which is an entirely different question entirely.

Waymo is for sure one of the (if not the only) good players out there though, gives me some hope.

> could just be, and seems to be in some cases at least, because Waymo doesn't behave like a human would, and people gets tripped up

Driving conventions vary wildly across states and even within them. And foreign drivers are a thing. A human who gets tripped up by a Waymo acting unusually will also get confused by someone getting used to no turns on right in Manhattan, driving on the right side of the road if coming in from the Commonwealth or adapting from California's protected left turners can turn into any lane, not just the leftmost. They'll also get confused by children and pets, who aren't bound by social custom, and deer, who aren't bound by physics.

... no? That children, animals etc. acts differently everyone knows. But what about a self-driving car that looks the same as every other car?

Anyway, it was Waymos own findings when they started out. They got into more accidents, none of which where their own fault, than expected and realized that they had to make it behave more like a human to not confuse human drivers.

> ... no?

Which part are you confused about?

> what about a self-driving car that looks the same as every other car?

My Aussie friends, when they visit America, don't put up a sign in their window saying they're Aussie and will occasionally try to turn left at a red.

> They got into more accidents, none of which where their own fault, than expected and realized that they had to make it behave more like a human to not confuse human drivers

Sure. I'm still filing this in the nothingburger file. If anything, it screams that we have a lot of people on the road who should not be.

Nothing.

Yeah? I thought that was a given...

I think the other way to think about it is that Waymo is probably in the 99th percentile for not being distracted and 99th percentile for reaction time, always, just from a pure sensor and computation standpoint.

Even the best human can't say that, at all times?

A year or two ago, would have agreed with you. I used to think they were cherry picking as well.

But Waymos have driven so many miles by this point, if they are hiding some data that would tip the scales back towards human drivers I have yet to see it. If there is a way to slice the data that makes Waymo’s look less safe I would welcome the correction.

If Waymo truly has 80-90% fewer crashes in the conditions they drove in, then it still has policy implications for places like Phoenix that do have good conditions.

I've heard that Waymo relies on having very accurate map data for the areas where they operate, so perhaps they could perform worse than human drivers in areas where they don't have good map data.

But I also trust that the company wouldn't deploy them in those areas until the quality data they need is available. So perhaps "safer in the environments where they are actually deployed" would be more accurate, but that's also the only thing that matters.

Speculating about what would happen if they were used in ways they are neither intended to be used nor are actually used feels a little silly. Most machines can be unsafe if you use them in ways they're not intended to be used.

I, and I expect most human drivers, also probably drive measurably worse if I'm in a city I'm not familiar with.

> By their own admission, this is not a straightforward comparison.

If they wanted to cherry pick, would they not omit that admission?

In any case, it seems plausible to me that the routes that Waymo drives are above average in human incidents, given that Waymo is probably overrepresented in high stress/traffic, inner city scenarios.

The comparison gets picked up as the headline; the admission does not. This is exploited quite often, e.g. in science reporting. I'm not saying this is what Waymo did - they don't seem to be bad actors - but absolutely, the pattern does occur.

If you were choosing between getting into a Waymo or a car driven by a human driver (where Waymo operates, for a route that Waymo would do), the data shows that the Waymo is safer.

No, it does not. For one thing, we don’t have access to all the data, just what’s being told us. For another, it at best shows that Waymo is safer than average. Safer than an attentive London Cabby? I bet not.

I know that anecdotal experience is definitionally just that, anecdotal. But I've had a handful of attentive London cabby experiences (and enough in-Waymo experiences) that give me conviction that Waymos are far safer than them. They're out there driving all day every day, it's obvious to me that a Waymo driver is going to be safer than even a professional.

One cabby pulled out of a t junction to end up alongside me on a motorbike – a Waymo would never do that.

> For one thing, we don’t have access to all the data

In the US, we do have access to all the data [1]. They're required to report every incident with an injury or any amount of property damage, and it's all available for download as CSV.

> For another, it at best shows that Waymo is safer than average.

No, it shows that Waymo is 6 to 12x safer than average.

[1] https://www.nhtsa.gov/laws-regulations/standing-general-orde...

London cabbys can be hangry or have broken up with their partners that day. What makes you so sure they're fully attentive on any given day?

https://waymo.com/safety/impact/#methodology

Worth reviewing the methodology, rather than making stuff up.

Perhaps the comparison should only be to other taxis. Since I cannot buy a Waymo, it is not really relevant whether it is better than an average driver (including all the drunk ones, and the speed racers, etc).

There are other safety differences with human-driven vehicles… interpersonal violence does happen with taxis (e.g. drivers sexually harassing/assaulting passengers, passengers robbing drivers) - by definition those things cannot happen with a Waymo

I appreciate the skepticism. While I suspect motor vehicles that cannot be distracted would be safer than motor vehicles that can be, it shouldn't be claimed without real evidence.

If I were Google, I'd partner with some insurance carriers to compare the number of claim events normalized to the number of drivers on the road (approximated with Android data) in a city (same time of year, etc) before and after introducing Waymo. If claims per driver decreases, then I would be more inclined to support the claim that they're actually safer and that they don't just "seem safer"

accidents are not equally distributed across humans. more serious accidents will be caused by people who are habitually doing things that are unsafe but, for various reasons, most places lack effective ways to stop these people driving so they keep causing accidents.

the metric is not some nebulous aspect of skill but the bottom decile of human drivers causing accidents. it is not difficult to believe that an AI can drive better than this group, it is not a high bar, below the 10th percentile are people who should not be driving but cause most of the accidents.

Reply on news.ycombinator.com