Comment by phsource | Hacker News Reader

Comment by phsource | Hacker News Reader

It seems like Waymo's hitting a point of diminishing returns. The past 4 years' worth of disengagements per 1,000 miles driven were:

- 2014 (last 3 months): 1.27 [1]

- 2015: 0.80 [2]

- 2016: 0.20 [2]

- 2017: 0.18 [3]

There are two possible causes off the top of my mind:

1. Waymo's cars are being tested in more demanding environments.

2. There's a floor to the number of disengagements due to technology and road conditions: situations that are different/unique enough that no model could accurately detect/respond to it.

Case 2 would be definitely be a problem for the full-autonomous (i.e. zero user-input) type of car. I'd love to hear other hypotheses/feedback from those closer to this kind of work!

Also, another interesting note: total miles driven on public roads is actually down from 2016 (352k vs. 636k earlier). Perhaps Waymo's finding the simulated/private-road tests more fruitful for winnowing down the final edge cases.

[1] https://www.dmv.ca.gov/portal/wcm/connect/dff67186-70dd-4042...

[2] https://www.dmv.ca.gov/portal/wcm/connect/946b3502-c959-4e3b...

[3] https://www.dmv.ca.gov/portal/wcm/connect/42aff875-7ab1-4115...

Waymo is doing a lot more testing outside California now. They've done Mountain View so thoroughly they were seeing the same situations over and over. Last year, they were rear-ended twice at the same intersection.

One disengagement every 5000 miles is a problem. It's too infrequent for humans to remain attentive, and too frequent to ignore.

The “Autopilot Valley”.

I think the devil's in the details a bit here. For example, for the first 251k miles driven in 2017 (71% of the total) they had 1 disengagement due to a "software discrepancy."

Then over the next 43k miles they had 8 disengagements due to software.

And then none since October.

Looks like a regression bug to me.

Similarly with "perception discrepancies" they had 14 disengagements for the first 175k miles driven (50%) and then 2 over the second half.

Also only 24 of the 63 incidents were "failure detections" where the vehicle immediately handed over control of the vehicle; the rest were handed over "safely" (I assume this means the driver took over control voluntarily.)

Also they don't say what the criteria is, but I suspect given what we've heard about the cars' conservative natures that "disengage for unwanted maneuver of the vehicle" is code for "the car is going too slow / being too passive at an intersection, etc."

So all in all I see positive trends above and beyond the overall numbers. But I'm bullishly biased about self-driving cars, so get out your salt grains.

Reply on news.ycombinator.com