Comment by niftich | Hacker News Reader

Comment by niftich | Hacker News Reader

We're asking a lot from this kind of software (for good reasons), but humans commit hundreds of leaps of faith of various severity on the roads daily -- failure to yield, failure to maintain a safe following distance, assuming other drivers immediately adjacent to you, like in lanes that are significantly slower than yours, will keep driving safely and carefully. Urban, peak-hour traffic on most US freeways is an exercise in collective insanity, riding people's bumpers at 55+ mph (and often significantly higher), leaving little room to stop for incidents [1][2][3][4] or debris.

But only a small subset of these situations result in significant accidents, because unimpaired humans, largely, have some intuition for self-preservation. On the other hand, we're expecting an algorithm coded by humans to perform better than a complicated bioelectric system we barely understand.

Waymo's self-driving program has opted to thoroughly understanding its environment, which is why their cars drive in a manner that bears no resemblance to how humans actually drive. We as a society have to eventually reconcile the implications of the disconnect.

[1] Unsafe lane change in traffic with different lane speeds: https://gfycat.com/CleanGleefulArawana

[2] Tailgating causes crash, swerve, multi-vehicle accident: https://www.youtube.com/watch?v=j0rj2sZ1KA4

[3] Inattention to incident causes further accidents: https://www.youtube.com/watch?v=hZL6OKwQGew

[4] Inattention in slowing traffic causes accident: https://www.youtube.com/watch?v=Ff7wbSwTuEk

human drivers routinely do things that are not justifiable by vehicle physics (e.g. stopping distances) because they have a comparatively sophisticated and unconscious understanding of our physical world, how changes may develop and how other human drivers or pedestrians might react. self-driving is a much harder and unlikely problem than the perpetual hype-cycle is willing to admit.

Mostly because they use the heuristic of "see, nothing happened last time" and do it again but with a little less safety margin.

It's partly that, but also a matter of human drivers trusting each other to some extent. In congested areas if you leave the recommended safe stopping distance then another driver will always cut in front of you. So people adapt by closing up the gaps and hoping the drive in front doesn't do anything stupid.

I find this one interesting. I rarely drive in the city, so when I do the distance in queues seems pretty crazy. But at some point I stopped caring that people cut in in front. The active cruise control keeps my distance now (apart from when we're stopped) and often people will use that gap. But... so what? Let them and keep your distance again. It really improves the experience.

If you try to account for it, it's not much time at all. And you're improving traffic flow by providing a means to change lanes (a bit more) safely. Most people "cutting in front" are actually just in search for a spot.

Agree with minor nitpick

>human drivers routinely do things that are not justifiable by vehicle physics (e.g. stopping distances) because they have a comparatively sophisticated and unconscious understanding of our physical world, how changes may develop and how other human drivers or pedestrians might react

Whatever "we" do is justifiable by implication because what is and isn't justifiable is set by consensus. Traveling 85mph with 1.5sec following distances might not leave much room for error but it's still "justifiable" because what's "justifiable" is a function of what everyone considers ok to do.

Self driving systems don't need to kill less people, they need to kill less people in edge cases that are so trivially avoidable to humans that the only people who have similiar kinds of accidents are incapacitated.

Are you saying that even if AI can't react as well as humans, the roads would potentially be safer with more AI drivers because of the cumulative effect of all the little safety rules they follow that human drivers do not?

Yes.

Computers won't get distracted. Computers won't accidentally mix up medications and get foggy. Computers won't get tired. Computers won't drive drunk.

I can create a self-driving car better than 50% of Southern Californians right now: "<beep> High humidity detected. I'm sorry, I can't let you drive today, Dave. <beep>"

People suck at driving in new places--take a plane flight to a new city, rent a car, and have a passenger record how many illegal things you do--you probably won't make it out of the airport before the first infraction.

People are good at driving in the familiar because they memorize it. A new stop sign at a previously uncontrolled intersection causes chaos for a while.

Computers will also memorize the local environment at some point. Remember how bad Google Maps was when they started? This will be the same.

It is a shame that sales pressures are stopping us forcing "AI" cars from being branded in an obvious way (bright orange and pale blue stripes?) - this would allow human drivers to treat the vehicle as a non-human, removing a lot of the dangers it would cause to lawbreakers who follow too closely.

Of course `AI hybrid` vehicles would still behave unpredictably unless they had some way to signal whether the human or the `AI` was in control of the vehicle.

You're comparing what humans do to what software could possibly do, which is a category error.

Software doesn't recognize color coded warning signs.

Software doesn't trust it's stationary object recognition at high speeds.

Software doesn't distinguish between high speeds on limited access highways and high speeds on pedestrian accessible roads.

Software has a higher crash and fatality rate per mile driven than humans, right now, despite driving in statistically safer cars.

Yes there are good reasons to be hopeful software can do better at those things eventually, but it's obvious now that it's not ready (or Tesla wouldn't have to trash-talk a dead man for not maintaining high enough torque on the steering wheel to inform the car he was steering, because he wouldn't be dead and they wouldn't have to require passive hands on the wheel).

>Software has a higher crash and fatality rate per mile driven than humans, right now, despite driving in statistically safer cars.

Do you have a source for that claim because everything I have seen has been inconclusive so far?

From https://en.m.wikipedia.org/wiki/List_of_autonomous_car_fatal...

3rd Tesla autopilot fatality happened at 210,000,000 km, giving a fatality rate of 9.2—14.3 per billion km of autopilot depending on if you take the average just before or just after the third death.

From https://en.m.wikipedia.org/wiki/List_of_countries_by_traffic...

USA has an average of 7.1 fatalities per billion vehicle-km.

I’m assuming most of the Tesla kilometres were in the USA. I don’t know if that is a reasonable assumption.

>3rd Tesla autopilot fatality happened at 210,000,000 km

That 210 million km (130 million miles) number is just flat out wrong for that time period. That number originally comes from Tesla's public statements after the first fatality in the US. That was close to 2 years before that 3rd fatality happened. Tesla should be well over 1.5 billion km by now which would bring the fatality rate to down below 2 per billion km.

[1] - https://www.tesla.com/blog/tragic-loss

Tesla doesn't publish current mileage or crash incident reports under autopilot; given their willingness to lie about Wei Huang's conduct before his crash I'd be surprised if their statistics actually held up as beneficial. The numbers I recall seeing were specific to Uber, who were attempting to operate a fully autonomous vehicle (Tesla doesn't claim to, except when Elon does press).

In any event, if you're going to compare injury and crash rates, you should compare Tesla to other luxury cars with FCW+AEB. Those systems alone create 40% fewer crashes in at least one analysis[1] but don't come with the risk issues of automated lane changes and steering control that create driver inattention. That improvement is over and above passive safety improvements in luxury cars.

1. https://orfe.princeton.edu/~alaink/SmartDrivingCars/Papers/I...

So just so we are clear, you stated as a fact that "software has a higher crash and fatality rate" while not having any evidence that what you said is true?

His evidence is the 4 driver fatalities resulting from Tesla's Autopilot, which is infinitely more than the 0 driver fatalities of any other self-driving system.

Including pedestrian, cyclist, and other-vehicle fatalities, Tesla's fatality statistics are so bad that they exceed the combined total of every other self-driving system back to their respective launch dates.

Where did that 4th fatality come from? I have only ever seen reports of 3.

Either way, you can't credibly use aggregate totals without at least mentioning the underlying rate statistics. That would be like me saying that over a 100 people die every day from manual driving but only 3 people have ever been killed by self driving Teslas and therefore Teslas are safer. Statistics doesn't work like that. Tesla's have more fatalities than any other semi-autonomous car because they have orders of magnitude more miles driven. It is far to early to tell whether they are either safer or more dangerous.

Thanks for that link. I’ll update the Wikipedia page if nobody else gets there first.

I’m having trouble finding out if the earlier Chinese death was or was not caused by autopilot. All I see are opinion pieces. Do you have a citation for that?

I don't think anyone has publicly released a definitive answer on that. Tesla doesn't have access to the car or data and I am not aware of any investigation by a trustworthy third party. However the details of the accident are similar enough to both the other accidents and known Autopilot flaws that it seems reasonable to conclude that Autopilot was likely active at the time of the crash.

And I appreciate you correcting that inaccurate mileage reference on Wikipedia.

Autopilot is also (usually) only turned on in situations where it is worthwhile, works well, has a benefit.

General human driving rarely has that luxury short of "do not drive at all".

Comparing the "best/better world" scenarios of where AP is active, versus all miles driven is an apples and oranges comparison.

People also have trouble with changes to otherwise familiar places. I watched an elderly driver last week get very discombobulated by a local detour, at low speed. And I can see that happening to me in a few decades too.

My prediction is that if self-driving cars commingle with human-driven cars on the same roadway, and are common enough to show up during your commute, road safety will improve slightly in aggregate, but the immediate environs of traffic surrounding a self-driving car will be less safe.

There will be drivers annoyed at the car whose behavior approximates a tentative, wary driver: strictly adhering to the speed limit, leaving ample space in front, avoiding lane changes, braking early, and occasionally pumping the brakes for no discernible reason. These self-driving cars will create a traffic bubble around them, making it less safe for everyone else, including the aggressive drivers who will cut it off or overtake it at a high rate of speed, and the average motorists who are just trying to align their driving to the emergent conditions of traffic, and are caught between conflicting pressures and unsafe adjacency to different speeds.

It's unclear to me why human drivers should be considered unable to adapt to more "tentative, wary" drivers on the roads. After all, humans vary quite a bit in how they drive. There are plenty of cautious drivers already out there.

They do adapt; it's just slightly less safe than being among a set of vehicles behaving more similarly. Same thing with a particularly tentative human driver. (Or a particularly aggressive one, of course.)

Humans are pretty good drivers if you don't count the ones who are drunk, looking at a phone or otherwise impaired.

If you a bunch of AI cars into the mix safety will decrease unless you toss them in in a way that results in them displacing the more risky drivers.

> humans commit hundreds of leaps of faith of various severity on the roads daily

And this is necessary (to some degree, probably not as far as we take it though) in order to drive a car in a practical manner. Instead of thinking about it as "safe" vs. "unsafe", it's more helpful to think about driving as trying to limit your risk to whatever you consider an acceptable level, based on your model of the world.

From least to most risky, some potential models are:

* Assume nothing beyond the basic laws of physics

* Assume nothing outside your visual range is breaking any road rules

* Assume the road outside your visual range is as you last saw it (no blockages / hazards etc.)

* Assume other vehicles around you will mostly follow the road rules (basically what Waymo does, afaik)

* Assume other vehicles around you are alert and have a sense of self-preservation (the normal human driving behaviour you describe)

* Bomb through traffic YOLO style

The other issue, especially for emergency braking, is that whether or not something is a collision hazard or not is dependent on velocity, not just speed. Until emergency braking systems can take control of steering as well as braking there will always be false positives and negatives no matter how good we make the system's perception.

While we're on the topic of Waymo, does anyone know whether they use driving data from Google Maps Navigation? It'd be an interesting upper hand, if so.

Yes, they definitely do. I think there was a TED talk on google self driving a couple years ago which outlined how they used maps.

Reply on news.ycombinator.com