Humans are remarkably safe at driving
jperla.medium.com
jperla.medium.com
I'm a software engineer at Waymo, speaking for myself.
I agree with the premise of the article. Humans are remarkably good at driving, and making a computer better will be hard.
Where I may disagree with the author is, I think the project is both feasible and worth doing.
It's worth doing because we drive a remarkable amount. Even at human safety level, more than a million people die every year in car crashes.
As far as feasibility, Waymo is, today, running a fully autonomous, no-human-behind-the-wheel ride hailing service in Arizona. It's not a dead-end demo, it's there. I know, a Phoenix suburb is not the same as NYC or Mumbai. Trust me, I know -- probably better than you! Nonetheless, what we accomplished was impossible a few years ago.
The history of AI is a history of moving the goalposts and then blowing past them. AlphaGo solves what was considered one of the hardest problems in AI, but now that I see how you did it, eh, it's not that impressive. GPT-3 passes the Turing Test and can almost pass a coding phone screen, but come on, nobody really thought that a machine that passed TT would be intelligent. It was just a few years ago that respectable people were saying we'd never solve protein folding. Now that we have, the goalposts have moved again. Don't worry, we'll blow past those, too. Except this time people will yawn.
So give us some time. This too will happen slowly, and then all at once.
I wrote this to help people in the industry have the right context in the right way.
As to Waymo's driverless testing, it's dangerous! See here what I wrote about why: https://jperla.medium.com/tesla-saves-lives-waymo-gambles-pa...
Ultimately self driving technology has to be better not than a human, but than human intervention. It has to be so good that it recognizes when a human incorrectly intervenes to override L4 decisions and then be able to override that human's overriding. Only then will autonomous vehicles overtake humans.
Yes, there are people with poor coordination, spatial perception, the list goes on. Genetics makes a difference. Yet also, training makes a difference. So for those with normal or above normal genetic bounds for the above, training helps immensely.
And driving, responding quickly to issues, is often about training. Hence all the driving ed programs now mandated in many localities, prior to obtaining a license.
Still, it goes beyond that. One becomes more skillful, better at driving, as time progresses. At least, those capable of that. Meanwhile, we're moving humans further and further from direct learning experiences, and with each 'crutch', reducing training scenarios as well.
Of course, we're on a path here. The goal is ultimately to move more towards safety. However I suspect that "things will get worse", before better, as drivers will be less trained on manual tasks.
Edit:
My main point is, we're going to get to a point of 'AI drives, you monitor/intervene" eventually, as the mainstay.
Yet new drivers, will be less able to intervene. They won't have developed, or trained, via the same number of "close calls", and direct driving experiences.
Eventually, asking drivers to "intervene", will be asking for the least skilled to take over, to monitor, to watch.
And there are a lot of areas that are very high bar to handle with AI. Think on this ; all current driving tech is literally 'low hanging fruit'.
It's been a lot of work, I grant that. Yet it's been a lot of work for just that low hanging fruit!
For example, most AI driving is urban, in the southern US.
How about:
- In rural Canada
- In the middle of a snow storm
- With ice, and white-out conditions on the road
- on rural dirt backroads, which are 20 or 30 km long, comprised of potholes and other such irregularities
- with zero mapping
- at -40C
I have extremely strong doubts that AI can even remotely handle that well currently. I'd trust a 5 year old to handle this task far better.
Now, what of drivers who never really drive, end up on this side road, and get 'stuck', with the AI unable to handle things?
When all of the above is normal in the winter. For example, often schools do not close in rural Quebec, for such typical, and normal, conditions.
https://jperla.medium.com/why-tesla-autopilot-ought-to-be-aw...
Maybe iteration 1 of all this should be detectors that help drivers do better:
1. Unsafe driving speed alarms (given conditions)
2. Notification of areas that are historically unsafe
3. Baby in the back-seat!
4. Alcohol in the driver's seat detection (even if you don't have an existing DUI/setup)
Any 1 of those implemented en-masse would likely reduce accidents/deaths world more than Full self driving AI (in its current form), and if that was truly the goal, I don't understand why these aren't the top little buggers to fix and release in 2021.
One of the few rationales I can summon is that when you look at driving and travel as a whole, the total time/mileage that falls in the distribution of long distance may be significantly lower than that of the total time/miles spent in the more difficult inner city driving situations. Moreover, since we're looking at businesses not trying to do humanity a favor but pursing a profit motive, the driving spent in inner city travel is deemed more profitable than the longer haul travel, likely because you have to compete with rail systems, air transit, and so forth. That's all guesswork though, curious if data exists on this.
I'm the same way. Sure, I'll take full self-driving if it's on offer. But TBH, if you can handle the 2-4 hour parts of a typical weekend drive to the mountains, that's a big win for me. Ditto for handling most of the 70 minute or so drive to the nearest major city.
It's not as exciting and doesn't open up the shared vehicle use cases and so forth. But full autonomy under most conditions on limited access highways still seems like a great feature a lot of people would love to pay for. But the driving around my neighborhood and environs actually isn't something I generally find to be a particular burden,
Goalposts: moved!
This is exactly my point.
No, the original formulation is just a little unclear and some interpret it as an even more elaborate test, involving yet another stage. Most versions of the test include at least three participants [1]. Many people forget the details and remember just that it's about recognising a machine through text communication.
We don't believe AI would ever be able to solve some classes of "impossible" tasks, like breaking crypto. But at the same time, we say AI will just keep blowing past all hurdles in some slow singularity-like fashion. Those two ideas can't both be right. I'm curious where the lines converge at.
I'm not sure. Why is this so? To save time? Sure, but I think the self-driving proponents that I have seen have not addressed the new problems that will be brought by self-driving cars. I don't think reducing crashes is anything but a marketing ploy because the real goal of self-driving is to make companies a lot of money. I think accidents will simply be transposed and not reduced.
One problem is the initial rollout period where there is a non-homogeneous mix of human and automated drivers. This impedance mismatch will be problematic and likely lead to more accidents and congestion.
In general, it's my belief (I don't have facts or research, just a hunch) that automated driving will actually increase congestion and traffic. We saw this with Uber and Lyft, where these things plus accidents were increased where these services were deployed. I've seen it first hand that the cars causing issues are likely to be Uber or Lyft. I will be very surprised if this is not the case with automated drivers. For example, consider the case of an automated car dropping someone off in a busy area, probably one of the most wanted use cases for automated car. This is because it could notionally drop the person off and then drive off and find parking. But that seems like an insanely hard problem because such a maneuver is actually difficult for humans and very tricky in busy areas.
I truly don't understand the actual need for self-driving cars. There is a lot of hype, for sure, but I personally see that it will generate more problems. Basically the only problem that it will solve is reduction in human time driving because I doubt it will be actually safer such that it's a major difference from now. I feel our society would gain much more benefit in concentrating on better transportation and urban design: planes, trains, automobiles, bicycles, and walking.
That is to say I believe human-driving is at a local maxima and it will take accepting a negative dip if we want to progress on this front.
Transforming our roads towards better supporting automated vehicles through signs, markings, and other infrastructure does seem to be a natural conclusion to make it really work, but that all rests upon the assumption that these massive fleets of self-driving cars are required components of our future. We already have major issues maintaining the infrastructure on our roads. Fully transforming them will take a lot of time and money. I really think a more holistic approach is needed because self-driving cars will be solving a lot of problems, in hard ways, that are much more simply solved by (essentially automated) train, subway, trolley, and bus systems. Going all in on self-driving cars seems to be the wrong goal. We need to be thinking about how to transform our infrastructure and urban design to have the right solution in the right spot. I do think self-driving cars have a place in this, but simply slapping self-driving cars onto our existing car and road culture and design is not the right solution.
I really worry about the U.S.' nature about letting corporations dictate where society goes. We're in this mess to begin with because of car corporations destroying public transportation in cities and requiring the new city designs to be built around cars. I think we'll continue to see corporations create solutions that generate more problems.
My grandmother, like many grandmothers, has no business being on the road. She is a wonderful person. She is also a horrible driver, a threat to herself and others by virtue of natural impairments (vision, reaction time, etc.)
Taking away her license would make all roads safer, but it would also obliterate her quality of life. The actual need for driverless cars, to me, is a means of retaining her freedom of movement with a far lower level of endangerment.
For what it's worth, I wouldn't be working at Waymo if I didn't believe self-driving cars are very likely to save lives.
> In general, it's my belief (I don't have facts or research, just a hunch)
Indeed.
The data I've seen essentially shows the in vitro safety of self-driving cars. Yes, these cars are driving out in the real world, but they are singular in a sea of human drivers. The reports I've seen is that the cars are overly careful and just put along. The safety data of such trials will be skewed. You can't simply extrapolate that safety data out to where now you have say 25% self-driving and 75% human drivers and so on. It's a dynamic problem.
I've seen presentations by Waymo managers. Just as little as a year or two ago, the cars were basically incapable of handling snow or rain. You may be offended by the marketing comment, but until we see holistic data that supports what the marketing says, that's the reality.
The existing data on Uber and Lyft is clear. They increase accidents and congestion. How do self-driving cars approach this problem? Because there's clear precedence that the introduction of a new paradigm of driving causes problems, even ignoring the aspect of self-driving versus human driver.
That's why driving in different cities for an experienced driver can be so nerve wracking... it's hard to predict what all those vehicles will do! Meanwhile they are also predicting what you will do so you're dangerous as well (don't stop in traffic just because a pedestrian walks into the middle of a busy street in Delhi or Naples, you'll cause an accident).
The economic incentives for autonomous driving are interesting, for but just saving lives, collision avoidance is where to focus. If you have tech as good as Waymo, integrating that into a car so it intervenes when it's highly confident the driver doesn't know what they're doing and highly confident it knows the right thing to do would be huge.
Please don't call them "accidents".
>Before the labor movement, factory owners would say "it was an accident" when American workers were injured in unsafe conditions.
>Before the movement to combat drunk driving, intoxicated drivers would say "it was an accident" when they crashed their cars.
>Planes don’t have accidents. They crash. Cranes don’t have accidents. They collapse. And as a society, we expect answers and solutions.
>Traffic crashes are fixable problems, caused by dangerous streets and unsafe drivers. They are not accidents. Let’s stop using the word "accident" today.
I suppose we could ban streets and cars... if they're all so dangerous?
https://www.patrickdaniellaw.com/crash-vs-accident-differenc...
But, then what does the meaning of accident become? With this line of thinking, nothing is an accident, everything has a cause and potentially a prevention. So once nothing is an accident, then we can start using the word accident again? I mean, most car crashes are unintentional(as are injuries at factories, and gun discharges) so why are they not accidents? Or are you arguing for the word and concept of accident to not exist at all?
Sometimes being more specific about causation is important; sometimes being more specific about culpability is important.
It's also pretty pretty close to, what's a good example, sui-cide, electro-cution, so we might refer to car-cide or ac-cide, i.e. death by driving, with some poetic license.
It's not a coincident, where the stress lies on in, for an hinest one-time case. Also cp. case, casus by the way, from the same root. It is rather formulaic, which is however reasonable if it should cover even the most minor cases.
The fact that accidental could come to mean by chance, involuntary was probably due to the consequences being a bigger worry than the causes, in most cases. Legalese has developed a more fine grained scale of responsibility, leaving room for doubt between dolus eventualis, dolus directus 1 and 2 and in contradistion so called negligentia, luxuria and culpa latis all of which should be nevertheless culpable, in theory. However it is not always clear who to blame. Certainly not the wording, is it?
https://battlepenguin.com/tech/self-driving-cars-will-not-so...
Even in smaller cities, building transportation infrastructure would greatly reduce congestion, and even reduce the need for future highway expansion. The common excuse is "America isn't built that way" but .. well it was at one time, less than 100 years ago. We use to have more streetcars and passenger rail than EU does now!
I didn't address the technical merits, but we've already seen leaked Google training data on here (that was really bad training data because it seemed to use automated/AI to build the training data and put boxes around a lot of things it shouldn't have ... can't find the link to that now), we know about Uber's engineers screwing up royally and killing a cyclist, we know about Tesla's autopilot getting people killed ... which shows even using sensors as safety features may make people less attentive to the road.
It's a hard problem space. Personally when I get really old and can't drive anymore, I'd prefer to live some place where I could still walk to cycle or take a train everywhere I really need to. I trust that stuff more than us finding the magic bullet to self-driving AI.
And I think what we can get from this article: good self driving AI requires advanced general purpose AI. And once you have that, good luck getting something that intelligent to drive you car. I imagine Marvin from Hitchhiker's Guide, "The most advanced robot in the galaxy asked to do the most menial tasks ... fix the engine Marvin. Fly the ship Marvin. Go buy groceries Marvin ... I think you ought to know I'm feeling very depressed."
Data gets thrown around as a mindless incantation with respect to autonomy. Yes you need data. But just collecting lots of data and feeding it into a computer somewhere doesn't make better autonomy just pop out of the other end.
I have also wondered if the better drivers have already found jobs with UPS and FedEx since they pay much better.
Specially too many near frontal collisions, some left dumb found, namely those done in curves and slopes, or trying to pass large and compact lines of cars.
I don't understand the decision making process of those few wild cards. Somehow they're lucky that everyone else aren't like them.
Per 1 billion miles driven, the US fatality rate is nearly 2x that of Germany.
https://www.inc.com/justin-bariso/why-getting-a-german-drive... https://en.wikipedia.org/wiki/List_of_countries_by_traffic-r...
"The death rate per 100 million miles traveled ranged from 0.54 in Massachusetts to 1.83 in South Carolina.Footnote1"
https://www.iihs.org/topics/fatality-statistics/detail/state....
This very controversial statement appears in the middle there without any real justification.
And the title is quite poor as well—it is in fact Tesla who’s gambling lives by aggressively rolling out their self-driving capabilities before they’re fully baked (and having paying customers bet test them), not Waymo (who runs very limited demos in very controlled conditions).
The article admits it has no evidence for this speculation ("Is Tesla doing this? Perhap not.") but at other points assumes it is true ("This is the Safety Paradox. Tesla cannot launch its safest, newest version of the algorithm for every driving mile because that would, paradoxically, cause more accidents since the safest, newest software would be above the L3 Barrier.")
It then makes the further unsupported assertion that you can measure the extent to which Tesla has "perfected" its self-driving capability by measuring the rate of Tesla accidents that occur when Autopilot is not enabled, theorizing that this is the only mode where Tesla has not hobbled its safety features. In this mode, the article theorizes, the full power of Tesla's self-driving technology goes to "correct for human driving, as if one has a virtual bumper car, protecting and cocooning the driver."
In general the site seems quite pro-Tesla and anti-Waymo.
Disclosure: I work for Google but also own a Tesla.
"Moreover, Waymo cannot get the requisite number of miles to either prove safety or truly get all the edge case long-tail test data to make it safer than humans. It’s too expensive to pay trained safety drivers, since you need more data and more miles to get edge case data as you get safer and safer. It would cost hundreds of billions of dollars or more to drive the billions of needed miles. It’s not even close to possible to get enough paid rides in remote Arizona to drive billions of miles. The unsupervised rides save them a bit of money, but risk lives, so at what cost to the population, to us?
"As a comparison, Tesla gathers test data cheaply. Tesla had at least 10 billion miles of data driven this year worldwide in every condition, and customers paid them for the privilege to drive them as (unpaid, non-professional) supervising safety drivers."
I believe he's right about the data, although I'm not entirely sure about the quality of the data. I am also not sure I'm happy with his, and Tesla's, solution. (In order to discover edge cases, you have to put the vehicle in potentially dangerous situations and hope that the unpaid safety driver can extricate it without killing anyone.)
* I once had a car driving way above the speed limit approach me from the rear, it passed me by jumping the curb driving on grass and sidewalk and swerving back onto the street in front of me.
* Another time I was driving back to a rental house in the Aspen area and the snowstorm had gotten bad enough to completely cover and obscure the road on the hillside I was driving on. My wheels went off the road, fortunately on the side away from the drop off and I was driving slowly, but then I couldn't get back on the road without using enough power that I feared I would swerve off the road on down the hill. This required a bit of puzzle solving before I could safely get back on the road.
* I've had bad GPS data that kept me circling my destination without every getting me there.
I just think that such unusual situations are going to be difficult for autonomous cars to understand very soon.
[1] I don't think manufacturers of self-driving cars (and can we please call them "auto-autos"!?) should compete on safety, rather, I think it should be mandated legally that they have to share data on crashes with a view to preventing further crashes. Like the aviation industry, eh?
I would bet that the author has not had the chance to observe common driving practices in New England by drivers unfamiliar with winter weather conditions. Ideally we would all slow down in the snow, but I would hesitate to claim that we all adopt those safety protocols.
The problem with trusting humans to use their own judgement is that it is very self-focused: "Am I going to die in an accident?", not, "Could I hurt or kill someone else"? Given how many people (at least in the US) refuse to wear masks for the health of those around them right now, I don't think those people would also be concerned about hitting others.
FWIW, I think it's possible to safely use your phone at a stoplight (I frequently do), but I saw my friend release the brake and start rolling and then look up from his phone, because he had seen it was green in his peripheral vision. That is not safe, and I think scenarios like this happen more than we realize.
All I have been able to find is:
* https://abcnews.go.com/US/red-light-camera-backlash-cameras-... "On the other side of the debate are statistics that show the cameras also cause accidents. A 2005 federal study demonstrated that while injuries from right angle or T-bone crashes decreased by 16 percent at red-light camera intersections, injuries from rear-end collisions increased by 24 percent."
* https://www.iihs.org/topics/red-light-running "When it comes to crash reductions, an IIHS study comparing large cities with red light cameras to those without found the devices reduced the fatal red light running crash rate by 21 percent and the rate of all types of fatal crashes at signalized intersections by 14 percent (Hu & Cicchino, 2017)." (Referring to https://www.iihs.org/topics/bibliography/ref/2121.)
"Previous research in Oxnard, California, found significant citywide crash reductions followed the introduction of red light cameras, and injury crashes at intersections with traffic signals were reduced by 29 percent (Retting & Kyrychenko, 2002). Front-into-side collisions — the crash type most closely associated with red light running — at these intersections declined by 32 percent overall, and front-into-side crashes involving injuries fell 68 percent." (Referring to https://www.iihs.org/topics/bibliography/ref/1510.)
"The Cochrane Collaboration, an international public health organization, reviewed 10 controlled before-after studies of red light camera effectiveness (Aeron-Thomas & Hess, 2005). Based on the most rigorous studies, there was an estimated 13-29 percent reduction in all types of injury crashes and a 24 percent reduction in right-angle injury crashes.
"Not all studies have reported increases in rear-end crashes. The review by the Cochrane Collaboration did not find a statistically significant change in rear-end injury crashes (Aeron-Thomas & Hess, 2005)." (Both referring to https://pubmed.ncbi.nlm.nih.gov/15846684/.)
"A study sponsored by the Federal Highway Administration evaluated red light camera programs in seven cities (Council et al., 2005). It found that, overall, right-angle crashes decreased by 25 percent while rear-end collisions increased by 15 percent. Results showed a positive aggregate economic benefit of more than $18.5 million in the seven communities." (Referred to https://www.fhwa.dot.gov/publications/research/safety/05048/....)
I was shocked at how easy it was to rent a car (with insurance!) in a country that drives on the other side of the road, and how not inept I was at driving like that. It's still a little nerve-racking.
I agree with the premise that humans are great drivers and replacing humans will be a tough challenge (which makes it fun). However, the idea that SDCs cannot be verified and the rockets vs SDC analogy is incorrect. Let's go even closer and just compare cars to SDCs. What makes your standard cars that has 150+ of HP safe? Better yet, most of this energy comes from explosion of fuel! Any of the mechanical failures in a car can lead to death (brake calipers breaking, ECU malfunction, pistonhead cannonball, etc). But yet, how often do you hear in the news someone dies because their car exploded while using it?
The reason why cars are robust is not because it's mechanical, deterministic, some sort of inherent physical laws, or manufactures test every single part coming out of the production line. But rather cars are safe because there is a robust safety system that heavily incorporates a verification and validation process with a promoted transparency process that mitigate risk. A good example of this is ASIL (ISO-26262) [1] and FMEA processes [2]. Other industries like rocket/military (like you mentioned) and medical both deploy the same type of model to make their product safe and robust.
There is no glaring technical barrier that prevents SDC. The hardest barrier for a product has always been cost vs profit (likewise for most products, economics of it always comes first).
[1]https://www.iso.org/obp/ui/#iso:std:iso:26262:-9:ed-1:en [2]https://en.wikipedia.org/wiki/Failure_mode_and_effects_analy...
We move with extreme precision and coordination on the unconscious level even when we seem clumsy or uncoordinated on the conscious level. We are acting out cultural and personal formats or roles, a kind of post-hypnotic suggestion. This is part of the reason that you can learn hypnosis and accelerate learning: we are operating at a fraction of our potential due to old programming, upgrade your software and it unlocks "new" capabilities.
With these types of questions, I feel like if you're within 1/2 an order of magnitude, you answered correctly.
Why Tesla Autopilot ought to be awful, until it’s perfect
https://jperla.medium.com/why-tesla-autopilot-ought-to-be-aw...
Waymo Gambles Lives with Unsupervised Self-Driving Cars
https://jperla.medium.com/tesla-saves-lives-waymo-gambles-pa...
Granted, I'm certainly not downplaying your conclusion about the challenge of designing a self driving car.
What does get harder is there is no objective idea of what is a "major" accident versus "minor" accident, so you can spend a lot of time arguing and manipulating statistics, whereas death is easier to quantify objectively and compare (and also happens to be the most serious thing so a necessary analysis anyway).
You say "void of content", then list content.