Waymo self-driving car crashed when driver fell asleep and triggered manual mode
qz.com
qz.com
The statement is the "Driver no longer works for waymo" not that "they were fired for the incident.".
That feels somewhat deliberate choice of words to me.
Their job is not to flake on duty and then, by pure accident, reveal the system's inability to recover from a failure more.
Surely?
Manual driving mode bell is not loud enough?
Google should have a camera pointing at the driver and use machine learning to detect someone that is asleep or medically impaired and automatically take over if the car starts to leave its lane.
Perhaps the driver could configure where it wants the car to go in such a case (if asleep: continue to final destination; if medical issue: drive to nearest hospital and call "Parent" or "Spouse" on cell phone and explain the situation; if smart watch can detect alcohol in your sweat therefore unconsciousness likely due to intoxication: drive to a trusted friend and call ahead, etc.).
At the very least, if the person appears to be asleep it could ask "Are you sure you want to return to manual control?". It should be able to tell the difference between someone sleeping and someone alert wanting control back.
It's been some decades since fiction inspired us, but we're getting close!
Humans are the weak link.
Source? The Waymo cars without steering wheels are just a concept design, not used in production.
The basic idea is that the switch can only be maintained in a non-braking state by an alert, not-incapacitated person. If you press too hard, it brakes; if you don't press hard enough, it brakes. You have to apply just enough force, and that's something only an alert person can do.
Clearly, using the gas pedal to turn off auto-pilot is not good enough.
By the way, we now have a pilot program in Hong Kong where cameras are installed in buses and use computer vision to detect if a bus driver is dozing off. The system then automatically makes the driver's seat vibrate and sound an alarm to wake the driver up.
Or conversely, suppose the switch must be maintained while the car is in autonomous mode, should the car pull over and stop on the side of the freeway? In terms of user experience I would expect the car to continue to the destination. The commenter discussing the use of sensors to detect the state of the driver and act accordingly makes more sense.
For a production car, sure. But these are test drivers in test cars.
I ultimately hope this is a question answered by legal standards rather than by "user experience." If we're requiring drivers to be present even in autonomous cars, then they need to pull over and possibly take other alerting actions.
> The commenter discussing the use of sensors to detect the state of the driver and act accordingly makes more sense.
Are they asleep or merely in a diabetic coma? Can we tell? If we can't, should we even try this?
Ultimately, I'm really concerned about this line: "Improvements in this case meant altering night-shift protocol to have two safety drivers instead of one."
Humans may be bad drivers, I'd say a good look at the data would suggest otherwise; even so, the lesson for Waymo should be "Humans are also bad engineers."
It's a trade-off: if the auto-pilot is doing something unsafe, it is better to have the human intervene immediately (this might mean stepping hard on the gas-pedal, or braking, or turning, depending on the scenario, which is impossible to know in advance). The assumption is that the software/data are not perfect (yet) and the human knows what's best. I don't think "don't trust the human test-driver" is one of the current parameters, especially since that is their literal job.
I like semi-autonomous systems because they still take some of the cognitive load off of driving. Paying attention is still easier than paying attention plus constantly having to adjust speed/steering/etc.
Cruise control is great. I still control where the car goes though. I still have to be 100% attentive to what the vehicle is doing. At no point am I relying on the vehicle to keep me safe.
Automatic transmissions are great too, except for very specific use cases.
> I like semi-autonomous systems because they still take some of the cognitive load off of driving. Paying attention is still easier than paying attention plus constantly having to adjust speed/steering/etc.
I think I would be more open to 'smart' information systems - beeps when leaving a lane, back up cameras, etc. Anything that takes piloting control is a step too far for me at this time. Cruise control is largely proven to work. I never worry about it failing when I use it.
I like the idea of emergency braking as one last backstop. It can react faster than I can, and if prevents something I’m happy. It’s not like I’ve given up the brake pedal to it.
Lane assist is similar. It’s very useful in high crosswinds, but it’s reliable enough even on well painted roads that you get used to it and end up relying on it. It feels a bit like power steering: makes the job easier but you never stop doing the job.
This is an example of something I might be open to, but I guess because I write software I have a million 'what if' nags worrying about edge cases.
> Lane assist is similar. It’s very useful in high crosswinds, but it’s reliable enough even on well painted roads that you get used to it and end up relying on it. It feels a bit like power steering: makes the job easier but you never stop doing the job.
On this I disagree. If you're not controlling the vehicle enough to keep in your lane, you're not paying enough attention - except in the high crosswinds scenario. I don't drive in high crosswinds, so I guess that is not a use case I am familiar with.
Like many of these it requires active input. It will turn itself off (with a nice audible and flashing warning) if you don’t keep doing your part. You’re not going to get more than 30-45s of ‘I don’t have to do anything’ before it turns off and it’s all on you again.
as for the steering and lane keeping, even at V8 it was perfectly fine across hundred mile stretches of interstate. where it had difficulty is when the right line vanished for splitting highways or off ramps. still it recovered to the left before fully going to the right. Not 100% but I bet over 90% of my 600 mile commute North was with both traffic aware cruise control and auto steering.
What took me the most to get used to was TACC which many cars have and have had for years. Once I learned it really did stop and move correctly everything fell into place. The steering was my least concern and I quickly understood its limitations. The payoff was, realizing I was drowsy by seeing something in the rear view mirror I did not recall passing and the car kept me and others safe. Needless to say that was a wake up call to take a break.
Is it perfect, no. However I am under no illusion that I will be able to hop in a FSD car of any brand in bad weather and have it take me to the hospital within the next five years and maybe even ten. I really want to see how they overcome snow and near white out conditions that I have driven because at the time I had to
"He regained alertness once the car crashed..."
Yeah I bet he did.
Waymo currently has Level 3 cars, which is why they employ safety drivers and haven't made their cars publicly available.
I have no idea how to know exactly though. Waymo is pretty tight lipped about this. I work at Google and I don't know for sure.
They should really have two test drivers there, though.
A passenger can hold the steering wheel until the driver comes to. Or they can even use a car with two sets of pedals, they're common in driving schools.
I hope they also have reasonable mandatory breaks. It would seem that there is pressure to stay on the road as long as your peers. Which could lead to drivers not taking breaks when they should.
Unless they set up a completely separate chain of command that is in charge of validating if an employee is legitimately being fired, then it's always going to end up following their incentive structure
Even if he did respond, there's still the problem of whether he can respond in time. Driving normally requires sub-second reaction times.
None of that should take away from the value of pursuing it however. Humans are in fact bad drivers, and have large numbers of failure modes like getting inebriated or sleepy or angry that a system can simply avoid entirely. But arguably even more important then safety, which is what tends to come up most, is utility (after all we put up with vastly less safe cars and driver practices for decades and decades before the present time already). Personal arbitrary-point-to-point mechanized transportation is effectively required for full participation in much of the modern world, it's just the danger and expense that makes it a quasi-right rather then a full one. The young and elderly being unable to take advantage is already a suboptimal state of affairs, and it's frankly a waste of everyone's valuable and limited human brain time that they're required to act as uncreative biocomputers in that role. It's all been worth it anyway because it's just that valuable, but removing a human having a required role is really just fulfilling what the tech should have been like from the beginning if our synthetic mechanical development didn't far outstrip our synthetic information collection and processing development. A certain amount of ongoing casualties and damage is entirely acceptable to achieve a societal improvement of that level of value, even if it wasn't much safer at all.
Things like the Apple Watch monitoring your awareness and activity level, and sending it to the car which uses that information to influence how it drives, or when it vibrates the seats to keep you awake or wake you up, and so on.
There's a lot more 'understand humans' to add to these systems before they're really ready.
If you have a human overseer in the seat, you want to find ways to continue to keep them engaged even if it isn't necessary for the vehicle's autonomy.
When it's time to disengage the vehicle you don't want to be fighting for control with the computer, manual override needs to be on a hair trigger in case the computer makes a sudden error. Unfortunately so long as there's a human in the loop, the propensity for human error is going to be there. I honestly don't know what Waymo could do to prevent this, multiple bells and whistles went off.
Waymo takes this stuff very seriously, they absolutely do not want to have to face the music should one of their vehicles get into an at-fault accident or suffer some other kind of catastrophic failure.
I just did a Google News search for "asleep at the wheel", and in the last week a few people have been killed, a few more hospitalized, and the more minor incidents don't even get reported. Waymo sincerely wants to solve this, but there's an awkward no-mans land between no autonomy and fully autonomous where Vigilance Decrement[1] is an issue.
Fortunately this was a minor accident. Waymo's test fleet reportedly covers 25,000 miles a day, and in regular day to day driving minor accidents happen about once every 160,000 miles. So if incidents like this are happening once a week for Waymo, that's about par for the course.
https://en.wikipedia.org/wiki/Vigilance_(psychology)#Vigilan...
If you tell someone that they will be fired the literal instant they doze off, all you are doing is guaranteeing that they will get fired, especially when you also task them with a boring job and ban all means by which they might boost their orexin levels during their shift. That is one of the hormones that regulate alertness, and responds to mental engagement. It cycles up and down with the circadian period. The result: bored at night; out like a light. The occupants of a self-driving vehicle will fall asleep at night. If you aren't testing that use case, you aren't doing your job.
Managing alertness levels at night is the company's problem. Bright lighting with plenty of blue light in the spectrum is part of it. Keeping the employees from getting bored is, too. Sleep countermeasures will have to be installed with user-monitored car automation, and the safety-driver/tester just saved Waymo from a future liability nightmare. Why are they no longer with the company?!
I think one solution may be to gamify safety-driving such that the human safety for an autonomous driver earns points for vigilance-related tasks. Identify potential road hazards before the autonomous system does. Count blue cars. Play the 49-state license plate game. Play a trivia game over the audio system. Hear about local points of historical interest near where you are driving. Anything other than silently watching moving road lines extend beyond the range of your headlights.
It's an interesting edge case that you might not think of, or thinking the odds might be "one in a million" and not design for it; I wonder how many driving hours etc have occurred so far (and other statistical analysis to breakdown different contexts) before this "one in a million" event occurred.
People will be driving nonstop cross country in autonomous cars. This could easily happen. NYC to LA in 2 days.
At this stage in the game, these prototype vehicles should absolutely detect whether or not their operators are paying attention, since they’re part of the safety system of the vehicle.
Also, did they not learn from Uber’s fatal crash? A sole operator in these vehicles is a terrible idea. One person to monitor road conditions and one person to operate any computer equipment should be the standard until we hit level IV or V autonomy.
I'm being serious when I ask. I question weather people can safely use these vehicles in these modes where they're supposed to not interact ... but pay close attention at the same time.
I think it is going to have to be full autonomy, or not at all.
Of course they can't. Watch people who've been looking at their phone while the light's red. Watch how they drive after it turns green. Some of them drive away like they're drunk, lurching and weaving, like they're reorienting themselves to the driving task. On the rare occasions I get even slightly engaged with the phone at a light, it's almost like I have to "reboot" back to driving mode when the light goes green.
The part of the "throwing it in the human's lap at the last second" that bothers me is that a lot of context gets missed when it's time to grab the wheel. At least for me, I get context a little bit at a time. That guy in the left lane is looking at his phone more than the road...<a few seconds later>...big truck's going to want in this lane when his ends, leave room...<more seconds>...I see taillights 12 cars ahead, what's going? And Level 3 is going to just hand me the wheel without me knowing any of that? Yeah, I see why Waymo is shooting for Level 4.
You also miss the interaction between humans. If another driver and I look at each other, we're at least aware of our existence together and I know what is up.
If a driver doesn't look at me ... does he know? does his car know? If he is suddenly given the wheel ... I don't think humans can handle that.
Complaining about a prototype having different UX than a production vehicle is like complaining about a racing car having different seatbelts than a road car. Different design requirements for different usages.
It's a prototype vehicle, where the self driving features are experimental and not certified by a third party. Extra safety systems are necessary that might not be reasonable once the tech is polished and regulated.
The car did warn the driver with two audio signals, what more should it do?
If you add safety measures to manual (possible outcome of GP’s comment) then you’re back at not having a manual safety override.
1. https://about.gitlab.com/2017/02/01/gitlab-dot-com-database-...
I think having a sole operator in a prototype vehicle drastically increases the odds of these accidents, and that companies like Waymo should run them with two operators and some drowsiness detection systems (which are common on luxury cars). That’s a relatively simple fix.
It’s important that testing with multi ton moving vehicles be handled with far more cars than most production databases are, speaking to your metaphor. The consequences of mistakes is much much higher with cars than it is with most databases.
It seems to me that a self driving car is preferable to having a driver that is asleep, zoned out or dangerously distracted by whatever is happening on their phone or in the care. Many accidents have that as the root cause.