Hacker News Reader

Comment by Gravityloss | Hacker News Reader

Thanks for linking to good source material.

> When the feature requests, you must drive

Is there some kind of expected delay from the system requesting driver attention to the driver assuming control? Ie something like 3 seconds?

View on HN

That's the problem, no such system can guarantee it gives you a minimum amount of time. Some situations appear and develop in less than 3 seconds (think small child jumping in front of your car from between parked cars). You might only have 1s to react and if you are not "plugged in" and fully attentive at that instant you missed the window to react.

The more you "disconnect" as a driver, the less likely you are to take over in a split second and this has been proven again and again with humans. A system that works 90% of the time is probably the worst because it's good enough to give you confidence but bad enough that it's a false one. The driver is in the position to buy the advertised "you can disconnect" feature but then only be able to use it under explicit risk of harm to themselves and others. It actually increases the mental load as drivers will try to both "disconnect" (type on the phone, watch a movie) while also "paying attention to the road". Of course both can't be reasonably done at the same time so both experiences are sacrificed.

It's like encouraging people to take a taxi after drinking but then expecting them to take over whenever the driver makes a mistake and holding them responsible and accountable for the outcome. You won't enjoy the drink, or the drive.

These half-way solutions are great for marketing and for people who are all about the hype. But not only are they the bad compromises, they're also fueled by bad and incomplete data, and by marketing departments wanting to sell more. Almost no self driving outfit provides data on how many times did a human make the slightest correction while the self driving was enabled. Or out of all possible driving conditions, what percentage of them were covered by those self driving miles.

My personal philosophy is that cars are either self driving or they're not. Meaning they can either match an average human driver in all conditions they're expected to encounter over the lifetime, or they're just assisting the driver. If it's self driving except when it's not then it's just like a student driver. You wouldn't confuse them with being perfect drivers just because of their perfect safety record, given the supervisor corrects their every mistake.

> Some situations appear and develop in less than 3 seconds (think small child jumping in front of your car from between parked cars). You might only have 1s to react and if you are not "plugged in" and fully attentive at that instant you missed the window to react.

Yeah, humans are weak at this task, which leads to numerous deaths on roads. Can we develop a tech for this? Something, that will watch the road and will assist the driver.

Humans already kill 3000 people every day while driving

Politics is emotional, and emotions do not always agree with Utilitarian ethics.

Self driving doesn’t just need to be safer, people also need to feel that it is.

Sorta. Self-driving cars cannot a injure or kill someone in any situation where a human wouldn’t have.

I feel this too. I would rather have a greater risk of death but be in control of my fate.

Americans drive more than 3 trillion miles per year.

That's exactly the sort of task I'd expect an automated vehicle to perform better on. If they can't beat humans at that they're not ready for the public road.

Of course hand-off could be for liability purposes, like

Car: "I'm sorry Dave, I'm about to plow into a school, this is your problem now"

Dave: [looks up briefly from his game of candy crush before dying in a crash]

The problem is, they can't do this today with the necessary classification quality. Elaine was killed, because someone at Uber wanted these cars to drive so badly. It also is about culture and prediction. If the other participants behave differently to your prediction, it can get hazardous quickly at highway speeds. I can't find a better link [0], but maybe it's good enough to get the point across. The term is "warning dilemma" and means that you only have perfect information at the time of impact. Before, the classification if there will be a problem or not is worse, the earlier you ask. But asking early is necessary for the human to have a chance to react.

[0] https://www.researchgate.net/publication/326568066_Towards_a...

From what I remember, Elaine was mostly killed because she stepped in front of a moving car in the middle of the night.

I’m not convinced that’s the best example, since the AI might have actually done better if it had full control of the car instead of defaulting to ‘please take over now’ mode.

If I remember correctly, the cars built in system for emergency braking in this scenario was disabled. So, any other car of that type would have avoided the accident. However, because the AI apparently knew better, it was disabled and someone died.

AI has the problem that not only does it need to observe the environment and determine what to do, but it has to predict the actions of non-rational actor (humans). We aren’t always predictable. AI isn’t always predictable either. The woman who died may have assumed that a car driven by a person would have stopped.

Right. I could have stated it better, what I meant was that the (safety) driver was not paying (enough) attention to the road and was basically driving Level 3 (the car drives and it will beep whenever I need to take over, so just relax and browse the web). It was impossible to react to the short notice and the warning was _not_ given 10 seconds ago. I personally think the system had sensed her as a succession of standing targets and never predicted any velocity vector. Braking for static radar targets is not happening, because of the clutter you get everywhere you would not drive at all. So the misclassification and misprediction was not detectable for the system (in it's world model, everything looked consistent) and it warned too late.

Press release: "While the self-driving feature of our car had been engaged earlier in the drive, the driver was in control at the time of the crash."

>That's exactly the sort of task I'd expect an automated vehicle to perform better on

If the AI could talk, it wouldn't be so much "I'm sorry Dave" as "I cannae change the laws of physics!"

An AI can think about the situation for several million instructions worth, if it is running at >GHz speeds, but it can't prevent a crash given a few milliseconds, because inertia. And for the same reason, it's futile to hand off to a human.

This sort of assistance is an excellent use of current technology, and a way to move forward to true autonomy. What needs to stop here is the two-faced, fatally ambiguous marketing, and if manufacturers cannot act responsibly, regulation will be needed (more than the current regulation for responsible driving, which only kicks in after the damage is done.)

Who would the car optimize for? Saving the kid or saving the driver from getting rear-ended or swerving into an object?

See kid => hit brakes. This is already way better than a driver who doesn't notice the kid and doesn't hit the brakes. The inside of the car is very well designed to keep the occupant alive in case of collision, the car should prioritise not hitting other people. I hope self-driving technology will at some point be regulated regarding what the car should attempt to do in those situations. With more vehicles having collision avoidance technology, the car behind you won't rear-end you because its AI will react fast enough while also refusing to drive dangerously close to the car in front.

Ok now bonus Problem. The car has to swerve and either hit a kid or an old lady, which do you hit and why?

I wouldn't have asked the question if it was that simple :) If you have no other cars around you, are traveling less than 25mph and have plenty of stopping distance, sure, 'see kid => hit brakes' should work to save everyone in that circumstance.

If your assumption is that you won't get rear-ended because everyone will be using automated collision avoidance technology... Well, we're at least 15 years away from that.

As someone who bikes and is used to seeing every other person on their phone while driving, fully self-driving vehicles can't come soon enough. But we need to have real discussions about who the self-driving vehicles will save in regular circumstances. I'm skeptical about them optimizing for, or even having the ability to optimize for saving people outside of the vehicle over protecting the occupants.

The systems probably already have some kind of an estimate of "surprise possibility in the next 3 s".

For example on a narrow road with a building right in the corner, going around the corner, anything might happen. You don't know what's around the corner. So the system will either go so slow that there's ample warning time or then require driver attention already before attempting to turn, because the driver needs to be ready to react to potential up-and-coming new information fast.

In contrast on a wide highway, it's a more easy-to-estimate environment, visibility is better etc. So as long as other cars are far away and the velocity differences are not huge etc, the driver can stay inattentive.

The challenge self driving cars face is that being "overall better" is probably not enough. Regressions in certain segments would hardly be acceptable for anyone. You can't tell people "FSD cars lowered overall deaths by 10% but we now kill 30% more toddlers because [technical reason]", or that "highway deaths are going up", or basically any segment that anyone could reasonably care about. Cars have to match and exceed human drivers in every category for them to be acceptable, and even 99% there may not be enough.

Another point is that there is a psychological difference between sitting behind the wheel in a vehicle knowing that around 10 in 100,000 people get themselves killed each year, and then sitting in a machine that kills 10 out of 100,000 people each year.

In the first situation, you feel in control, and at least think that you can do what is necessary to avoid that you end up in the statistics.

Personally I would be very skeptical about trusting my life to a machine that has a non-zero chance of getting me killed, even if the machine, on average, performs better than a human, because most of us think that we are better than the average driver.

Unless, you know, there’s a reflective truck passing by. Or faulty recognition of lines on the road steer you straight into a barrier.

>situations appear and develop in less than 3 seconds (think small child jumping in front of your car from between parked cars)

I trust full self driving tech to handle this situation better than a human driver more often than not. The computer doesn't get distracted and has better reaction time.

The Tiger Woods driving experience as informal industry use case standard to focus on is as good as the informal two golf bags carrying capacity.

> full self driving tech

Full self driving tech is presumably (and by definition) fully able to handle things by itself. We're talking about "partial self driving tech" (SAE L3) which relies on the driver taking over when it doesn't know what to do. The context switching for a regular human absolutely kills the reaction time in these situations making this a "I'll mostly drive myself but when I can't you're almost guaranteed to cause a crash" type issue.

Reply on news.ycombinator.com