When a human kills someone with a car it is almost always in a way we can empathize. When you’re around cars you have pretty good mental models for the humans that drive them and how the car will behave.
When you’re around human piloted cars you can look at the driver and get a pretty good idea of intent. You can tell if the driver sees you, you can tell what their mental state is, if they’re paying attention, what they intend to do. You can sum up a person with a glance, this is the power of evolution, we’re really good at figuring things out about other living things.
Crossing the street in front of a car is a leap of faith, not troubling at all when there’s a human there, but a robot? There’s no body posture, no gestures, no facial expressions, nothing to go on. There’s a computer in control of a powerful heavy machine that you’re just expected to trust.
Robot cars don’t make human mistakes, they make alien mistakes like running down kids on the sidewalk in broad daylight, things which don’t make any sense at all that make people feel like they aren’t anywhere safe.
It won’t take but a couple of cute kids killed in a surprising matter to shut down the whole autonomous experiment.
This is something that's very solvable though. Robot cars should and almost definitely will have a way to communicate to pedestrians. I agree with the general point though around a greater possibility of very out of the norm mistakes.
From the pedestrian's perspective, it can be hard to see the driver at all (small movements of the hand can be invisible in sun glare; direction of gaze likewise), and also hard to tell what they're doing. Just because they're slowing somewhat as they approach doesn't mean they see you or intend to stop.
We tend to overestimate the power of the human brain. There is a lot we don’t know yet, but we shouldn’t treat it as magic and unsolvable by AI.
"There is lots of evidence that AI can be much _better_ than a human driver, although currently on things like well mapped highway driving with clear conditions."
The only thing I know about is claims from Tesla, comparing Autopilot (in "well mapped highway driving with clear conditions") versus general human drivers (and which, as I recall, had some further sketchiness).
and perfectly visible unambiguous markings. The bar is rather low.
Also while it is scary to us and it will take a while there are already self driven vehicles we just take for granted like elevators and driverless trams/trains. Sure they are much easier to make but they weren't trusted at first.
Trains are on a track, the scenario never really changes. The variables are low. Just like in an elevator, which is largely a mechanical device anyway. There is no real decision making.
I read a statistic long ago - don't know how true it is, but it feels truthy - that half of all traffic fatalities happen between 9pm and 3am on friday and saturday nights. The fact that autonomous systems will never be intoxicated, distracted, or emotional makes me feel much safer.
https://injuryfacts.nsc.org/motor-vehicle/overview/crashes-b...
If the systems are broadly as safe as humans _including_ a significant set who are drunk / high / distracted, that feels subjectively much less safe even though the statistical number of accidents is the same.
The danger with an autonomous vehicle is it not seeing you. The danger with a driver is not noticing you.
There is a car controlled by computer. Pedestrian (child) abruptly enter into road from behind cover. The Computer knows that with current speed it is impossible to stop. Its other choices is to drive into the sidewalk killing an old lady, drive into the opposite lane risking the life of a car owner and people in another car.
A Human driver can decide on instinct, usually protecting themselves. The Computer needs to have an algorithm that decide who will live and who dies.
Then the car was going too fast. Full stop. The rest of your scenario is irrelevant.
How can you possibly say that with a straight face?
There will be situations where a car is going a perfectly fine speed for the situation and then the situation changes in a way the car cannot have seen, known about, or anticipated resulting in a crash. This happens all the time with human drivers. It will happen with AI drivers too.
Furthermore, we don't subject anything else that moves to this burden, why would we do so for cars (AI or otherwise)?
There’s a thing about the car companies having essentially stolen public space from the everyone else when they made it incumbent on pedestrians to watch out for cars, and a desire to reverse this.
Of course this would pretty much invalidate them as a transportation mechanism, but that’s the point.
Yes we do. There are many jurisdictions where in a pedestrian-car collision, the car is presumed to be at fault unless the driver can prove otherwise.
The parent's situation was not one of those. This was a situation where "Pedestrian (child) abruptly enter into road from behind cover." and there's an old lady on the sidewalk. In other words, there's a situation with limited visibility, limited room to maneuver (since the only other option is to go up on the sidewalk) and pedestrians present (the child may not be known but the old lady was). If you're in that situation and you're going too fast that you can't stop on a dime, you were not going a "perfectly fine speed for the situation".
And humans don't go past parked cars on a street at 5mph.
a) A computer can initiate braking a small fraction of a second faster than a human, which is great but not such a huge difference in braking distance.
b) A computer cannot brake any harder than a human, certainly not "much harder". The max deceleration rate is traction-limited, which any remotely modern car (last ~25 years) can easily sustain with an untrained driver thanks to modern ABS.
(As a side hobby I instruct in car control and accident avoidance clinics. Blindly braking hard is not often the best answer.)
That is sacrificing the counterfactual children that wouldn't have been killed if the bad human driver had been replaced by an average autonomous car.
As much as we engineers want to look at numbers and say "see, safer!" That does little to help people who have had deaths, or the visceral impact that death can have.
This is a key area of driving that has been completely overlooked in AVs so far - giving feedback to other non-car road users.
Not hard on the face of it (excuse the pun).
Nonsense. You can empathize with someone texting and killing someone?
This whole post reads like an attempt to appeal to people’s emotional attachment to human drivers coupled with fearmongering about robots.
You are placing far too much emphasis one our ability to “read” other drivers intent and the impact this has on automobile accident fatalities. Many accidents occur without any chance to see the offending driver e.g. accidents at night, someone switching lanes when you are in their blind spot, a drunk driver suddenly doing something erratic, etc. Moreover, this so-called advantage of human drivers is statistically meaningless unless you believe that the number of deaths due to automobile accidents is at an acceptable level and that it cannot be improved with technology, in this case, AV. I certainly don’t believe that. In the not too distant future, I believe this position will be laughable. Through adoption of autonomous vehicles, many predict we will drastically cut the number of fatalities. Will there be issues along the road? Most certainly. But as long as the overall number is falling by a significant amount, we simply cannot justify our love affair with humans “being in control”. We’ve proven to be perennially distracted, we have terrible reaction times, we have extremely narrow vision, we panic in situations instead of remaining calm, etc. and yes, these faults do lead to the deaths of children. These are not theoretical deaths like the robot scare tactic examples, these are actual deaths from human drivers.
Who are these many people, and why should we believe their predictions?
> We’ve proven to be perennially distracted, we have terrible reaction times, we have extremely narrow vision, we panic in situations instead of remaining calm, etc. and yes, these faults do lead to the deaths of children.
We've also proven that all software has bugs, and developers keep introducing new bugs in every single release. There is no reason to think that self-driving car software will be any different. Whats worse is that when software is updated, these bugs will now be pushed out to tens of thousands of cars - instantly.
Bit much to call someones position nonsense when they're just skeptical of obvious stuff :)
But I’ll go further and double down and say the entire post is nonsense. Why? Because the author’s skepticism doesn’t extend to the human factor. The position is not an accurate representation of the facts i.e what causes accidents (humans) and the known data around AVs today. If AV risk is so obvious as you claim then why does the enormous amount of data show that AVs are involved in the less accidents and lead to less fatalities than cars operated by humans on a mile per mile basis? And how is the negligent human driver not obvious as a source of automobile fatalities? The notion that we are safe because we can read humans is not substantiated by anything. Maybe you believe this number of fatalities is acceptable or the best we can do but I certainly don’t. There will be flaws in autonomous vehicles, no doubt. But will there be a net reduction in automobile related fatalities as a result? Like anyone else, I can’t predict the future. But to paint a rosy picture about how our ability to read other drivers is somehow safer relative to AVs is nonsense. It just is. The data doesn’t support this argument. And separately, if we’re talking about will happen in the future, the notion that humans will ultimately prevail over AVs because for safety reasons seems preposterous. We can debate the “when” in terms of AVs but debating the “if” seems pretty out of touch with the way society has progressed with respect to our willingness to depend on technology.
And your over-enthusiasm for AV doesn't extend to the human factor. We all have our own blinders ;)
>The notion that we are safe because we can read humans is not substantiated by anything.
That is your own misinterpretation. I did not read the comment that way.
>If AV risk is so obvious as you claim then why does the enormous amount of data show that AVs are involved in the less accidents and lead to less fatalities than cars operated by humans on a mile per mile basis?
What you mean when you say AV, is actually "AV + Human". We're running controlled experiments, limiting the unknowns, and we're mandating a human be present - because the current AV technology sucks.
> We can debate the “when” in terms of AVs but debating the “if” seems pretty out of touch with the way society has progressed with respect to our willingness to depend on technology.
People used to say that about flying cars 40 years ago.
That future may not even want or need cars.
The other, more realistic future is one where human-level AVs are always just out of reach. Where causes of accidents are just as intransparent as with human drivers, we're all a little bit safer but a patch can cause catastrophic divergent behavior due to the innate non-linearity of the problem.
That future may not want or need cars either but it may not even be considered.
That sounds pretty unlikely to me. "just out of reach" is a very narrow band, and to be stuck there despite decades of improvements would be pretty strange. I think there are only two likely outcomes for the next 50 to 100 years: either we don't even get particularly close, or we'll slowly but surely surpass the median human driver.
Agreed, 100%. Instead of AV, we could focus on remote presence and other tech to make cars and/or daily commute unnecessary.
The fellow in front of me steps in front of the car which suddenly accelerates and hits him knocking him down. Immediately before impact it decelerates and comes to a stop.
The fellow ended more shaken up than anything else, but it was a very near thing and a reminder that it's precisely that assumption of the predictability of human drivers that fails in many crash situations.
In other words, your judgement that you understand what other drivers are going to do is more fragile than you imagine.
On my 2018 Lexus, AEB won't come to a complete stop, but it will definitely slow from say 40mph to 20mph, so hitting a pedestrian at 20 hopefully has a much better outcome than hitting them at 40+.
This expresses well the perceived difference between catastrophic human and machine errors.
I have two cars, a ’99 4Runner and a Tesla Model S.
When driving my S, I’m at an involuntary elevated state of alertness compared to my 4Runner. I think it’s discomfort at driving a car that might suddenly kill me through failure modes that I can’t possibly anticipate.
While I think my S’s technology is much more likely to result in my safe travel than my dumb iron 4Runner, my subconscious maintains a different assessment.
I expect this dichotomy in perceived risk is generational and will disappear after a decade or two of lower accident rates in autonomous vehicles compared to human drivers.
I can tell that I already have an inordinate amount of trust in the system several weeks in on boring highway driving.
This.
It doesn't matter if autonomous cars are objectively safer, 5x, 10x, 100x, doesn't matter. They have to be subjectively safer, which is part PR problem, UI problem, and part human stubbornness problem.
I think we trust humans to make a reasonable decision in a trolley-problem scenario (rightly or wrongly). Or rather, we trust the human we're in a vehicle with to value their own life, and thus our own, more than those outside of the vehicle in most scenarios.
I expect there is research to investigate this, though I wouldn't know where to begin looking.
*I've definitely had a few bad drivers, of course.
And I do think twice about taxis.
Assuming some utilitaristic equation for minimizing total injury, an AV bus probably won't swerve, and probably won't brake significantly except if colliding with a heavy or fast vehicle.
[1] citation needed.
In summary: in a study done in 1980, 93% of Americans thought themselves better than an average driver.
Except for when they haven't slept, or aren't paying attention etc.
That is not a good baseline to compare self-driving cars against. That would be horrific.
We are not rational. We do not respond well to a car running straight into a barrier or stand-still vehicle at 100 km/h without even attempting to brake (such as the failure modes that Tesla has demonstrated) even if it does well most of the time and has better statistics than an average human.
And further, as noted in the thread, a good part of driving is assessing other vehicles and vehicles behaving oddly (even if it is objectively better in isolation) is really bad and will increase the risk of collisions.
I think google have experienced this, that people do not respond well do the driving technique of their car since it doesn't behave as a human - not that it does anything wrong.
That is the rational choice given the human psyche.
We can barely even convince people that vaccines are good.
I believe a more careful approach will get broader adoption and likely save more lives.
== Obviously, not everyone can be above average. Exactly half of all drivers have to be in the bottom half when it comes to driving skills and safety. ==
Maybe, but the bottom half is not necessary drivers worse than average.
I do not know how you would calculate "average". But there are people on the road that could pull down the average a lot. So that more than 50 percent are better than average.
They sure do claim that to elementary school children. But using mode sounds absolutely insane to me. Does any field actually do that?
Maybe you could carefully bucket/smooth your data and then use the peak, but that's a far cry from taking raw numbers and looking at the most common.
The nice thing about LIDAR is that you can gather that depth/shape information and with sufficient calibration map the shapes in the camera image to the depths in the LIDAR image. You can do the same thing with two cameras so I'm not sure why LIDAR would be preferred here.
The stereoscopic effect is very poor on driving distances, doesn't help us that much. Primarily we use clues form the environment to gauge distances. We also have to focus our eyes to the correct distance - that also tells us how far an object is.
Primarily we have had an insane amount of training to understand our world. An understanding that a self-driving car will never achieve unless singularity happens. It might not need that, but it will need other ways to compensate for it.
The biggest problem with relying on visual data is that you get very noisy data. Poor lighting and reflective or glossy surfaces cause problems (I'm not sure what current state of the art is, it's been a few years since I looked at the research).
As far as I understand the big advantage of LIDAR is that you get nice and clean depth data and it's not so computationally expensive.
LIDAR has its advantages, like precise 3D positions under ideal conditions. However there are downsides as well. Cost is a big one, but that's becoming less of a issue over time. Another is sensitivity to rain, fog, blowing sand, etc.
A complicating factor is human driven cars will assume cars to act like they have human limitations. So higher speeds when humans can see well, and low speeds when humans can't.
Not sure Tesla's current sensors will do it, but seems like camera based systems are likely to be quite competitive with LIDAR. Maybe instead of 3 forward cameras, 6 or 8 so there's overlapping views (for stereoscopic vision), handling failures better, and allowing a narrower field of view at a higher zoom.
More range will be a huge help, that way an autonomous car can slow more gently when uncertain and drive more like a human. After all superhuman reflexes aren't much use if you get rear ended all the time.
I almost spilled my beer about to comment that a camera or two are equally if not more powerful than lidar. To me personally, Lidar feels like an incomplete solution to 3D mapping when high res images from a smartphone camera can provide so many more data points from different angles.
My thinking was vehicles should have an idea where other vehicles are without the need for comp viz. Like beacons saying "hey Im here." And we can try to calculate relative direction and distance. The vision bit should ideally come in to validate and confirm other things like road signs etc. Ideally we should add that data to mapping software andnthe car should know these things without "seeing it".
Minor nit but humans can't drive--certainly not safely--in all conditions. You can certainly get to a point in fog, blizzards, and even very heavy rain where you really would like to get off the road if possible. (Not always possible of course and in snow particularly, pulling off to the side of a highway isn't a great option.)
There are certainly other senses (lidar, ultrasound, radio signals) that robots could avail themselves of that would be helpful even in conditions where vision also worked.
Computer vision could be making those go-stop decisions for you, much more effectively than human drivers.
Heck, imagine a "smart" parking lot that tracks its available spaces and communicates with your car. You enter the parking lot and hand over control, and the car and lot work together to park you safely in the best available space.
It does lack any spatial indicators to the driver, though. It just plays a beep, but it doesn't seem to play the beep from the side of the car the danger is in, nor does it beep louder when they're closer. Still, it works remarkably well. Reliably detects pedestrians and vehicles (even while I'm moving, which is impressive, because it doesn't go off for parked cars when I start moving).
I think it's the Dodge ParkSense, but I bought the car used so there's a possibility it's something aftermarket.
It's saved my ass a number of times. I'm in jacked-up pickup truck country, so it is often impossible for me to see pedestrians about to walk behind me because of the lifted truck, and likewise the pedestrians can't see me until they're behind me.
https://spectrum.ieee.org/cars-that-think/transportation/sen...