Uber Neglected Simulation Testing on Self-Driving Cars, Insiders Say
theinformation.com
theinformation.com
He's describing, incidentally, how professional drivers drive.
Pilots of small planes do the same thing. After a while you can just feel small accelerations and change of directions without needing to check some of the instruments.
If the roads were perfect, you wouldn't necessarily want them (day to day), because it diminishes the minute vibrations that make their way up to the pedals, into the steering wheel, etc., and those things give you feedback into what the car is doing that are invaluable.
Of course, the roads aren't perfect, and most of us aren't doing high-performance driving on them, and we don't need anywhere near that degree of feel, so the benefits of driving by wire far outweigh the cons of analog driving.
Ever notice how your butt tries to move more than your hands and feet when making a sharp turn at a moderate speed turn, and how you instinctively correct for it in your seat by leaning or shifting around? Know the phrase 'Fly by the seat of my pants?' Pretty much all of that rolled into one bundle of instinctive attunement.
You can get a lot done (and get wonderful demos) through research vehicles. The issue with sim is similar to the issues with AVs as a whole - getting to 90% is much easier than getting to 99.9999%.
Also, there is this weird thing in AV (and specifically in-house sim) where pointy-haired managers think "if we put 30 SWEs on the problem - we'll make progress!". While raw eng horsepower matters in some software problems, when you are on the line between research and engineering, more SWEs might actually make getting a robust product up harder.
Another thing we've seen is in-house efforts overfit solutions. They don't really have a wide viewpoint (nature of how secretive the space is), and therefore go down paths which are just plain wrong but no one is there to challenge them.
Nobody (well, theoretically someone, I'd like to be proven wrong) has a self-driving algorithm which is formally proven. If we can't get, to use your example, web engineers to formally prove their web applications, when they have all these structural advantages, then how can we expect something unproven (har, har) to be formally proven? So there needs to be an extensive QA test suite.
And for all of these other relatively-mature engineering sectors like web engineering, how many of these companies are really investing in QA? In strong and comprehensive test suites, in security reviews, in performance testing and UX scoring before release? Almost nobody, right? I mean, the only reason why society can get civil and biomedical engineers to submit to strict quality control is through regulation, but no regulatory body has any real idea of how to apply that to software. And autonomous driving systems are supposed to be different?
Mythical Man-Month came out 38 years ago. The Pareto Principle, not so much as being explicitly a 80/20 split but about how the vast majority of the effort is needed to achieve actual completion rather than just a proof of concept, was originally coined in 1896. Suddenly it doesn't apply to Uber?
Is it really so surprising that it's so difficult to find senior practioners in these areas when few companies are committed to that level of quality, and the companies which are tend to be so successful that they can afford to pay compensation at rates so far above the rest of the market so as to ensure that few of their senior engineers ever leave?
Society usually doesn't need a formally proven cat pics sharing app, and most web apps, including facebook, are that level of application. They would rather pay a cheaper price than to have a perfect expensive thing.
You'll also notice that as organizations get bigger, they do invest more in testing, QA and regression detection.
Needless to say, this is expensive. I’ve heard estimates that it costs roughly $1000/line of code.
The point is that there are methods to increase reliability, but there isn’t the financial incentive to do so.
If there is only a single developer working on a project, that developer can manage much much more code if there are solid tests behind it than if she was just winging it with every commit.
At some point the developer starts to forget the code they write and starts relying on the tests they built long ago.
I always assumed people would integrate an external simulator like CarMaker rather than build their own.
I'd be exceedingly interested in hearing about your product when it comes available (although my interest is in simulation for maritime autonomy rather than automotive).
That being said, I still want this tech on the road ASAP, not because it's perfect, but because human drivers are So. Fucking. Awful. They kill thousands of people a year due to negligence, incompetence, and occasional malice. Plus, the additional sensors available and the software response time mean autonomous drivers can outperform even sober, attentive, skilled human drivers in most situations.
A couple of weeks ago, I saw someone talking scary stuff about what someone here called "semi-autonomous murderbots" in a FB post that appeared on my feed immediately after several posts about a local accident, where a bicyclist was killed by a road-raging driver, who then hid/destroyed his car (the murder weapon still has not been found, even though his trial is now over). If that jackass had a self-driving vehicle, the bicyclist would still be alive today.
So are big corporations going to do kind of a careless job testing live vehicles in a rush to market? Yes. Will people get hurt? Yes. Is it worth it? Yes.
Hey, another human driver on the freeway here. Sorry, but I don't want to be killed because you want to play with some toys. It is completely unethical and immoral to ship unproven tech that has the potential to kill people.
I'd suggest running an experiment with OPT-IN volunteers in a neighborhood or small city for a year or however long it takes and then coming back with data.
Drunken or distracted driving, by itself, doesn't kill. It leads to the behaviors that kill, like lane drift and speeding and running lights. These are all things that autonomous cars handle really well.
That's the thing. Fair enough, autonomous cars cannot be drunk or sleepy, but they have _others_ issues that "are most easily addressed by human driving". A human might be distracted, but it won't see a bicyclist for 10 seconds while thinking "I shouldn't do anything in case it's a false positive". Human driver won't directly swerve in a concrete wall because it followed to wrong white line. Human driver will predict better the trajectory of other humans, especially infants.
All in all, I do agree that automation can bring improvements to road safety, but in the current situation, it is not clear at all if this can counteract the automation downsides. It will surely improve, but hastiness might not be the way to go here...
So it's not nearly as bad as it's made out to be. the fatality was a consequence of multiple human errors, both consciously and neglectfully. The automation was not in error.
All the arguments about the driver that should have been alerted or reacted are simply not autonomous driving so unrelated.
Again, I'm not saying that automation will never be a good replacement for human drivers, but as of right now, it is not clear at all if its positive aspects counterbalance its shortcomings.
After reading the Atlantic's Waymo article, I'm more convinced that Uber is doomed in this race. They're not investing properly in simulation, and Waymo is getting an enormous amount of programming done based on controlled simulations that Uber isn't - billions of miles, and repeatable scenarios.
Tesla is really in the game, I think, because they're able to collect so much data from the AutoPilot systems currently on the road, even if they're not in use. They still get all the telemetry for complex real-world situations.
But at some point, you have to let experimental software out on the road, to learn from the real world. Simulation is insufficient. Or, we simply refuse to allow auto-autos at all, permanently establishing that humans will always be more capable, without evidence. So it's a question of when, not if.
There's a couple of currents going in the arguments here. Some say "not yet", and some say "never". I think the "never" people are nuts. Auto-autos that broadly outperform human drivers safety-wise seems achievable to me in the not-too-distant future.
Yes, if they are drunk or clearly driving recklessly. Otherwise, it's an accident. Their insurance will pay, and they will go on probably with some emotional scars but nothing really in the way of "punishment"
If you want to simply reduce vehicle related deaths, you could address this through various ways like improving road safety/visibility, adding more driver assistance aids. Or heck, design a protocol to disallow cars from starting with a drunk driver or without a seat-belt, etc etc. Maybe people who get into even minor accidents need extra training, etc. etc. There's so many low hanging fruit here..
Disclaimer, I have never been to Nashville, but I've heard about it from people who have been there and did a bit of reading 5 minutes ago.
http://soundtransit3.org/calculator
It's a sizable chunk of transit. It's all about the priorities of a city.
Singling out the "previous high profile fatality" is magical thinking. Drawing statistical "evidence" from it is pseudoscience. Meanwhile, a quick googling shows over 800 cyclists killed per year by cars, and nearly 50,000 injured, in the US alone.
Obviously there is going to be a disparity in the number of deaths. What is important is the number of deaths per mile.
Further complicating matters is the fact that most of the miles being driven by these autonomous vehicles are being done with human operators trained to be alert and the unique quirks of an autonomous vehicle. They will likely be much prepared to stop an accident from occurring when the car makes a mistake than an average consumer.
Autonomous driving may in fact be revolutionary and reduce the number of traffic deaths dramatically, but it is still early enough in development that there isn't any evidence to bank on anything at this point. There certainly isn't enough to rush the tech out the door either on the public either.
I don’t doubt it will get there and, when it does, these profit-oriented corner-cutting behemoths had better have a shit ton of data to prove it. when they’ve proven the premise you’re taking for granted.
Then I’ll happily see this tech widely deployed.
Second, yes, they can't come soon enough because traffic is a meat grinder: it's human sacrifice to the gods of speed and convenience.
However, you do not test dangerous industrial equipment in the street with kids and dogs and bikes crossing the freeway, etc...
You just don't do that. It's insane.
A big robot with a couch on it should never travel faster than it can reliably stop quickly enough to prevent any conceivable collision. That's the hard limit. Until we can do that, any machine heavy enough and traveling fast enough to kill is a killbot.
First Law of Robotics?
But that said, aren't you holding automation to a different standard than humans? Is it okay to have the big robot with a couch travel quicker than it can stop to prevent any conceivable collision, just because a human is in charge of the brakes? Because that's what cars do now! I can conceive of situations where it's not possible to hit the brakes on time in an ordinary car - hell, I once hit a bicyclist when I was only going about 15mph, because she basically appeared out of nowhere before I (or she) could stop.
Cheers!
> But that said, aren't you holding automation to a different standard than humans?
Yes and no.
Humans drive preternaturally well en mass. So much so that when I realized it I began to suspect that "Guardian Angels" might be real. Then I learned more about human unconscious motor neurology. Driving performance is one of the few places where the unconscious mind is blatant. Nevertheless we have set up a system that amounts to a "mayhem lottery". When I say "they can't come soon enough" I really mean it.
Now eventually auto-autos will be super-human in capabilities and performance. They don't have blind-spots, they can use radar and lidar and mass sensors, etc. They will also have the benefit of perfect hindsight across the whole fleet (if we set it up sensibly!) Further, traffic infrastructure will evolve to facilitate automated traffic, with sensors and comms embedded in signals and even the road. They'll be able to share knowledge in real-time of local road conditions and keep track of people and (non-self-driving) vehicles around them cooperatively. Sooner or later, it won't be possible for someone to surprise one of these things.
And all this will happen at an ever-accelerating rate. (Which means that if you just wait, making them gets exponentially easier!)
In the meantime, it's not okay to use public space to test crude early prototypes. People have already died.
I was truly shaken and upset when I read that the Uber car that killed that woman crossing the road with her bike had determined 1.3 seconds before it struck and killed her that it needed to initiate emergency braking, yet the emergency braking system had been disabled because it didn't work well enough. The machine knew it had to stop, it had the capability to stop, but it was prevented from stopping by bad decision-making driven by the desire to put unsafe machinery on public roads.
That's tragic and truly pathetic, and I would say the makers committed murder. Certainly, if you or I built a robot and sent it out into the world and it killed a person we would have to defend our actions with something better than, "Well it's not perfect. Give it some time."
We could make a e.g. self-driving golf cart that could safely take my mother to her doctor's appointments and back. She's got dementia so we don't want her to take the bus even though she's physically able. It would have a top speed of 3~5 mph, and obviously it would have to travel restricted routes where it wouldn't interfere with traffic, and a there are a host of other concerns. But it would work, we could build it today, there's a market for it, and it would have a really hard time killing anyone in a collision. It would be lightweight as well as slow-moving and covered in airbags inside and out with pre-impact detection and triggering.
Once you start to think outside the box (sorry) there's all kinds of scope for incrementally automating traffic.
Trying to go straight to imitating full-on automobiles with human drivers on public roads is insane, it's hubris. And again, people have already died. So this is not a theoretical concern. There's been a funeral man.
> Is it okay to have the big robot with a couch travel quicker than it can stop to prevent any conceivable collision, just because a human is in charge of the brakes? Because that's what cars do now!
It's not okay for robots to kill people just because people do, no.
(I think a sane transportation network would have at least four interlocking but separate systems: pedestrian, bike, car/truck, & rail. Cf. "Pattern Language" C. Alexander. But I'm getting off on a tangent.)
> I can conceive of situations where it's not possible to hit the brakes on time in an ordinary car - hell, I once hit a bicyclist when I was only going about 15mph, because she basically appeared out of nowhere before I (or she) could stop.
That sucks! Sorry dude.
But yeah, what I'm saying is a robot moving in public should have a hard limit on its kinetic energy (half mass times velocity squared, I just looked it up.)
A robot moving in public with enough energy to injure, maim, or kill is an extraordinary hazard at our current state of development.
If you want to make a safe one, you can keep it lightweight, or you can slow it down.
The baseline here really isn't that high. How many miles have you driven, and how how many accidents have you been in?
For better or worse, our laws acknowledge that being alive has unnecessary risks, and accepting the social order means accepting the standard of risk caused by other people.
But in this case, the outcome of interest is quite rare. Despite the high numbers of people dying in car related accidents each year, the number of accidents that occur for the total number of miles driven by US drivers is very small. So in order to draw inference about whether autonomous vehicles improve on that rate, you need a huge amount of miles driven by autonomous vehicles. None of the companies have really approached that level of testing, to say nothing of the fact that most of the miles driven so far aren't on similar, real world conditions that US drivers face, making any statistical conclusions even more difficult.
I don't believe the Uber accident applies, because it was caused by human error, not the driving automation. The emergency braking was deliberately disabled (it would have prevented the accident), the detection system was not programmed to inform the driver, and the driver was not paying attention. If the Uber accident is disregarded, then we're at 0 fatalities, or infinitely perfect automated driving. Which is, of course, BS. But it's as rational as the argument others are making.
Given the inadequate data set, I'm relying on inference, not experience.
Everything "has the potential to kill people" if you put enough of it in a box and drop it from high enough. The question is how much potential, and compared to what. If we want to use strong moralizing language like this, we could just as easily go the other way: "It is completely unethical and immoral to delay technology that has the potential to save lives." That would be true in exactly the same ways, which is to say, it's complicated and we need to avoid moralizing.
No, you couldn't "just as easily" go the other way. Those are not identical positions. My position promotes caution over over-confidence. Further more, the "potential to save lives" is not the exclusive domain of autonomous vehicles. So nobody is "delaying" saving peoples lives. Various methods that can be utilized to reduce causalities that do not rely on technology that has not been shown to work in a conclusive fashion. Planes were unproven tech at some point, and people rightfully refused to fly in them, and they aren't as unsafe now. Technology gets better, and we can hope that autonomous vehicles would get better too.
But since you aren't, I'm guessing even you don't really believe what you're saying. Which is good, because a key assumption of your theory is proven wrong by the article you're commenting on.
Yes, these vehicles will eventually save lives once their error rate drops below that of a human driver. But there's no reason to think that running over a bunch of real people is necessary for progress until these cars work well in simulations. As the article explains, they get orders of magnitude more testing done in simulation than on the road.
Uber's rush to test things in the real world was not about advancing the technology or saving lives. It was about getting a short-term commercial advantage on the cheap.
The data and experience gathered from real-world testing should (I'm not saying "will") accelerate the progress to the point you mention - "once their error rate drops below that of a human driver".
Let's look at the Uber case, since everyone infers so much from it. One fatality. One. And the program was immediately shut down. And how did it happen? Engineers had deliberately disabled the automated emergency braking system to reduce "erratic behavior", but had no automated notification to the driver. And the driver was distracted, doing data entry for the test as he was instructed to do. If the emergency braking system had been on, it would have stopped the car on time - they have the data for this.
This sure looks like human error to me.
And that single accident put Uber's program drastically behind schedule, relative to more cautious/thorough competitors. Uber paid a market price for its eager carelessness.
We don't have reason to believe that. Even Uber now agrees that more simulation should have been done, and they're trying to catch up. All the necessary data for this could have been gleaned from instrumenting human-driven cars, and then testing the object recognition, object avoidance, and other driving algorithms against the resulting data.
Uber's Arizona death wasn't "human error". It was negligence. They also weren't "behind schedule" in any meaningful sense; some exec picked a too-aggressive goal, and people cut corners in a vain attempt to meet it. They shouldn't just be paying a "market price". They should be facing charges for criminal negligence leading to involuntary manslaughter.
If Uber was more focused on doing things the right way their program would be much further ahead.
Aren't self driving error rates already below human drivers? The fatality rate is extremely skewed by one incident in which the system detected an obstacle but wasn't allowed to stop the car because it was disabled (so... not a failure of the self driving aspect of that system) and another in which a partial self driving system warned a driver repeatedly to take the wheel, but they did not.
In both instances the problem was a failure to force action by the human drivers (or failed reliance on them). In the former case, the system would have braked, in the second, the system should brake instead of asking for help from the driver.
Meanwhile the overall crash rate for self-driving cars is lower, correct?
I don't think that's correct in general, and especially not so if you compare the conditions that each are driven under. Said a bit differently, the human error rate is in all conditions (rain / snow / nighttime / unmarked roads / gravel) where most self-driving miles are on well-marked city and highway roads in near ideal conditions.
But you're already no-true-scotsmanning your way to giving the self-driving cars unrealistically good scores. Yes, a cause behind the Uber crash is indeed Uber disabling a safety system they thought unnecessary. But we have no reason to believe some other companies aren't also either turning off or just never adding safety systems.
My objection is not to better testing. It's to bad comparisons. Including, as here, with somebody handwaving at data that, at best, couldn't answer the question.
Do you get to decide this? No. Does Uber get to decide this? No. Is this attitude going to produce a backlash against the idea of the technology, similar to the GMO backlash? Probably.
I also agree with GP. We need to take more _calculated_ risks instead of trying for strictly risk-free science. Personally I'll gladly accept more dead astronauts (and hail them as heroes) if they push spaceflight further.
How noble of you.
Self driving cars are a hard problem, a very hard problem. So is CAPTCHA and machines are still not that great at solving them.
The problem space of autonomus vehicles is a large one, and even with how many mistakes human drivers make, I still don't think we're anywhere near the tech to make safe autonomous driving vehicles. We're at a minimum, 15 years off.
I want you to think about all the technology we use every day. Our phones, desktops, laptops, are full of bugs. Hell I found bugs in the ticketing system I buy metro cards from the other day. We see display kiosk crash all the time.
A Tesla car with lane assistance (please stop calling it auto-pilot) recently drove a car into barrier. Another kept driving for hours without hands on the wheel (fixed in an update. Was that driver passed out? No way to know.)
Now you might say, this is different. The engineers working on self-driving tech are at the level of the ones that build bridges, or make pacemaker software (which by the way, some were found to have a bluetooth vulnerability recently) or plane avionics.
Maybe, but Uber certainly isn't. They're a culture of scum shit shop. This isn't a small thing here. It's a huge, huge problem space that no one completely understands yet. Why are there not any self-driving cars on the streets of NYC or Chicago? Because it's difficult, and dangerous and if there were, people would be dead right now.
For all the money we're investing in self driving cars, we could build tech that already works. Singapore and London both have fully autonomous trains; some that arrive at 30 second intervals and carry millions of people a day. America needs to get its normal infrastructure back before we go down this fantasy world of self driving cars.
I wrote more about this a while back:
https://penguindreams.org/blog/self-driving-cars-will-not-so...
These arguments don't make a lot of sense to me:
- Do you think display kiosks would still crash all the time if the industry spent a few hundred millions at improving them? - Do you think the metro ticketing system would have as many bugs if the transit agency spent a billion on it?
If not, why are you comparing very different situations? Maybe we need to use average number of bugs per amount spent on dev instead of average number of bugs per project or per # lines of code
That seems like a huge leap?
What if the driver had a gun and just expressed his rage with bullets?
What if the self-driving vehicle, prior to the rage part, hit a bug and hit the cyclist?
And by self-driving vehicle you mean e.g. Level 5? The driver could not take control to express his rage?
This doesn't seem like a huge leap to me at all. If he'd been driving responsibly, he probably would not have hit the cyclist. And autonomous drivers don't get angry or careless. Your follow-on about imaginary scenarios kind of underscores the weakness of that argument.
And I do agree that a raging driver taking control from the automation continues to be a problem. But that's no worse than the original situation, now is it?
Bugs are not imaginary scenarios. Also, did the US outlaw gun ownership without the rest of the world noticing?
> And I do agree that a raging driver taking control from the automation continues to be a problem. But that's no worse than the original situation, now is it?
I only replied to one point: that you cannot state with 100% conviction that the cyclist wouldn't have died had there been a self-driving vehicle.
You could back down and just concede you really meant to add at least one condition: "if that jackass had a completely autonomous vehicle, with no manual input possible, it is highly unlikely the cyclist would have been killed"
I was out car shopping recently, and even affordable midrange cars (Subaru Crossfire and Honda HRV) offer technology that could have done this, as a standard feature.
For the USA:
37,000 vehicle deaths / year
2.35 million injuries / year
1,600 child deaths / year
8,000 killed in crashes involving drivers ages 16-20
$230.6 billion per year in crash associated costs
If any company can show that these statistics can be reduced (halved/thirded/decimated) in any meaningful way with their autonomous tech, that's when we'll finally reach a new era in transportation. How can anyone advocate for human-driven cars once the implications become clear? Let's say that the Uber autonomous car reduces vehicular injuries and deaths by only 5%, well shouldn't that be enough evidence to say, put that on our roads? I think that the current level of tech is already at a point where it will save lives on average more than a human driver, what more do you need from a regulatory point of view?
Take the UK for example. It has many of these safety improvements, and gets one quarter of the risk of dying in a car accident (one half the risk per mile driven, since Americans drive further).
https://en.wikipedia.org/wiki/List_of_countries_by_traffic-r...
Getting out of a "local minimum" by definition requires things to get worse before they get better - that's what makes it a local minimum. So it requires faith, which is belief in the absence of clear evidence. Faith is always uncertain, which is why people get strident and appeal to emotion when it is challenged.
It's anything but clear
The means are the ends. An organization that disregards safety during product development is not going to have the culture required to actually produce a product that improves safety.
Just read about the hoops one has to jump through to run a simple medical study if you still think we don't need a dose of utilitarianism.
2) right now self driving cars are more deadly than human drivers by several orders of magnitude. Thats the reality.
And, as you mentioned, we need to accumulate billions of test miles before we'll have statistically comprehensive data.
There are currently at least 4 confirmed deaths from autonomous vehicles[2]. While beat is correct in stating that those are not enough deaths to draw reliable conclusions from (and Model S owners are not like the average driver), there would have to be more than 600 million km driven autonomously to be comparable to the death rate of normal cars. Probably significantly more, as the cohort of human drivers that is equivalent to Model S owners is way less often involved in accidents in the first place.
[1] = https://en.wikipedia.org/wiki/List_of_countries_by_traffic-r...
[2] = https://en.wikipedia.org/wiki/List_of_autonomous_car_fatalit...
2. You're misusing statistics. There isn't enough data on autonomous driving performance to make your claim, not enough road miles.
We can't demonstrate empirically, because there is far too little data at this point (and refusing to do the thing to collect the data because we don't have enough data is not a sound moral argument).
But we can infer a great deal. And I've already established grounds for inference...
- Better sensors. Sensors that can detect 360 degrees at all times, and see into infrared, not to mention LIDAR. The sensors can easily outperform human eyes attached to human heads. - Better mechanics. Automation can be directly connected to steering and brakes, as opposed to modulating through human hands and feet and into other mechanical linkages. - Better reflexes. Less lag from observation to action, compared to the eyes/brain/hands system. - No distraction. Human minds wander. - No intoxication. - No sleeping! Falling asleep at the wheel causes many fatal accidents. - No emotions, no anger, no aggression.
So emotionally and mechanically, autonomous systems are clearly superior to humans. The only real argument left is interpretation of the environment. Can they detect hazards more effectively than humans can? Let's look at some hazard categories...
Lane drift - easily solved. Running red lights and stop signs - easily solved. Speeding - easily solved. Maintaining braking distance in traffic - easily solved.
I could go through more, but you get the idea. We're focusing on "What if a pedestrian steps out in front of it and it doesn't recognize them as a hazard?", and ignoring much more common categories of accident-causing situations, categories where machines easily outperform humans.
So "empirical base"? Sure, no basis. But inferentially, the argument for the superiority of autonomous driving systems is very strong.
Your argument is "Sensors are better than humans!", and that's a non-argument, inferential or otherwise.
Detecting sensor failure is child's play. We have over a half-century of experience at detecting and handling sensor failures in computer systems. Hell, I left the gas cap off my car once, and every idiot light in the car flipped out on me. Cruise control stopped working. Etc.
Detecting glare, flashing lights, and other visual interference is likewise well-established technology. And how does the autonomous car respond? By slowing down, staying within safe parameters. Meanwhile, one of the most common causes of fatal accidents is humans driving too fast for conditions, often due to poor visibility. I trust the self-driving car more than the human driver for that.
There's a lot of problems we don't know the solutions to, but that certainly does not mean that these teams don't know the solutions either.
You are commenting in a thread about how a team of those "much qualified software engineers and other specialists" failed to do simulation testing. So yes, there is reason to believe that these companies are failing to address many of these potential problems.
I believe the evidence strongly suggests that autonomous driving will not only catch up, but greatly exceed human drivers in the foreseeable future. Mechanically, automation is superior to humans for simply operating the car. And the large categories of causes for fatal accidents are low-hanging fruit for automation - lane drift, running red lights, driving too fast for conditions, etc. And sensors can observe the environment as well as or better than eyeballs.
The only real point of contention is in interpreting unexpected hazards, like the infamous bicyclist. But interpreting sensor input is subject to continuous improvement, due to programmer experience, log data, and Moore's Law. So even if that point is weak now, there's no reason to believe it will not improve substantially - in fact, it is unreasonable to believe that it will not improve.
Large categories of potential causes for fatal accidents are things which humans very seldom have issues with but computer programs often struggle with like inferring intent and identifying whether something is animate, which can easily cancel out reductions in other types of accident due to (e.g.) better average lane discipline. Especially when you consider the benchmark for fatalities per billion km driven is around 3.5[1] including fatalities caused by humans considered unfit to drive, and humans who are considerably less experienced and safety-conscious than the likely early adopters of SDVs. SDVs have already had one fatality and several incidents where human intervention prevented accidents in a few million miles of low speed, good condition driving. I think we're a very long way from seeing any indication computers alone will ever beat that benchmark, still less a plausible case that the additional deaths resulting from riskier testing programmes will be cancelled out by lives saved if the first few SDV models go on sale a few months earlier.
[1]a few European countries; it's double that in the US, which also suggests that if you want to reduce road deaths there's a lot of other steps that can be taken which don't involve risky technology experiments.
The alternative is to simply never allow it at all, to assume that humans will always be better drivers than computers. Which opens up an ethical dilemma. If auto-autos could save tens of thousands of lives a year - and it seems at least plausible that it could happen - then to reject it out of hand is to cause tens of thousands of innocent deaths a year.
There is a big difference between humans killing others as one off incidents vs one bug, defect or security flaw making a crash possible with the whole fleet. It can be far more catastrophic.
On the other hand, that's the kind of thing that gets addressed by brute-force improvement. Security is a well-understood field, and the main problems we've seen with car security so far have been caused by insufficient effort, not insufficient technology.
[citation needed]
All the current evidence is that humans grossly outperform computers at the task of "driving vehicles to arbitrary locations in all conditions", and that that isn't changing any time soon. Like, it isn't close.
HUMANS: =========================================================
MACHINES: ==
You can talk about your speculative perfect-performance car if you like, but there's no evidence that it will ever exist. Why not give it hyperdrive too?
Driving vehicles to all locations in all conditions? Yeah, humans will have advantages there for a long time to come. Driving on highways and city streets within the bounds of traffic laws and expected velocities? I think computers will be better than humans very soon, if they aren't already so.
And yet, so far, Uber's self-driving vehicles have a higher fatality rate than the status quo.
Sure, it could be improved over time, but the whole point of TFA is that Uber is currently not doing the requisite diligence and testing with their vehicles to ensure safety.
If we're seeing these fatalities with these companies having only tens of cars on the road, what can we expect if they deployed millions?
This is despite automatic numberplate recognition on most freeways, many cars having trackable cell/wifi/bluetooth connections, automated RFID identifiers for toll roads, every car having a unique VIN which is printed on nearly all components and will be registered by scrapyards, and tracking on all mobile phones (presumably the driver had a phone!).
What's known is that the driver, after hitting the cyclist, drove to his mother's workplace, and she left with him. The car he was driving has not been seen since. If it was driven into a remote lake somewhere, it will probably never be found.
One would think a murder investigation would (a) know this, and (b) have the ability to get data from a military or private satellite, both before and after the suspected incident.
Failing that, tracking data from the pairs phones would likely indicate which lake, and probably which part of the lake even.
The driver was convicted of vehicular homicide (max five years), and leaving the scene of an accident. He can't be forced to testify about the location of the car (Fifth Amendment), and I don't know if he was charged with obstruction of justice. His mother was charged with obstruction-type charges, and her case hasn't been tried yet.
This narrative is so pervasive and I must admit a bit strange to me. Millions and millions of people strap themselves into thousands of pounds of metal and then drive 80 down the highway while texting and changing the radio and telling your kids to please stop fighting surrounded by other people doing the exact same thing. And we do that every single day and almost all of us are fine and will never have a 'major' accident. Humans are pretty fucking great at driving.
Besides to say that human are bad (or good) at driving would require some comparison, we're bad at it compared to what? As far as we know humans are the best drivers in universe. There's no evidence that self-driving is currently better than human drivers and there's no evidence that self-driving cars will soon be better than human drivers and there's no evidence that if we allow tons of self-driving cars on the roads then they will be better than humans. It's just blind hope, it really makes me want a word for whatever the opposite of a luddite is, instead of a belief that we can destroy machines to benefit people we believe we can destroy people to benefit machines. It's all twisted up.
If you want to be an Uber or Tesla crash test dummy and walk/bike across a test track while they see if their vehicle can avoid you then by all means go for it. But don't just foist that upon average people.
If autonomous cars can drive at the skill level of average drivers - and not get drunk or angry or distracted by forwarding cat videos - then they will significantly outperform humans.
Additional evidence in the form of reason... autonomous drivers have better sensors than humans. Human drivers have limited vision, that only points in one direction at a time. Mirrors offer some slight augmentation, but. Ever backed out of a parking space in a crowded parking lot and gotten into an accident because a car you couldn't see before was suddenly right behind you? Common situation. Autonomous drivers won't have that problem. They can see out the back and to the sides, 360 degrees at all times. (I'll note that some of the first deployed autonomous tech was cars that can parallel-park themselves.)
Additionally, autonomous systems have better reflexes. Remember the two second rule? That's to deal with the delay from your eyes to your brain to your hands and feet. Machines can physically respond to changes in situation faster than humans.
Now, let's level up. Imagine a system where autonomous vehicles are transmitting short-range telemetry to nearby vehicles, letting them know their status, including velocity. They can relay back emergency status info - so the car that slams its brakes to avoid a pedestrian doesn't cause someone six cars back to rear-end the car in front of them. Merge-related accidents (related to parking-lot problems) can be reduced to nothingness with telemetry. Think about how Waze is doing this already with driving data for their mapping software. It'd work better with less human involvement.
I can go on and on, but really, I'm gonna throw a bullshit card here. Fear of autonomous vehicles is magical thinking trumping reason. Humans do magical thinking all the time. We think safe things are scary, and we think scary things are safe, for symbolism rather than fact. That's why people in this discussion used the phrase "murderbot", but not "murderer".
As long as you use the word "if" you can say pretty much anything you want.
>Additional evidence in the form of reason... autonomous drivers have better sensors than humans. Human drivers have limited vision, that only points in one direction at a time.
That is not evidence. It is your own opinion. Nothing of the sort has been proven to work reliably.
>Autonomous drivers won't have that problem. They can see out the back and to the sides, 360 degrees at all times. (I'll note that some of the first deployed autonomous tech was cars that can parallel-park themselves.)
They can't "see". They can only sense what they have been programmed to sense. The envelope of what they can process is tiny. Sensors have flaws, humans programming those sensors have flaws. The human eye and visual cortex has been evolving for millions of years.
>I can go on and on, but really, I'm gonna throw a bullshit card here.
The biggest "bullshit" here is that you are claiming fantastical things that autonomous cars can do that have not been proven in the slightest. You implying that other people are stupid for being apprehensive about unproven tech is quite rich...
"A self driving car could kill me or others at any moment, and whether that happens is completely outside my control" is a reasonable fear. People may over-weight it compared to more familiar dangers that are in fact greater, but that's not magical thinking.
Magical thinking is more about believing in a causal relationship between unrelated events. Astrology is an example, but also things like "if I don't shave my beard my hockey team will win" or "I thought something bad about my kid and then they got hurt, I'm a terrible parent".
There is exactly zero proof that self-driving vehicles are better than drunk drivers. There is exactly zero proof that self-driving vehicles are better than angry drivers. There is exactly zero proof that self-driving vehicles are better than texting drivers. There is zero proof they will soon overtake any of those categories.
>Additional evidence in the form of reason... autonomous drivers have better sensors than humans.
Teslas have to ignore stationary objects because they can't accurately estimate the height of those objects. My 'sensors' know when a tire shred is on the ground and about 2 inches high and when a bicyclist is on the ground and about 5 feet high.
>Additionally, autonomous systems have better reflexes. Remember the two second rule? That's to deal with the delay from your eyes to your brain to your hands and feet. Machines can physically respond to changes in situation faster than humans.
Watch the video here: https://www.bbc.com/news/technology-44439523
I have driven while texting, I still sometimes drive angry and I have (unfortunately) driven drunk in the past. I have never made the very simple mistake made in that video. Where is the quick mechanical reflex you are leading me to believe exists in self-driving vehicles?
>Now, let's level up. Imagine a...
No. Your imagination is running wild imagining something that doesn't and won't in the near future exist. Your imaginary car not only is covered in sensors, but can p2p communicate with the cars around it, has no lag in processing complex input, can behave itself among human drivers, etc. None of this actually exists, none of it is actually out there and we really don't have good reason to expect it any time soon.
>I can go on and on,
I believe you could, but it's just your fantasizing, it's not a realistic portrait of what is actually happening.
>I'm gonna throw a bullshit card here.
Going to? What about the several you've already sent my way lol.
>Humans do magical thinking all the time. We think safe things are scary, and we think scary things are safe, for symbolism rather than fact.
Exactly! That's why people in this discussion called humans "So. Fucking. Awful." at driving and then wrote three paragraphs imagining a utopian, non-existent self-driving system which magically solves all problems and is really quite safe. Before diagnosing magical thinking in others you might attempt detecting it in yourself.
Your video is an interesting example. That is not a fully autonomous vehicle, but rather an assisted one, one that expects human engagement. In the example case, it failed because of human error. Now, there's a good argument that semi-autonomous vehicles will lead to additional human error, but that's not saying what you're saying.
If you live in an icy climate (I do), and have been driving for a long time (I have), then at some point, you would remember the transition to anti-lock braking systems. Before them, we had to learn to pump brakes in certain ways on icy roads, to prevent locking. Now, just push down the pedal, and a computer does it for you - one that senses brake lock before it happens. It made winter driving much safer.
Your criticism of Tesla sensors is flawed. You're assuming that their sensors will not get better. That seems an unreasonable and unlikely conclusion. And you're ignoring my point about 360 degree sensor coverage. I'm an excellent driver, but I find myself in danger regularly because I can't see everything.
Magical thinking, oh yeah.
You're really contorting yourself here, just give me an apples to apples comparison of human driving safety to self-driving vehicle safety and I'll be happy. (Hint: none exists)
>That is not a fully autonomous vehicle, but rather an assisted one, one that expects human engagement. In the example case, it failed because of human error.
No no, no word games. That is a Tesla marketing/liability line. What you see in that video is the current state of self-driving. And a product marketed as self-driving which can't steer around a stopped vehicle in the lane isn't really self-driving at all. Besides if what you're talking about is Level 5 type self-driving systems, you have nothing supporting the idea that we get there soon or that current self-driving systems will get us there.
I mean besides if drunk drivers could navigate around that car then you're basically admitting that drunk drivers are still better than the current state of self-driving technology. (something I agree with)
>ABS
...
I think you're misunderstanding me. I'm not against technological progress which makes vehicles safer. I'm against the understanding that self-driving technology is making cars safer.
>Your criticism of Tesla sensors is flawed. You're assuming that their sensors will not get better. That seems an unreasonable and unlikely conclusion. And you're ignoring my point about 360 degree sensor coverage. I'm an excellent driver, but I find myself in danger regularly because I can't see everything.
And you're assuming that their sensors will leap and bound ahead magically. You're trying to make me argue that humans are better drivers than the perfect system that exists in your head. I'm saying the perfect system that exists in your head is irrelevant because it is not going to become reality.
As for drunk drivers navigating around that car? There's an intersection a couple of miles from my house that has a stoplight. Southbound traffic is approaching from highway speeds, climbs over a hill, hits a 40mph zone, then the stoplight is a couple hundred meters. There have been multiple fatal collisions there by, well, drunk drivers, slamming into cars stopped at the red light at highway speeds (or faster). I'm terrified any time I'm stuck at that light after dark, especially on weekends.
Of course, an autonomous car would detect the speed zone change and the red light.
>Of course, an autonomous car would detect the speed zone change and the red light.
Would it? I think it's actually unclear that it would and besides this experience of yours is filling in for the place in your comment where you should be showing me the data that shows self-driving cars are safer.
Assuming that it isn't a solved problem is kind of irrational. If your argument is "Well, maybe the car won't see the traffic light it already knows is there", then perhaps your argument isn't very good.
Then you're kind of going against your original comment where you said you wanted self-driving vehicles on the road asap. All you have to do is google: Tesla Speed Sign and you'll find tons of problems and examples of Teslas misreading signs. Here are some examples:
My AP1 car has interpreted a 35 mph sign as an 85 mph sign.
---
My understanding is AP1 visually identifies speed signs, but can screw up (reported that our highway 80 is misread as 80 mph)
---
Addendum: a 45 mph zone, permanently signed (white/black sign large - not orange/black sign small), which by the way AP2 completely ignores. See error-ridden, and in this case an example that creates a safety/financial hazard not present with AP1.
---
And on some highways in the area, especially where there has been recent construction, the speed limit database thinks the speeds are 45 or 50 MPH in sections - when the posted speed limit is supposed to be 60 or 65 MPH. If the car is running under AutoSteer when it hits one of these sections, the car immediately slows down, because AutoSteer won't exceed the speed limit by more than 5 MPH on non-highway roads.
https://forums.tesla.com/forum/forums/new-car-has-trouble-re...
>(and doesn't have an up to date on-board map of them)
This is also something that doesn't always happen in real life. Here's a quick example just from googling:
The "new" speed limit database distributed last summer (from TomTom?) has numerous areas with missing speed limit data or incorrect speeds (both high and low), which causes issues with AutoSteer. And if this happens while driving at highway speeds, AutoSteer will try to quickly slow down in the middle of high speed traffic. While you can disengage AutoSteer and retake control, the other option is just to temporarily leave AutoSteer/TACC running, and manage the speed yourself until the software sees it's back in an area with the correct speed limit.
https://forums.tesla.com/forum/forums/fooling-speed-limit-de...
>But I'm assuming that low-hanging fruit like reading traffic signs and signals is a solved problem.
You know what they say about assumptions lol.
>Assuming that it isn't a solved problem is kind of irrational.
One, it has nothing to do with rationality. Two, it's actually not a solved problem.
>If your argument is "Well, maybe the car won't see the traffic light it already knows is there", then perhaps your argument isn't very good.
Or maybe the state of self-driving technology isn't actually as advanced as you believe. I'm glad we can agree that these systems have no business being on the road though.
So we need this:
-real numbers of AI to have a meaningful discussion on real statistics
-we need good statistics, so not compare different car types and driver ages/categories
-IMO we have a low hanging fruit related to preventing drunk drivers to use a car, a system to detect a driver that is not looking ahead for more then n seconds, someone needs to handle this, it would improve the numbers a lot
-when we compare say a chess AI we compare it with professionals so it would be more honest if we want to say that self driving AI is better we should compare with a driver with experience that is not drunk
-we should not RUSH this because it will backfire,wait for hardware and software to be ready so if a concrete object is in the front of the car the radar can detect it's position and the cameras and image recognition can detect what it is and not ignore it like it is a road sign, hardware and software should be ready to detect signs, bottles, people, bikes, animals as good as humans, if is not as good as a human then we already lost since a human also many more "features" like intuition,
One interesting point of the development is that every failure is an opportunity to learn. The same cannot be said for human drivers. When the automation doesn't detect a hazard correctly, they can analyze it and tweak it so it detects better next time. Do humans do that?
Because it feels to me they are faking it to buy time until some new hardware is ready. They have the data but the hardware is to slow so they can't analyze it in real time so they ignore it, even if say they want to put a concrete related update it may be too intensive and it won't run in current hardware.
A "auto" can learn from a failure, and every car using that AI system learns. 1 car results in thousands of systems all learning together.
That's the win of cooperative cybernetics, and something we humans haven't grasped very well at all.
I wonder how many drivers learned from that one?
I am not against the self driving car idea, I can't drive because of my eyes and my son also has bad eyes(not as bad as me ) and epilepsy so he probably will not be able to drive a car, I am disappointed because the startups fucked this up by pretending that they can solve the problem with the crap weak hardware and software they have, as long as you can't detect the exact object position and use image recognition to correctly tag all objects and compute all speed vectors this can't be solved and it is clear the current hardware Tesla and Uber use is not enough.
Self driving may the future but is not the present and it is not the near future either.
Imagine an AI doctor,would you feel good if you know is better then a drunk doctor?
Would you send your child with a self driving car that is better then a drunk driver but not then yourself? ,after 1000 kills the ANN would MAYBE adjust some weight somewhere and the changes will increase a bit
I agree self driving is not the present, but I think it's the near future - 10-20 years.
And to be absolutely clear - I believe that full-on self-driving technology, with no human assist or failover in normal on-road cases, will be a better driver than me within the next decade. I've had my driver's license for 37 years, and I love driving. I've had two moving violations in my life (last one over a decade ago), haven't been in an accident in 17 years, and haven't been in an injury accident ever. I'm a really good driver. And I'm kind of neurotic about being in a car with other people driving - it makes me nervous, even when I know they're good too.
But I'm totally looking forward to the day I can trust my car to do the driving for me, knowing it's better than I am.
The AI needs/has advantages like you said but those are needed to compensate the big missing part , the human intelligence, I am afraid of stupid scenarios where say a bridge would collapse and all the self driving cars would continue jumping of the bridge because this exact bridge colapse was not programmed into the car.11
Until most of people will afford a self driving car you would still have to face the drunk, or the ones that text so you will not remove those of the roads.
These guys made a remote video face-and-eye tracker that they sold to automotive companies wanting to research different dashboard designs -- they would look for blinks and head-nods to measure how fatigued a driver was, and to optimise the dashboard design to keep the driver awake.
I'm sure they are now selling their technology to automotive OEMs for placement in driver-facing camera ECUs.
Their technology worked OK back in 2003. I'm sure it is very robust and mature by now.
And so on. There, of course, are many examples of things in which computers have outpaced human capabilities, and it's certainly plausible that some day this could apply to cars. But it hasn't happened yet and it's also more than plausible that none of this will happen in the lifetime of anyone now living.
But like all new technology, it's improving. People aren't.
Its arguable that self-driving cars are better already. What we need to do is, secure the roads so they won't get confused (fence them & remove pedestrians/bikes). I know, we can't do that everywhere but we could do it for long-haul etc. Like freeways already do.
I think the vast majority of human drivers are alert and sober. It's the long tail of drunk / aggressive / distracted drivers that causes crashes.
>Its arguable that self-driving cars are better already. What we need to do is, secure the roads so they won't get confused (fence them & remove pedestrians/bikes). I know, we can't do that everywhere but we could do it for long-haul etc. Like freeways already do.
I mean as a human driver, I obviously love limited access roads (and human drivers perform very well on limited access roads). We won't have more of those though - the cultural paradigm for urban planning is one of tearing down freeways, adding bike-lanes, and ensuring pedestrian friendliness for maximum walkability.
Driving a car, on the other hand, is usually NOT a repetitive, dull process. You're comparing apples to oranges.
It likely allows everyone to learn from all the mistakes and improve the process for the whole industry - similar to the passenger airplane safety record improvements learned from every single accidents/crashes.
At the very least we should have a standard set of crash or near-miss scenarios so that we can collectively learn lessons when things go wrong.
Particularly when people have already paid the price with their lives.
- Such vehicles would not be street legal.
- They would not be safe for passengers.
- They would be unable to carry the necessary payloads.
- Most of the weight of a car is in the chassis anyway, not the body.
I would think another big reason though is that the handling and response of such a car would be very different from a real car. Stopping distances, acceleration, cornering, wind behavior, would all be different.
It seems to me the real problem would be that A: You need the car to be on the real streets, so at a minimum, it must be street legal. B: You need it to be on all the streets, including highways and such, so we're at least up to a Smart car level of body. From there it really isn't that great a leap to a full sedan chassis. Would one of those little cars even have room for all the requisite sensors? Even if they did, sedans are just so much more useful.
So it seems like the minimum practical car to test with is a sedan, and that's enough to kill people. Even the little Smart cars are plenty enough to kill people.
On slow roads, I've often seen them driven around summer communities. It would be silly for the government to not allow this.
> They would not be safe for passengers.
The above commenter pointed out "enough trust for 2 tons of metal and a passenger," so I'm assuming the idea is that they don't have a passenger in them at this stage. Additionally, regardless of whether an SUV is safer for the individual behind the wheel, shouldn't it be that individual accepting the risk and not a random pedestrian who didn't consent to this experiment?
> They would be unable to carry the necessary payloads.
Do they really carry payloads at this stage? Isn't the focus on building a safe autonomous system?
> Most of the weight of a car is in the chassis anyway, not the body.
The golf carts are 1/8th to 1/4th the weight of a crossover, for example.
The substantial issue I'd guess would be the differences in all aspects of performance between a real car and a golf cart, but at this stage companies like Uber should clearly just be testing detection of pedestrians et al.
https://www.youtube.com/watch?v=HJnjuhBIhwc
That stage of testing is past, though.
It seems reasonable that there should be some regulation on "x miles per incident" before autonomous vehicle companies are allowed to move forward from carts to full-sized cars.
Based on the stats that have been posted on HN, Google is very safe. Uber, on the other hand, is not.
- For the perceived feeling of safety in larger cars.
- For the off chance that once a year they might need to take some extra large load (e.g. help a friend with moving).
AKA. emotional needs, not practical ones.
But my overall point is that I believe for most people (myself included), choice of a car is atypical in a way in which it includes emotions instead of pragmatism.
I get that SUV's aren't for everyone. Acting like they are for no one is silly though.
I grew up where it snowed a lot. I've never owned anything that sat particularly high nor with 4WD, and I've never missed a day of work due to snow.
My street floods.
If the water is above the air intake of a Honda Civic, your SUV isn't going to do you any good.
I frequently want to carry a large number of items.
I'm continually shocked at how few SUVs exceed the cargo volume of our 35mpg Scion xB. I've moved furniture for a friend because his Equinox wouldn't carry it. I rented a Nissan Xterra to carry a bicycle box that fit less snugly in the Scion than it did in the Nissan. SUVs might have a use, but carrying things is not one of them.
Until you threw in the ol' "I have a family" (because the children can not be seen in a Volvo wagon), I thought you were doing a parody of "Excuses SUV Owners Use to Justify Their Big-Ass Vehicle". I'm pretty sure I was wrong.
That's like saying you don't need a fullsize truck because the methheads seem to do just find hauling scrap metal in their Dodge Caravan.
For a load with a constant amount of space usage (e.g. kids + sports stuff) having more space will make hauling that load less of a chore. If you do it regularly then the luxury of a bigger vehicle is often considered worth the cost.
I can and have hauled fridges, office desks, etc on the roof of my wagon. I always prefer to use my truck for those things unless the item can't stand getting dirty from the truck or I'm going so far that the extra fuel economy and better driving experience of the car makes it worth it. Strapping a desk to the roof of a car and not doing a crap job is a chore. Just tossing it in a truck and strapping it down takes like 30sec.
A vehicle with more space can haul more crap or haul the same crap more easily.
As an aside, I wouldn't let my kids be seen in a Volvo Wagon. I'm white trash, not some hipster. A rusted out Subaru is more my style.
That's like saying that one kinda, umm, perhaps didn't completely comprehend my point. Unless you're arguing that a Dodge Caravan has more space than a Ford F-150 (and maybe it does, dunno).
having more space will make hauling that load less of a chore.
Yes, I completely agree, that's why I like the Scion over an SUV.
As an aside, I wouldn't let my kids be seen in a Volvo Wagon.
So you're worried about the image you project as well. Fair enough, I think that might apply to many vehicle owners including, to a degree larger than I care to admit, myself.
The classic SUV, in other words a truck chassis, is an ever shrinking part of the market. FJ Cruiser, Xterra, 4Runner...
Also, both the FJ Cruiser and Xterra have been discontinued, which further emphasizes that "traditional" SUVs hardly exist these days.
Though to be honest, me & my SO are driving a compact MPV which is larger than our basic needs (no children so far) - we were wondering whether to get something smaller, but it was the best used vehicle available within the budget at the time. And we frequently find use for the extra interior space.
Carpooling is not a meaningful comparison. Carpooling requires picking up (and tolerating) other people on a semi-rigid schedule. Hailing a self-driving car has all the benefits of driving solo, plus you don't have to drive.
This sounds like something a caucasian single person would say.
Anyone with small kids would immediately have to think about the logistics of child car seats plus a stroller/wagon (at a minimum), plus whatever else would be appropriate to bring to the target destination (e.g. chairs and a cooler box for a picnic). Those are certainly not thing people do in an off-change.
Also don't forget cultures with tight-knit families (e.g. indians) where it's not uncommon for grandparents to live with you (and by extension travel with you) as well.
Anyone with small kids would immediately have to think
about the logistics of child car seats plus a
stroller/wagon [...]
My theory is the increasing car sizes are driven by increasing waist sizes; a 220 lb couple could be cramped in a car their 140 lb grandparents might have found plenty large enough.
[1] The average family is 3.14 persons in the US in 2017 according to https://www.statista.com/statistics/183657/average-size-of-a...
WTF? White people don't have trucks (or SUVs) and families?
Of course caucasians buy SUVs too...
It'd be really nice to have a golf-cart-ish car, a little SmartCar, a proper sportscar, an SUV, and a big pickup truck, and use the appropriate one in each situation.
But, even people that can afford the spare cash for that kind of stable of cars rarely do it -- it's just too inconvenient, too much maintenance (or too big a garage to keep them all in to minimize maintenance), depreciation, insurance, etc. and nevermind the constraints of city or near-city living where space is at a premium.
So, people just get the closest thing that fits most of their use-cases, and that's most often SUVs and crossovers.
I'm and example; while I'd be happiest in a small sportscar and it'd work for 85+% of my trips (and save fuel money), my occasional and unpredictable need to carry big packages for work keeps me using an SUV for my daily driver. Even if I wanted the extra expenses, I have winters and no spare garage. So...
Source: I drive a small car in Oakland.
I suppose this could be solvable with proper road infrastructure design and actually punishing people for bad parking.
2. I understand that Google does the vast majority of its testing without any iron at all, but rather tests new software using recorded input.
https://www.google.com/search?q=google+self+driving+bubble+c...
But who cares? It still seems like it would be valuable to collect many millions of simulation hours and eventual live testing hours with far safer vehicle types first, at minimum just as a proof of concept and a prove-out of the appropriate testing and evaluation infrastructure (e.g. we’ll probably be a lot better at extending the audit process that evaluates autonomous golf carts into an audit process for common cars than we would be just jumping straight to auditing cars without ever having audited anything else).
In fact, it seems irresponsible to not prove out the entire end to end ecosystem (from simulations to live testing to auditing to a commercially approved deployment in some limited scenario) using a much smaller scale application and vehicle type.
I can understand the arms race motivation for cutting corners and jumping straight to the widest use cases on the part of corporations. But I can’t see why society would want it handled that way.
Because these self driving cars aren't being developed to solve a hypothetical problem, they're being developed to solve the very real and current problem of cars killing 40,000 people in the US each year.
If extra safety measures result in killing ten fewer people in testing, but delay the introduction of self-driving cars by a year or two, that could mean thousands of preventable deaths still occurred. Are the lives of those people definitely getting killed right now worth 100x less than the lives of those who might get killed with these tests?
And this isn't baseless speculation. People are already obviously dying all the time, and improving these cars' software would be much harder without the kind of real-world data they're currently collecting.
Possible, but speculative.
> Also, estimating how safe sekf-driving car is you should take into account all the cases where human took over.
Sure, but on the other hand, the SDC's are continually improving as a group in a way that is not true for humans. This makes it harder to extrapolate into the future one way or the other.
> Also there are many accidents in weather conditions where self driving car would not be able to drive at all.
If you mean environments where humans also can't drive worth a damn, yes, probably true, although SDC's may yet prove better, since that's a technical problem rather than a social understanding one.
But 90%+ of accidents involve human error, so even if you can't completely eliminate deaths, you can still radically reduce them.
Yes, exactly. This is the reason you would want the technology to be developed slowly and carefully, incubated in a safer scenario for a long time to ensure it can be effectively audited and deployed when it comes time to use it for the intended, safety-improving commercial traffic use cases.
What you sounds sounds like good reason to more strongly favor a lengthy test and prove-out cycle on safer and more limited vehicle types, long before testing on commercial traffic vehicles.
The rate of auto-related deaths happening today certainly would not be a reason to hastily rush the technology and jump directly to extensive testing on the final commercial traffic end use cases. Trying to "hurry up" and prevent those deaths could counter-intuitively lead to more deaths because the infrastructure and auditing processes were not proved out and slowly ramped up to real traffic scenarios.
You assert this, but provide no reasoning. I mean, the status quo is "hurrying up", since there are no general standards companies are being held to, and yet only one company has killed anyone, and surprise surprise, it's the company with an extensive history of ethical problems, and they screwed up in a way that's really obvious and foreseeable.
The status quo is a free for all, and yet we don't have much in the way of deaths and injuries from the millions of miles being driven by computers. It seems like companies are already themselves being fairly safe, so how many deaths would you actually be avoiding by implementing more stringent standards right now?
In many industries, from food development to drug development to medical devices to clean energy tech to child car seats to film ratings we have nuanced and highly involved government oversight and regulations to try to ensure levels of safety, ethical and thorough clinical testing, ratings systems that map to outcomes, etc. etc. Yes, of course they are not always perfect, but that is beside the point entirely. The ubiquitous and widespread use of such regulations is a direct statement that companies cannot be trusted, have repeatedly proved through history to be untrustworthy regarding safety over and over across many industries, and that it's a civic need to have independent oversight. We can debate the amounts, regulatory capture, etc., but absolutely could not ever believe something like saying the status quo is just an industry free-for-all.
In fact, cases where start-ups try to skirt these issues in under-regulated industries are extremely rare and do not at all represent any type of status quo in any sense. There is absolutely no sense in which that maps to the way things work. It is rare for a industry to come up with its own testing and evaluation strategy where societal outcomes aren't brought up as first-principle constraints on how the innovation has to proceed.
> "It seems like companies are already themselves being fairly safe, so how many deaths would you actually be avoiding by implementing more stringent standards right now?"
This is just so egregiously unrelated to reality that I do not have the time resources to even start to reply. My mouth is agape that someone would think this.
> "You assert this, but provide no reasoning."
I agree with this for self-driving cars. I am not an expert of self-driving cars, and feel it's perfectly fair for me to say that in virtually every other industry, regulations are needed to ensure corporations don't actively harm people, falsify results, rush to market, etc., and that it logically and obviously extends to self-driving cars without any need for some onerous debate. The idea that companies are just trustworthy to rush this stuff to commercial markets does not need to be taken seriously. It's just obviously not so.
Beyond that, I completely agree I am not the expert on it, and would defer to e.g. the NTSB to decide how companies are behaving with regard to proper testing. The only part I would add is that as a citizen, my strong preference is for the testing to be gradual with a high emphasis on validation and a gradual proof of concept moving from simpler vehicles and situations to more complex ones. Regardless of what the raw machine learning capabilities are in terms of self-driving, this is needed to prove out the evaluation and auditing infrastructure as much as anything else -- essentially to prove that we have a framework for holding the companies accountable in the right ways.
I'm specifically talking about the current development of self-driving cars. I'm not sure why you thought I was generalizing to all activity among all industries.
> This is just so egregiously unrelated to reality that I do not have the time resources to even start to reply. My mouth is agape that someone would think this.
What's weird about it? Again, we're not talking about solving some hypothetical future problem, in which case I would agree with you about being hyper-cautious and safe. Tens of thousands of people die every year right now. Where is your mouth-agape-level concern for those people?
When a disease like Ebola breaks out, sometimes an experimental drug is used, earlier than would normally happen. This is not because we are collectively giving in to greedy companies wanting to destroy all government regulation, it's because we recognize that the dire need for a solution sometimes outstrips the need for caution. Here is a recent example: https://www.nature.com/articles/d41586-018-05205-x
Do you disagree with this kind of response for serious disease breakouts, or does 40,000 deaths a year just not count as sufficiently dire? Would 100,000 breach that threshold? 500,000?
If the choice is between getting self-driving cars more quickly with minimal regulation -- which is what's happening right now -- versus much more regulation that will slow their development and introduction, it seems straightforwardly obvious that more people are likely to die in the status quo environment in the latter scenario than people die from testing in the former scenario. Companies would really have to go full Mad Max to kill even close to as many people as our current infrastructure and design kill.
> it logically and obviously extends to self-driving cars without any need for some onerous debate.
That's what I'm challenging, and I'm doing so on the basis that right now, things seem to be going fairly well. One person has died, which is very regrettable, but compared to the absolute mountain of corpses our usage of cars has created in general? It's obvious to me what's more important.
Especially when the evidence from the current development of self driving cars is that a self driving car killing someone results in an entire testing programme to be suspended for months, possibly permanently, whilst the company is excoriated for taking unnecessary risks and is considerably less likely to develop a commercial application of SDVs as a result. It also shows that SDVs kill at a higher rate than non-SDVs even with human backstops, which is a much better test of whether they will actually save lives than "does a handful of test vehicles kill as many people as a few hundred million humans"
I guess I am saying this seems to be straightforwardly false to me, to a huge huge degree, and it worries me that people would naively use the rate of auto deaths under current circumstances as a justification to allow self-driving technology development to proceed unregulated.
It seems we just believe very strongly in positions that are completely and diametrically opposite to each other, so we are unlikely to find productive communication on a message board about it.
Why?
> It seems we just believe very strongly in positions that are completely and diametrically opposite to each other, so we are unlikely to find productive communication on a message board about it.
I mean, if you're not willing to explaining your reasoning, then yes I suppose that's true. Although I think part of why you may feel that way is that, based on one of your earlier comments, you seem to think I'm some kind of libertarian who's just generally opposed to regulation. I'm not; if anything I'm more of a social democrat. I just think that in this particular case, things are different from normal for a new, potentially dangerous technology.
My reasoning on the above point is fairly simple and straightforward:
1. The status quo for SDC development has been relatively minimal regulation. Even California mostly just makes companies report some basic data.
2. Most companies seem to be taking safety very seriously as it is, such that injuries and deaths have been rare. Only one death, and from a company widely acknowledged as ethically-challenged.
3. Given 2, it seems like the number of deaths that you'd avoid with greater regulation is very low, simply because, well, very few people are dying as it is.
4. Conversely, letting humans drive cars has resulted in an enormous number of people dying each year.
5. Given 3 and 4, slowing down SDC development with greater regulation seems likely to indirectly kill many, many more people than the status quo of minimally-regulated development will kill.
Are you expecting Waymo to, any day now, start rampaging around cities and killing people by the dozens? What does the scenario look like where SDC development kills more people than human drivers currently kill? You didn't address my analogy with disease outbreaks; why not?
You've done a good job arguing for the importance of regulation in general. You haven't done a good job arguing for why you think greater regulation in this particular scenario will avoid killing more people than the current setup kills.
https://www.roadandtrack.com/new-cars/car-technology/a100168...
Giant vehicles are a bad idea, period. But they're no less bad for robots.
[1] 3000 deaths per day is 1.25 million annually, from https://en.m.wikipedia.org/wiki/List_of_countries_by_traffic...
Edit: Uber’s program is a fiasco wrapped around a joke, but Waymo has taken an extremely careful approach and it’s a mistake to conflate the two.
This is by no means proven, and evidence suggests that it is wrong.
Cardiovascular disease is the number one cause of death, but I would still like some testing to be done on possible preventative measures before they're given to patients.
Don't forget the feedback loops. People use cars because they need cars, but they need cars precisely because they use them.
Have you ever left the city for a destination which is not another city, or a suburb?
Sure, if space ships were impossible or too expensive to even consider, we wouldn't be planning to visit Mars; but it isn't equally valid to not visit Mars altogether as an alternative to visiting Mars, just because going to Mars is harder than going to the corner store.
> Have you ever left the city for a destination which is not another city, or a suburb?
Sure. My point is not about vacations, or Mars trips, but precisely about regular commute. As more people started having long-range vehicles, the landscape of work and living spaces changed so that now people are required to have such vehicles, and if you don't have one, you're significantly disadvantaged on the job and residential markets. The same thing happened with cellphones, bank accounts, plastic cards - cool tools at first, and then soon requirements for living in modern society. I'm just pointing out this phenomenon for consideration.
I'll set aside for a moment how you seem to think that the only reason to leave the city is for a vacation. My point is that there is a tremendous benefit to people being able to choose precisely where they want to live and work, and that the car is the means to that end with the fewest limitations. More people could probably take mass routes in larger vehicles, especially if bicycle racks became larger and common enough to rely on, but that won't diminish the value of the car, even for a commute. Cities also seem dead-set on installing trams and subways, when buses clearly offer the best bang for the buck (but nobody likes to say that they ride the bus, probably because buses are unloved).
Personally I think the more effective route is to encourage less commuting altogether, which would be an obvious natural effect if the roads were on the market, but in practice it would have to take the form of a "credit". Some companies (e.g. Microsoft) have company shuttles, those could be encouraged as well (since roads are not on the market, and thus do not encourage pooling).
- the rate of traffic fatalities are unacceptably high
- the benefits of automobile usage far outweigh the costs, even with today's fatality rate.
The changes necessary to get fatalities completely down to zero would likely increase costs and reduce benefits too much. But reducing fatalities by an order of magnitude? Very feasible IMO.
https://en.wikipedia.org/wiki/Epidemiology_of_motor_vehicle_...
> This makes motor vehicle collisions the leading cause of death among young adults of 15–29 years of age (360,000 die a year) and the ninth cause of death for all ages worldwide.
> It is estimated that motor vehicle collisions caused the deaths of around 60 million people during the 20th century,[6] around the same as the number of World War II casualties.
Approximately 1% of all deaths are in car incidents.
Why? Isn't the entire premise of a fully autonomous vehicle that you can get wasted and still hop in your own car and get home safely?
Or even do it on a closed track. That way you could throw all sorts of things at it like kids, bikes, garbage cans, whatever.
Should be able to see what works and what doesn't work pretty quickly.
EDIT: Never mind! That's what Waymo does in the article another person posted.
https://www.theatlantic.com/technology/archive/2017/08/insid...
I don't know if Uber/Waymo/Others have "solved" this problem, but ML systems have a tendency to "learn" the wrong thing. You could run millions of simulations (both virtual or on a closed course) with "test obstacles", but the second the car encounters one a child with red balloons in the early morning on a foggy day it might not know what to do with it because that exact scenario wasn't in any simulations.
Agreed the test track is not ideal, that's why I was wondering if they could throw real recorded data at it. No risk to the public, but puts the system through real-world trials.
The car saw the obstacle and detected a collision, but there was no warning system to alert the driver about it.
Meanwhile, the driver was looking at a laptop and taking notes on the system logs (which they were told to do), not watching the road.
But the failure that killed Elaine Herzberg happened well before then, when the decision to disable the emergency brakes was made. It appears they disabled other safeties as well.
I don't disagree that playing a little loose with allowing new tech makes sense. We need to understand what we don't understand about them, accidents of course will happen, etc. But this looks a lot like foreseeable negligent homicide to me.
Like, let's say I had a grammar check. But the grammar check was bad. It flags every other sentence as needing a grammar change, anything other than a very simple "subject-verb-object" sentence structure looks wrong to it. For every correct grammar problem it identifies, it finds 20 incorrect grammar problems.
As a result, I turn off the UI for the grammar check.
Then I write a document that has a grammar problem.
It seems stupid to say that this is either a failure of my use of the software, or a UI failure of the software. The problem is that the fucking software doesn't work, specifically that it fails in a way that makes it too "noisy" to use.
And sure, if my job is to catch grammar errors, then it's also my fault. I'm supposed to be backing up the system. But we also know that I won't be perfect and will have failures of attention sometimes. But regardless of that, the grammar checker just didn't work.
And, in this analogy, bad grammar kills people.
A lot of aerospace money has been spent learning how to do things safely.
It would be a shame to spend the same money all over again, re-learning the exact same lessons in automotive.
I was not considering the different sensor inputs that would not work on an existing game.
This precludes the necessary conversations about test interfaces, for example, making the task of the simulation team closer to one of reverse-engineering than of simulator integration.
The only difference between disabling the emergency braking mechanism and, say, intentionally disabling safeties on industrial machinery is the novelty. I think it was pretty obvious that something like this would happen, and that's why so many people have been making noise about it. Now that we know for a fact that Uber is willing to make this choice, there's (a) no excuse for not coming down on them for it, (b) no excuse for Uber not completely overhauling their safety protocols with the help of an external ethics and legal advisors, and (c) no excuse for engineers working on this for not at least seriously thinking about their own ethics and possible moral culpability.
Some people want to focus on the future. Self driving cars will be great! Gotta break some eggs to make omelets! Fine. But those who are killed before that bright future deserve some accountability, those who don't want to be killed deserve some respect, and the makers of said omelets can bloody well be expected to be held responsible for their actions.
Sadly, sometimes this means saying goodbye to your colleagues and friends, and looking for new adventures in an organisation that is capable of acting in a more constructive and mature manner.
If 10 different executives knew or should have known that their decision had life-critical implications, then I think all of them should face the appropriate penalty. (Here, I presume it's for involuntary manslaughter due to criminal negligence.) And I think the same penalty should apply to developers who knew or should have known that their actions put lives at risk.