Self-driving Uber car kills Arizona woman crossing street
reuters.com
reuters.com
So it's not just "don't drink and drive", knowing that they'll probably reoffend soon anyway. Every crash and especially fatality can be thoroughly investigated and should be prevented from ever happening again.
Hopefully there's enough data in the investigation so that Tesla / Waymo and all other car companies can include the circumstances of the failure in their tests.
Also my problem with this is that a human death is functionally treated as finding edge cases that are missing a unit test, and progressing the testing rate of the code... and that really bothers me somehow. We need to avoid treating deaths as progress in the pursuit of better things
There's also NHTSA (which is indeed part of the DoT).
It looks to me like we don't need any new agency at all, just a very small expansion of NHTSA's mandate to specifically address the "self-driving" part of self-driving cars.
What's more, a lot of people seem to think Waymo's tech is further along. So not only does this project have to succeed, it is also the underdog. So no wonder they're aggressive.
This implies they're using the same systems and underlying models. If one model hit a pedestrian because of a weakness in training data plus a sub-optimal model hyperparameter, and therefore was classified a pedestrian in that specific pose as some trash on the street, how do you share that conclusion with other companies models?
It may, it also might not. If sharing data fails, further methods would be needed - but it's a good start for figuring out what data should be recorded for comparison. If the data is entirely incompatible, then we should have regulation to require companies to at minimum transcribe the data into a consumable format after an event such as this.
> ... if the models interpreted the pedestrian as trash on the street, then how it responded (driving over it) is not inappropriate.
If the models saw anything at all, they should not have driven over it. Even human drivers are taught that driving over road debris is dangerous. At minimum it puts the car/occupants at risk - in extreme cases, the driver may not recognize what it is they are driving over.
If this isn't a case where the car was physically unable to stop - it's more likely the telemetry didn't identify the person as an obstacle to avoid at all.
I don't think all self-driving cars have the same sensors.
If you have LIDAR model X, and they were using LIDAR model Y, will your system "magically figure it out?
If your car has cameras at 5ft high, and the data is from a camera 6ft high?
Sure, someone could release the data, but will it screw up your models more than it fixes them?
(I totally agree the data should be released, I'm just not sure other self-driving cars will directly benefit. Certainly they can indirectly benefit from it.)
The idea you present is possible, but I have to wonder how viable it makes the idea of self driving cars.
That being said, when the first real cars were introduced to the world and later improved upon there were many more fatalities than we're likely to experience with self-driving technology.
Every technology is dangerous. Every technology costs lives to some extent when spread across billions of people. I'm sure forks take more lives each year than self driving cars.
Weigh this against the potential lives saved. I posit you'd be killing more people with drunk drivers by slowing innovation than you'd be saving via luddism.
The great thing about computer generated test cases is you don't need real footage of every hypothetical awful thing that could happen i.e. a truck losing control and rolling over sideways. These could be a stage 1 test -- a pre-requisite to real testing. Like the hazards test before you are allowed to sit the drive test with a real drive test employee [1] to make sure you're not a retard.
[1]: https://www.vicroads.vic.gov.au/~/media/images/licences/hpt_...
We all make similar decisions all of our lives, but nearly always at some remove. (With rare exceptions in fields like medicine and military operations.) But autonomous vehicles are a very visceral and direct implementation. The difference betweeen the trolley problem and in autonomous vehicles is in time delay and the amount of effort and skill required in execution.
Plus, we're taking what is pretty clearly a moral decision and putting it into a system that doesn't do moral decision-making.
Au contraire. Go read building codes some time. There's a saying that they're "written in blood" - every bit, no matter how obvious or arbitrary seeming, was earned through some real-world failure.
The death itself isn't progress, of course. But we owe it to the person to who died to learn from what happened.
Or, you know, don't jump on comments for their not explicitly addressing the hobby horse you're riding. Frankly I just wanted to express a better engineering context around the loss of life without getting into the political bullshit for once.
http://www2.ensc.sfu.ca/undergrad/euss/enscquire/Vol10No2/pa...
Then by all means lets stay at home and avoid putting humans in rockets ever again because if you think space exploration will be done without deaths you are in for a surprise.
Does the NTSB have regulation on how black boxes are allowed to function?
Would be better if the code and data was open for public review.
Making it public means more eyes on the data, which can lead to a better understanding of what went wrong.
The real news isn't "Self-Driving Car Kills Pedestrian", the real news is "Self-Driving Car Fatalities are Rare Enough to be World News". I'm one-third a class M planet away from the incident and reading about it.
They are rare only because self-driving cars are; I don't think the total driven miles of all self-driving cars are enough that even 1 fatality would be expected if they were human driven; certainly Uber is orders of magnitude below that point taken alone.
There are lots of fatalities from human-driven cars, sure, but that's over a truly stupendous number of miles driven.
There's different estimates from different sources using slightly different methodology, but they are all in the neighborhood of road fatalities of 1 per 100 million miles traveled. [0]
Waymo claims to have reached 5 million miles in February [1], Uber (from other posts in this thread) is around 1 million miles; the whole self-driving industry is nowhere near 100 million, and has one fatality. So it's way worse, as of today, than human driving for fatalities.
Of course, it's also way too little data (on the self-driving side) to treat as meaningful in terms of prediciting the current hazard rather than simply measuring the outcomes to date.
[0] see, e.g., https://en.m.wikipedia.org/wiki/Transportation_safety_in_the...
lol
Not saying this is the case here, but it could be. As others have said, we need to wait until we know more before jumping to conclusions.
This means that there's a fixed distance from which the optimal driver can stop a car doing xmph. Yes, an autonomous vehicle has a faster reaction time* to begin the stop, but no matter the reaction time, a stop cannot be instantaneous from any substantial amount of speed.
If it takes 20 feet to stop a car doing 20MPH, it will take 80 feet to stop a car doing 40mph. If there's a human between the initial brake point and 80 feet from it, that human will be hit, no matter who or what the driver is.
[1]http://www.phoenixnewtimes.com/news/cops-uber-self-driving-c...
Accidents can still occur even if the car is perfect.
It, or the driver, could do more than just stop though. You can change directions, even at 38mph.
Then we have to get into other questions, would I as a driver willingly sideswipe a car next to me to avoid hitting a pedestrian? Is it reasonable to expect an AI to make the same value decision?
An algorithm however is a different deal, what should happen is already decided in an algorithm, so in some way its already settled who gets killed.
You can't avoid what is hiding.
0) https://www.nytimes.com/2018/03/19/technology/uber-driverles...
That would increase the chance a particular vehicle could avoid an accident which it couldn't, on its own, anticipate.
It would be much safer if all roadways with speed limits over a few km/h were fenced, with tunnels or bridges for pedestrian crossing. Arguably, we would have that now, but for legislative efforts by vehicle manufacturers many decades ago. Maybe we'll get there with The Boring Company.
"Most people" (which is, in reality, "most of my geek friends with high disposable income") shifts to "everyone" by the end of sentence. Also, my devices seem to know where I am...within a few city blocks: I do not like your requirement of always-on mandatory tracking, both from privacy and battery life POVs.
Even worse, this has major false negatives: it's not a probation tracker device - if I leave it at home, am I now fair game for AVs? And even if I have it with me and fine position is requested, rarely do I get beyond "close to a corner of X an Y Street," usually the precision tops out at tens of ft: worse than useless for real-time traffic detection.
Moreover, your proposal for car-only roadways is only reasonable for high-speed, separated ways; I sure hope you are not proposing fencing off streets (as would be the case here: 50 km/h > "a few km/h", this is the usual city speed limit).
Some do argue that speed limits for unfenced roadways within cities ought to be 30 km/h or less. And although fatalities are uncommon at 30 km/h, severe injuries aren't. I live in a community where the speed limit is half that. But there's no easy solution, given how prevalent motor vehicles have become. Except perhaps self-driving ones.
0) https://www.gps.gov/systems/gps/performance/accuracy/#how-ac...
Edit: spelling
Or maybe it couldn’t, but then the whole « narrative « of the experiment is in serious jeopardy.
Not really. Self driving cars are supposed to be better than average human driver. That does not imply that they NEVER make mistakes.
I do not know specifics of this case, but a general comment: If somebody is hiding behind a bush, and (deliberately or by mistake) run in front of the car, there is no way the car can anticipate that. There is no way to avoid accidents in 100% of the cases.
I can think of many situations where I have avoided hitting pedestrians because of my avareness of the situation. Eg: Pedestrian with earphones looking at phone crossing against red light just because the left-turning wehicle in the left lane stopped for a red arrow while I had green going straight. Pedestrian mostly behind the car, just seen thru the window of the car.
Pedestrian behind high snow-walls going towards normal pedestrian crossing, no lights. Almost completely covered by the high snow-walls and a buss parked at a buss station 50 m away from the crossing. 50 km/h road. Since I had seen the pedestrian far away already I knew someone would show up there at the time I arrived there. On the other hand I would never pass a buss like that in high speed, pedestrians like to just run across in front of the buss. And high snow-walls next to a crossing is a big red flag too.
I live in Sweden though, where pedestrians are supposed to be first class citizens that has no armor.
This looks like the exact situation the selfdriving cars are supposed to be able to avoid. A big object in the middle of the street. I expect the car to try to avoid this even though the bike didn't seem to have any reflexes. If the LiDAR doesn't catch this, I don't think they should be out in traffic at all.
Yes, but this is a 4-lane roadway. I can totally imagine driving cautiously and slowing down near residential areas where houses are close to the road. However, this seems like a different case.
Thinking about it seriously, it may be shouldn't. Also these things could lead to a crash pile up with other vehicles coming in from behind.
> Chief of Police Sylvia Moir told the San Francisco Chronicle on Monday that video footage taken from cameras equipped to the autonomous Volvo SUV potentially shift the blame to the victim herself, 49-year-old Elaine Herzberg, rather than the vehicle.
> “It’s very clear it would have been difficult to avoid this collision in any kind of mode [autonomous or human-driven] based on how she came from the shadows right into the roadway,” Moir told the paper, adding that the incident occurred roughly 100 yards from a crosswalk. “It is dangerous to cross roadways in the evening hour when well-illuminated managed crosswalks are available,” she said.
0) http://fortune.com/2018/03/19/uber-self-driving-car-crash/
Based on the layout of the presumed crash site (namely, the median had a paved section that would effectively make this an unmarked crosswalk), and based on the fact that the damage being all on the passenger's side (which is to say, the pedestrian would have had to have crossed most of the lane before being struck), I would expect that there is a rather lot that could have been done on the driver's side (whether human or autonomous) to avoid the crash.
> Herzberg is said to have abruptly walked from a center median into a lane with traffic
So that explains that. However, contrary to the thrust of your argument, the experience of the sober driver, who was ready to intervene if needed, is hard to dismiss:
> “The driver said it was like a flash, the person walked out in front of them,” Moir said. “His first alert to the collision was the sound of the collision.”
And also:
> “It’s very clear it would have been difficult to avoid this collision in any kind of mode [autonomous or human-driven] based on how she came from the shadows right into the roadway,” Moir told the paper, adding that the incident occurred roughly 100 yards from a crosswalk. “It is dangerous to cross roadways in the evening hour when well-illuminated managed crosswalks are available,” she said.
No human could brake that well, and simply jamming the brakes would engage the ABS leading to a longer stopping distance. Not to mention the human reaction time of 0.5 - 0.75 seconds would have prevented most people from even lifting the foot off the accelerator pedal before the collision, even if they were perfectly focused on driving.
I was taught that the entire point of ABS is so that you can just jam the brake and have the shortest stopping time instead of modulating it yourself to avoid skidding. Do you have any source to the contrary?
Wikipedia backs me up but adds that it also decreases stopping distance on dry and slippery surfaces, while significantly increasing stopping distances in snow and gravel. I’m from a country with a lot of snow so that makes sense.
In terms of split second reactions, it's pretty much optimal still to just jam the brakes if you have ABS. It's much better than braking too little, which is what most non-ABS drivers would have done.
That said, the point of ABS is in the rare event that you have to brake full power, the system automatically help you do it at a near optimal (slightly skidding) without additional input, and you remain full steering ability.
If you don't have ABS you'd need to train that emergency stop ability on a daily basis to even come close.
Many cars, such as my POS Ford Focus, use a single-channel ABS system. These systems will oscillate all four brakes even if only one is locked. Combined with the rear-wheel drum brakes, the ABS considerably increased stopping distances on dry road.
Older dumb ABS systems would simply "pump the pedal" for the driver, and would increase stopping distance in almost all conditions, especially single-channel systems. Newer systems determine breaking performance via the conventional ABS sensors and additionally accelerometers. These systems will back off N G's, then increase the G's bisecting the known-locked and known-unlocked condition, trying to find the optimum. These systems _will_ stop the car in the minimum distance possible, but very few cars use it.
Maybe it's society's fault, for building open-access roadways where vehicles exceed a few km/h.
As others have said in the comments, whole point of having technology is defeated if it performs worse than humans. Assuming vehicles were parked, a sane human driver will evaluate the possibility of someone coming out from between them suddenly and will not drive @40 Miles an hour speed.
If that's the case most drivers on the road are very far from "sane drivers." I've been illegally passed, on narrow residential streets, many times, because I was going a speed that took into account the fact someone may jump out between parked cars.
http://fortune.com/2018/03/19/uber-self-driving-car-crash/
> she came from the shadows right into the roadway
also we were told radars would have have solved exactly this limitation of humans
> Uber car was driving at 38 mph in a 35 mph zone
also we were told these car would be inherently safer because they would always respect limits and signage
> she is said to have abruptly walked from a center median into a lane with traffic
I don't know other driver, but when someone is on the median or close to the road I usually slow down in principle because it doesn't match the usual expectations of a typical 'safe' situation.
I've been advocating against public testing for a long time, because it's just treating people safety as an externality. Uber is cutting corners, not all company are that sloppy, but this is, overall, unacceptable.
Whoever owns the algorithm. Or at-least in whoever's name the license/permission was issued. If its an organization, the top management signing off on this has to take the blame.
There are several states is USA that are more progressive than others (CA namely). But with many working groups in and around the legal side - it hopefully will be a thing of the past.
In Australia, they are mandating by some year soon (don't have it on hand) that to achieve a Safety Rating of 5 star, some level of automation needs to exist. Such as lane departure or ABS will become as standard as aircon.
>> It's physically impossible for a human to intervene on the timescales involved in motor accidents.
That remains true even without any automatic driving tech - you are responsible even for accidents which happen too quickly for anyone to intervene. Obviously if you have some evidence(dashcam) showing that you couldn't avoid the accident you should be found not guilty, but the person going to court will be you - not the maker of your car's cruise control/radar system/whatever.
The assists are certainly helping more than anything, so I feel that the Mazda is much safer to drive in heavy traffic than the older Outback.
The cruise has autonomy over controlling the speed only, and applying brakes, but it is still autonomy. Of course since my hands never leave the wheel it may not fit with what you have in mind.
Having said that, Mazda (or Bosch?) really nailed their radar, having never failed to pick up motorbike riders even though the manual warns us to not expect it to work.
I feel more confident in a system where the ambition is smaller, yet execution more solid.
Fwiw I also tested the AEB against cardboard boxes driving through them at 30km/h not moving accelerator at all, and came away very impressed by the system. It intervened so last second I felt for sure it wasn't going to work, but it did - first time was a very slight impact, next two were complete stops with small margins.
This stuff is guaranteed to save lives and prevent costly crashes (I generally refuse to use the word "accident") on a grander scale.
I do love that even though they have a ton of driver alerting features, hands have to be on the wheel at all times.
Either you have full autonomy without hands or you don’t. There is no middle ground, it’s a recipe for a disaster.
Let me put it this way - my Mercedes has an emergency stop feature when it detects pedestrians in front of the car. If I'm on cruise control and the car hits someone, could I possibly blame it on Mercedes? Of course not. I'm still the driver behind the wheel and those systems are meant to help - not replace my attention.
What we have now in these semi-autonomous vehicles is nothing more than a glorified cruise control - and I don't think the law treats it any differently(at least yet.).
Now, if Uber(or anyone else) builds cars with no driver at all - sure, we can start talking about shifting the responsibility to the corporation. But for now, the driver is behind the wheel for a reason.
The San Francisco Chronicle late Monday reported that Tempe Police Chief Sylvia Moir said that from viewing videos taken from the vehicle “it’s very clear it would have been difficult to avoid this collision in any kind of mode (autonomous or human-driven) based on how she came from the shadows right into the roadway." (bit.ly/2IADRUF)
Moir told the Chronicle, “I suspect preliminarily it appears that the Uber would likely not be at fault in this accident,” but she did not rule out that charges could be filed against the operator in the Uber vehicle, the paper reported.
On top of the speedometer it has the GPS speed to compare as well, I can't see how there is any excuse for being over the limit.
The quoted stats from UK advertising were that at 40mph 80% of pedestrians will die from the crash, at 30mph 20% will die.
Had the car been doing just under the limt e.g. 33mph then there's a much better chance that the woman would have survived.
http://www.dailymail.co.uk/news/article-5332443/Police-chief... [30 January 2018]
https://www.telegraph.co.uk/news/2018/01/31/motorists-should...
Can you explain why you think this rule changed a decade ago?
Because sometimes that 10% is argued as a margin of error for humans supposedly not paying attention how fast they're going, but if that's the case then there's really no reason why the robot shouldn't drive strictly under the speed limit.
If you explicitly programmed a fleet of robots to deliberately break the law, then I think it's not enough consequences if you just fine for the first robot that gets caught breaking that law, while the programmers adjust the code of the fleet to not get caught again.
Consequences should be more severe if there's a whole fleet of robots programmed to break the law, even if the law catches the first robot right away and the rest of the fleet is paused immediately.
Take a look at the cost of airplanes, even small ones.
This comment by Animats in the context of self-driving trucks is quite telling. He warns precisely of this danger.
As we understand more about the risks associated with autonomous driving, we should expand and enrich this data-set, and to ensure public safety, testing against such a dataset should be part of NHTSA / Euro NCAP testing.
I.e. NHTSA and Euro NCAP should start getting into the business of software testing.
I think the idea is to build a "Traincar of Reasonability" to test future autonomous vehicles with.
You might want to check out her research https://hal.pratt.duke.edu/research
I'm sure Dr. Cummings would be more than happy to talk about issues facing validation and verification in the context of NHTSA/FAA.
It is more an issue of how sophisticated these vehicles should be before they're let loose on public roads. At some stage they have to be allowed onto public roads or they'd literally never make it into production.
If they're not willing to dogfood their own potential murder machines then why should the public trust them on the public roads?
They were unwilling to legally obtain a self-driving license in California because they did not want to report "disengagements" (situations in which a human driver has to intervene).
Uber would just set their self-driving cars free and eat whatever fine/punishment comes with it.
https://www.reuters.com/article/us-enron-skilling/former-enr...
Enron's founder, Kenneth Lay, was also convicted and faced as much as life in prison. However, he died before sentencing: http://www.nytimes.com/2006/07/05/business/05cnd-lay.html
This is a strange question to ask. The regulation is not there to benefit Uber, it is to benefit public good. Very few companies would follow regulation if it was a choice. The setup of such regulation would be for it to be criminal to not comply. And if Uber could not operate in California (or the USA) if they did not comply, it would in their interest to provide the requested information.
Essentially, Uber engages in regulatory arbitrage but taking into account the cost-benefits of breaking the law. I.e. if it breaks the law but is profitable for them, they seem to do it.
My 'money' is on people with money figuring out loopholes, like plausible deniability.
One of my biggest complaints about the self-driving car space is that real lives are at stake; light-touch "voluntary" rules suitable for companies that publish solitaire clones aren't going to cut it here.
[1] http://money.cnn.com/2015/12/10/news/companies/gm-recall-ign...
I can’t say anything about GM‘s rep in the US, but here in Europe they are not doing so well. Chevrolet was killed in 2015, and Vauxhall/Opel are doing only ok-ish. Chevy had SO many recalls in the years before they killed it.
Uber already has a terrible reputation with everyone in the tech industry for the sexism, bullying, law breaking, and IP theft. Do they really want to be the self-driving car company with a reputation for killing people?
It doesn't take a lot for people to think "Maybe I'll take a lyft" or "Maybe I'll ban uber from London because of their safety record"[0].
They aren't going to kill the market for this - the other players not only have big incentives to make sure they look safe, but you've got a really unique problem when your biggest competitor is a company that controls access to news and which adverts your customers will see.
[0] https://www.theguardian.com/politics/2017/oct/01/sadiq-khan-...
Uber doesn't get to pick and choose what regulations they wish to follow.
> Uber would just set their self-driving cars free and eat whatever fine/punishment comes with it.
That sounds quite negligent and a cause for heightened repercussions if anything happens.
The strange attitude you display is the _reason_ there are regulations.
It’s the only real solution to corporations misbehaving.
The only working regulation is one that is a existential threat to the company. Which means- huge financial punnishments.
Still, corporations, while being essentially a different kind of life, are not entirely separate entities - they are composed of people. Fear of jail might just be enough to get some high-level executives to exert proper influence on the direction of the corporation.
As well as some core set of tests that define minimum competence, these tests could include sensor failure, equipment failure (tire blowout, the gas pedal gets stuck, the brakes stop working) and unexpected environmental changes (ice on the road, a swerving bus).
Manufacturers could even let the public develop and run their own test cases.
So we will have overall a much smaller number of deaths caused by self driving cars, but ones that do happen will be completely unexpected and scary and shitty. You can't really get away from this without putting these cars on rails.
Moreover, the human brain won't like processing these freak accidents. People die in car crashes every damn day. But we have become really accustomed to rationalizing that: "they were struck by a drunk driver", "they were texting", "they didn't see the red light", etc. These are "normal" reasons for bad accidents and we can not only rationalize them, but also rationalize how it wouldn't happen to us: "I don't drive near colleges where young kids are likely to drive drunk", "I don't text (much) while I drive", "I pay attention".
But these algorithms will not fail like that. Each accident will be unique and weird and scary. I won't be surprised if someone at some point wears a stripy outfit, and the car thinks they are a part of the road, and tries to explicitly chase them down until they are under the wheels. Or if the car suddenly decides that the road continues at a 90 degree angle off a bridge. Or that the splashes from a puddle in front is actually an oncoming car and it must swerve into the school kids crossing the perpendicular road. It'll always be tragic, unpredictable and one-off.
Are you working on the next season of Black Mirror?
In all seriousness, my fear (and maybe not fear, maybe it's happy expectation in light of the nightmare scenarios) is that if a couple of the "weird and terrifying" accidents happen, the gov't would shut down self-driving car usage immediately.
Scary stuff. I hope these self-driving cars will be able and designed to work while completely offline, with no built-in way to ever connect to a network. But given the biggest player in the field seems to be Google, they'll probably be always connected in order to send data to the mothership and receive ads to show you.
I'm just envisioning a scenario where the car automatically pulls to the side of the highway, locks the doors and dishes you with a 15 second ad, and then the doors unlock and the journey resumes as normal.
Don't hold your breath about that. There will be a huge load of data ready to be sold to advertising companies just by listening what passengers talk about when passing near areas/stores/billboards/events etc.
Your fear is very much grounded in reality. US lawmakers tend to be very reactionary, except in rare cases like gun laws. So it won't take much to have restrictions imposed like this. Granted, I believe some regulation is good; after all the reason today's cars are safer than those built 20 years ago isn't because the free market decided so, but because of regulation. But self driving cars are so new and our lawmakers are by and large so ignorant, that I wouldn't trust them to create good regulation from the get go.
All safety functionality was introduced and used way before regulators even knew that it's possible.
Edit: please explain the downvotes, ideally with examples
It's amazing how reformers get demonized even after their platform is accepted wholesale by the rest of the world.
They're still very reactionary in that, which is precisely why it isn't very effective when a subset of them do react: there are plenty of smart things that could get proposed, but the overlap between people who know what they're talking about and people that want the laws is exceptionally small, so consequently dumb, ineffective stuff that has no chance of passing anyway gets proposed. What does get proposed is a knee-jerk reaction to what just happened, and rarely actually looks systemically at the current laws and gun violence as a whole. Example: the Las Vegas shooting prompted a lot of talk of bump stock bans. Bump stocks are so rarely used at all, nevermind in violence, and they will generally ruin guns that weren't originally made to be fully-automatic very quickly if they're actually used for sustained automatic fire. Silly point to focus on suddenly. After the Florida shooting last month so much focused on why rifles are easier to obtain than handguns. And it's because overwhemingly most gun violence is handguns. Easily concealable rifles are already heavily regulated at the federal level for that very reason.
<off-topic> This is non-sense. Typical semi-auto are wayyy over build. Unless mechanical wear or explicit tempering of the disconnector, there is no risk whatsoever to fire thousands of rounds with a bump stock. Actually, plastic/wood furniture are more likely to burn/melt before the mechanical parts will actually fail. As worst, you might bend a gas piston, but the rifle will otherwise be fine.
The underlying reasoning behind the push against the bump stock ban is that it was basically a semi-auto ban, as you can trivially with a bit of training bump fire any semi-auto without a bump stock from either the shoulder or the hip with a mere finger. </off-topi>
Tubes on low- to mid- range civilian DE guns can burn out very quickly, and are in fact designed to do so long before you get damage to the more expensive parts of the gun - I've seen it happen in most of the cases (which are admittedly quite few in number despite how often I'm there) where I've seen someone using a bump stock at a range. In the most recent case I think the guy was on his 3rd mag and it ruptured. It was a M&P 15 Sport II, if I recall. Not a cheap no-name brand, but about as low-cost as you can get and missing all the upgrades in the version they market to cops. High-end ARs would fair better, I'd expect, but high-end ARs are again so rarely used for actual violence because they're usually only purchased by people shooting for a serious hobby in stable life situations. And honestly I feel the same people buying those probably feel bump stocks are tacky and gaudy like I do.
Even in the most liberal interpretation of the proposed law, I don't think any bump stock ban would become a semi-auto ban. I could see the vague language getting applied to after-market triggers, especially ones like Franklin Armory, but you've gotta have some added device for any of the proposals I've seen to even remotely apply.
And for a good reason, they are constitutionally prohibited to .
In Black Mirror, all cars in the world will simultaneously swerve into nearby pedestrians, buildings, or other cars.
I won't be surprised if someone at some point wears a stripy outfit, and the car thinks they are a part of the road
Instead, LIDAR should exactly identify potential obstacles to the self-driving car on the road. The extent to which machine learning is used is to classify whether each obstacle is a pedestrian, bicyclist, another car, or something else. By doing so, the self-driving car can improve its ability to plan, e.g., if it predicts that an obstacle is a pedestrian, it can plan for the event that the pedestrian is considering crossing the road, and can reduce speed accordingly.
However, the only purpose of this reliance on the machine learning classification should be to improve the comfort of the drive (e.g., avoid abrupt braking). I believe we can reasonably expect that within reason, the self-driving car nevertheless maintains an absolute safety guarantee (i.e., it doesn't run into an obstacle). I say "within reason", because of course if a person jumps in front of a fast moving car, there is no way the car can react. I think it is highly unlikely that this is what happened in the accident -- pedestrians typically exercise reasonable precautions when causing the road.
[1] https://www.cs.cmu.edu/~zkolter/pubs/levinson-iv2011.pdf
It's a point of clarification that the originally listed study doesn't take into account, but which could be important to the broader discussion. Especially considering that while this vehicle had LIDAR, the other autonomous vehicle fatality case did not.
> as are basically all other prototype fully-autonomous vehicles
As I pointed out with examples above, no, they are not.
You can get depth sensing (time of flight) 2D camera Orbecc Astra for $150 or 1D laser scanner RPLIDAR for $300. Of course they are probably not suited for automotive, but for me even extra $2000 for self-driving car sensors isn't that much.
LIDAR is also not perfect when the road is covered in 5 inches of snow and you can't tell where the lanes are. Or at predicting a driver that's going to swerve into your lane because they spilled coffee on their lap or had a stroke.
With erratic input, you will get erratic output. Even the best ML vision algorithm will sometimes produce shit output, which will become input to the actual driving algorithm.
Neither are humans, and a self-driving car can react much faster than any human ever could.
Why did this happen? What steps have we make sure it will never happen again? These are both methods of analysing & fixing problems and methods of preserving a decision making authority. Sometimes this degrades into a cynical "something must be done" for the sake of doing, but... it's not all (or even mostly) cynical. It just feels wrong going forward without correction, and we won't tolerate this from our decision makers. EVen if we will, they will assume (out of habit) that we won't
We can't know how this happened. There is nothing to do. ..and.. this will happen again, but at a rate lower than human driver's more less opaque accidents.... I'm not sure how that works as an alternative finding out what went wrong and doing something.
Your comment is easily translated into "you knew there was a glitch in the software, but you let this happen anyway." Something will need to be done.
In a nearby town, people have petitioned for a speed limit for a long time. Nothing happened until a 6 year old boy was killed. Within a few weeks a speed limit was in place.
I think any attempts to address such issues have to come with far-ranging transparency regulations on companies, possibly including open-sourcing (most of) their code. I don't think regulatory agencies alone would have the right incentives to actually check up on this properly.
If self-driving cars really are safer in the long-run for drivers and pedestrians - maybe what people need is a better grasp on probability and statistics? And self-driving car companies need to show and publicize the data that backs this claim up to win the trust of the population.
Perhaps, similar to airline crashes, we should expect Uber to pay out to the family, plus a penalty fine. 1m per death? 2? What price to we put on a life?
This is a tough problem, but if we stop or limit the allocation of resources to protect the secretive intellectual property that is autonomoilusly running people over, that is the most effective incentive I can see. Plus it's pretty easy to do.
We don't even have to force them to disclose anything, by affording less legal protection, their employees will open-source it for us.
I mean, people play the lottery. That's a guaranteed loss, statistically speaking. In fact, it's my understanding that, where I live, you're more likely to get hit by a (human-operated) car on your way to get your lottery ticket than you are to win any significant amount of money. But still people brave death for a barely-existent chance at winning money!
Maybe because its unexpected - the victim is not involved until they are dead?
If the road was filled with self-driving cars there would be less accidents but I wouldn't understand them and with that comes distrust.
Freak accidents without explanations are not going to cut it.
Also, my gut feeling says this was a preventable accident that only happened because of many layers of poor judgement. I hope I'm wrong but that is seriously what I think of self-driving attempts in public so far. Irresponsible.
Definitely, though my interpretation of your statement is "self driving cars have only killed a couple people ever but human cars have killed hundreds of thousands". If that's correct, that's not going to win anyone over nor is it necessarily correct.
While the state of AZ definitely has some responsibility for allowing testing of the cars on their roads, Uber needs (imo) to be able to prove the bug that caused the accident was so much of an edge case that they couldn't easily have been able to foresee it.
Are they even testing this shit on private tracks as much as possible before releasing anything on public roads? How much are they ensuring a human driver is paying attention?
I think this is really key. The ability to put the blame on something tangible, like the mistakes of another person, somehow allows for more closure than if it was a random technical failure.
Tangent: is there a land vehicle designed for redundant control, the way planes are? I've always wondered how many accidents would have been prevented if there were classes of vehicles (e.g. large trucks) that required two drivers, where control could be transferred (either by push or pull) between the "pilot" and "copilot" of the vehicle. Like a driving-school car, but where both drivers are assumed equally fallible.
But the problem is Uber's business plan is to replace drivers with autonomous vehicles ferrying passengers. i.e. take the driver cost out of the equation. Same goes for Waymo and others trying to enter/play in this game. It's always about monetization which kills/slows innovation.
Just highway-mode is not going to make a lot of money except in the trucking business and I bet they will succeed soon enough and reduce transportation costs. But passenger vehicles, not so much. May help in reducing fatigue related accidents but not a money making business for a multi-billion dollar company.
That being said, really sad for the victim in this incident.
Why? This is what the self-driving cars industry insists on, but has nowhere near been proven (only BS stats, under ideal conditions, no rain, no snow, selected roads, etc -- and those as reported by the companies itself).
I can very well imagine a greater than average human driving AI. But I can also imagine being able to write it anytime soon not being a law of nature.
It might take decades or centuries to get out of some local maxima.
General AI research had also promised the moon once again in the 60s and 70s, and it all died with little to show of in the 80s. It was always "a few years down the line".
I'm not so certain that we're gonna get this good car AI anytime soon.
Because machines have orders of magnitude fewer failure modes than humans, but with greater efficiency. It's why so much human labour has been automated. There's little reason to think driving will be any different.
You can insist all you like that the existing evidence is under "ideal conditions", but a) that's how humans pass their driving tests too, and b) we've gone from self-driving vehicles being a gleam in someone's eye to actual self-driving vehicles on public roads in less than 10 years. Inclement weather won't take another 10 years.
It's like you're completely ignoring the clear evidence of rapid advancement just because you think it's a hard problem, while the experts actually building these systems expect fully automated transportation fleets within 15 years.
>It's why so much human labour has been automated.
But how much human labour that is as complicated as driving has been automated? As far as I can tell automation is very, very bad when it needs to interact with humans who may behave unexpectedly.
>b) we've gone from self-driving vehicles being a gleam in someone's eye to actual self-driving vehicles on public roads in less than 10 years. Inclement weather won't take another 10 years.
>It's like you're completely ignoring the clear evidence of rapid advancement just because you think it's a hard problem, while the experts actually building these systems expect fully automated transportation fleets within 15 years.
Actually plenty of experts within the field disagree with you.
“I tell adult audiences not to expect it in their lifetimes. And I say the same thing to students,” he says. “Merely dealing with lighting conditions, weather conditions, and traffic conditions is immensely complicated. The software requirements are extremely daunting. Nobody even has the ability to verify and validate the software. I estimate that the challenge of fully automated cars is 10 orders of magnitude more complicated than [fully automated] commercial aviation.”
Steve Shladover, transportation researcher at the University of California, Berkeley
http://www.automobilemag.com/news/the-hurdles-facing-autonom...
With autonomous cars, you see these videos from Google and Uber showing a car driving around, but people have not taken it past 80 percent. It's one of those problems where it's easy to get to the first 80 percent, but it's incredibly difficult to solve the last 20 percent. If you have a good GPS, nicely marked roads like in California, and nice weather without snow or rain, it's actually not that hard. But guess what? To solve the real problem, for you or me to buy a car that can drive autonomously from point A to point B—it's not even close. There are fundamental problems that need to be solved.
Herman Herman, Director of the National Robotics Engineering Center @ CMU
https://motherboard.vice.com/en_us/article/d7y49y/robotics-l...
Driving is not complicated at its core. Travel along vectors that intersect at well-defined angles. Stop to avoid obstacles whose vectors intersect with yours.
Sometimes those obstacles will intersect with your vector faster than you can stop, which is probably what happened to this woman. As long as the autonomous car was following the prescribed laws, then it's not at fault, and a human definitely would not have been able to stop either.
> Merely dealing with lighting conditions, weather conditions, and traffic conditions is immensely complicated. The software requirements are extremely daunting.
Which is why self-driving cars don't depend on visual light, and why prototypes are being tested in regions without inclement weather. Being on HN, I'm sure you're well familiar with the product development cycle: start with the easiest problem that does something useful, then generalize as needed.
> With autonomous cars, you see these videos from Google and Uber showing a car driving around, but people have not taken it past 80 percent. It's one of those problems where it's easy to get to the first 80 percent, but it's incredibly difficult to solve the last 20 percent. If you have a good GPS, nicely marked roads like in California, and nice weather without snow or rain, it's actually not that hard.
Right, so the experts agree with me that the problem the pilot projects are addressing is readily solvable, and that general deployment will take a number of years of further research, but isn't beyond our reach. This past year I've already read about sensors that can peer through ice and snow. 15 years is not at all out of the question.
If a ball bounces in front of me, I slow down expecting a dog or a child running after it. No self driving car now, and in 30 years is going to be able to infer that.
Driving is essentially interacting with the environment, reading hand signals from people, understanding intent of pedestrians, bicycles and other drivers. No way any AI can do that now.
Trains travel along a straight line, not a vector in 2D space.
> If a ball bounces in front of me, I slow down expecting a dog or a child running after it. No self driving car now, and in 30 years is going to be able to infer that.
Incorrect. I don't know why you think humans are so special that they're the only system capable of inferring such correlations.
Quite a lot actually.
These days you can produce food for several thousands of people using a few hundred people and plenty of machines.
Part of the reason why we haven't yet reached a Malthusian catastrophe is this.
Automated driving is more like a fully automated chef, that can create new dishes from what his clients tell him they like. Without the clients being able to properly express themselves. That's a lot more complicated than following a recipe.
Difficulty of automation goes roughly trains < planes << cars.
Automated trains are simple, but don't provide much value. Automating planes provided value because it's safer than just with human pilots. Automated cars are a different league of complexity.
Repetitive, blunt, manual labor now, and probably much basic legal/administrative/medical work in the near future. But we still pay migrant workers to harvest fruit, and I don't imagine a robot jockey winning a horse race anytime soon.
Driving a car under non-ideal conditions is incredibly complex, and relies upon human communication. For example: eye contact between a driver and pedestrian; one driver waving at another to go ahead; anticipating the behavior of an old lady in an Oldsmobile. Oh, the robots will be better drivers eventually, but it will be awhile. We humans currently manage about one death per hundred million miles; Uber made it all of two million. I expect we'll have level 5 self-driving cars about the same time we pass the Turing test.
Harvesting fruit is far more complex than driving. It's a 3D search through a complex space.
> Driving a car under non-ideal conditions is incredibly complex, and relies upon human communication.
No it doesn't. The rules of the road detail precisely how cars interact with each other and with pedestrians.
> We humans currently manage about one death per hundred million miles; Uber made it all of two million.
Incorrect use of statistics.
Are you making a joke? "a 3D search [for a path that reaches the destination safely and legally] through complex space" is exactly how I would describe driving. (Also, driving is an online problem.)
I mean, yeah the topology is not as complex as a pure unrestricted 3d space but it's also more complex than pure 2d space. It's a search through a space, and it's complex, I don't know if nitpicking about the topology adds a lot here?
Even navigational paths that consider all of the junctions, ramps, etc. are simply reduced to a weighted graph with no notion of any dimensions beyond forward and backwards.
The solutions to not killing people whilst driving aren't rocket science but too many humans seem to be incapable of respecting the rules.
[0]: http://www.slate.com/articles/technology/future_tense/2017/1...
If self-driving cars are, at their best, roughly as capable as a human driver.
This is a big 'if'.
The solution to not killing people is a kind of rocket science. In fact, it's probably harder than rocket science[0]. It's predicated on a lot of things that are very, very, very hard. The fact is that humans, who are already pretty capable of most of these very very hard things, often choose to reduce their own capabilities.
If the best self-driving tech is no better than a drunk human, however, then we haven't gained much.
---
[0] though perhaps not harder than brain surgery.
However, the process your describing, of collecting human-level performance data, requires the ability to gather all of the data relevant to the act of driving in a manner consumable by the algorithm in question. This is the simulation problem, and it's very, very, very hard (it's why genetic algorithms have traditionally not gotten much further than toy examples, in spite of being a cool idea). Perhaps it is the case that it is very important to have an accurate model of the intentions of other agents (e.g., pedestrians) in order to take preventative action rather than pure reaction. Perhaps it is very important to have a model of what time of day it is, or the neighborhood you're driving in. The likelihood that it is going to rain some time in the next hour. Whether the stock market closed up or down that day.
It also assumes that neural networks (or the more traditional systems used elsewhere) are sufficiently complex to model these behaviors accurately. Which we do not yet have an answer to yet.
So, when I say, 'a big if', I mean for the foreseeable future, barring some massive technological/biological breakthrough. That could be a very long time.
I do think that self-driving cars will be safer, but it's upon it's proponents to prove that.
Each of the crashes that self-driving cars can be fixed and prevented from happening again. The list I gave are human flaws that will almost certainly never be fixed.
I further agree with you it's up to the proponents to prove that. It's a good thing to force a really high bar for self-driving cars. Then assuming the technology is maintained once AI passes the bar it should only ever get better.
If those situations recur exactly as they happened the first time, sure they can be prevented from happening again.
That is, if a car approaches the exact same intersection as the exact same time of day, and a pedestrian that looks exactly like the pedestrian in this accident crosses the street in exactly the same way, with exactly the same other variables (like all the other pedestrians and cars around there that the sensors can see), the data could be enough the same that the algorithm will detect it at close enough to the original situation to avoid the accident this time.
But it's not at all clear how well their improvements will generalize to other situations which humans would consider to be "the same" (ie. when any pedestrian in any intersection crosses any street).
Not if you put neural networks / deep learning in the equation. This stuff is black boxes connected to black boxes, that work fine until they don't, and then nobody knows why they failed - because all you have is bunch of numbers with zero semantic information attached to them.
The only questionable parts will be where the vision system fails, and those are similar actually to human problems. Because human vision also often fails (sunlight on windshield, lack of attention, darkness, etc.)
Are you in very vague words implying that AGI has been invented? AI might have matched humans in image recognition, but it is far away in general decision making.
And finally, I am tired of listening to "safer than a human". That should never be the comparison, but a human at the helm and an AI running in the background which will take over when the human does an obvious mistake -- you know, like a emergency braking system,
One of the big questions I have about autonomous driving is if it's really a better solution to the problems it's meant to solve than more public transportation.
Do we even know yet what's happened?
It seems rather in bad taste to take someones death, not know the circumstances then wax lyrical about how it matches what you'd expect.
It boggles my mind that a forum full of computer programmers can look at autonomous cars and think "this is a good idea".
They are either delusional and think their code is a gift to humanity or they haven't put much thought into it.
Autonomous cars, as they exist right now, are not up to the task at hand.
That's why they should still have safety drivers and other safeguards in place. I don't know enough to understand their reasoning, but I was very surprised when Waymo removed safety drivers in some cases. This accident is doubly surprising, since there WAS a safety driver in the car in this case. I'll be interested to see the analysis of what happened and what failures occurred to let this happen.
Saying that future accidents will be "unexpected" and therefore scary is FUD in its purest form, fear based on uncertainty and doubt. It will be very clear exactly what happened and what the failure case was. Even as the parent stated, "it saw a person with stripes and thought they were road" - that's incredibly stupid, but very simple and explainable. It will also be explainable (and expect-able) the other failures that had to occur for that failure to cause a death.
What set of systems (multiple cameras, LIDAR, RADAR, accelerometers, maps, GPS, etc.) had to fail in what combined way for such a failure? Which one of N different individual failures could have prevented the entire failure cascade? What change needs to take place to prevent future failures of this sort - even down to equally stupid reactions to failure as "ban striped clothing"? Obviously any changes should take place in the car itself, either via software or hardware modifications, or operational changes i.e. maximum speed, minimum tolerances / safe zones, even physical modifications to configuration of redundant systems. After that should any laws or norms be changed, should roads be designed with better marking or wider lanes? Should humans have to press a button to continue driving when stopped at a crosswalk, even if they don't have to otherwise operate the car?
Lots of people have put a lot of thought into these scenarios. There is even an entire discipline around these questions and answers, functional safety. There's no one answer, but autonomy engineers are not unthinking and delusional.
It is not that we think that software is particularly good, it is that we have a VERY dim view of humanity's ability to do better.
[1] https://www.nhtsa.gov/press-releases/usdot-releases-2016-fat...
In fact, NHTSA statistics includes miles driven under adverse conditions (rain, snow, etc) while I'd bet that this is not the case for Uber.
Considering that humans would likely slow down if they see a pedestrian - even if one appeared suddenly - this is even more disconcerting.
I doubt that's true, but even if it were, I believe the rules of the road are "always yield to pedestrians, even if you have the right-of-way".
It is not possible to extract more punishment on an individual than their death, technically, unless you believe in an afterlife.
It is not punitive and meant to 'correct' the builder; it is meant to 'prevent' the builder from cutting corners.
Not true at all. Not that I'd recommend it, but there's torture, and punishing family. They were far less outrageous in Ancient times.
JFYI, it is most probably apocryphal: https://skeptics.stackexchange.com/questions/18558/were-roma...
still it's a nice one, I have heard the same about engineers/architects in ancient Babylon and Egypt.
Hammurabi's code:
> If a builder builds a house for a man and does not make its construction firm, and the house which he has built collapses and causes the death of the owner of the house, that builder shall be put to death.
> If it causes the death of the son of the owner of the house, they shall put to death a son of that builder.
https://www.fs.blog/2017/11/hammurabis-code/
Heard about it on Econtalk's recent episode with Nassim Taleb.
http://journaltimes.com/news/local/confronting-a-legacy-of-a...
"In order to gain the trust of the employees he asked his wife to enter the building while the movement was taking place."
Self driving tech is up against the marketing arm of the auto-industry, they've got to be better than perfect to avoid public backlash. If they're a bit slower but far safer then I think they'll do well.
o At speed we're pretty ok with cars hitting things people drop on the road, examples of cars hitting wagons and babies are already plentiful
o Visual recognition & rapidly updated multi-mode sensor data, backed by hard-core neural networks and multi-'brained' ML solutions, have every reason to be way better at this job than we are given sufficient time... those models will be working with aggregate crash data of every accident real and simulated experienced by their systems and highly accurate mathematical models custom made to chose the least-poor solution to dilemmas human drivers fail routinely
o AIs have multiple vectors of possible improvement over human drivers in stressed situations. Assuming they will bluntly stop ignores how vital relative motion is to safe traffic, not to mention the car-to-car communication options available given a significant number of automated cars on the road -- arguably human drivers can never be as smooth as a fleet of automated cars, and "slamming the brakes" is how people handle uncertainty already
Presently the tech is immature, but the potential for automated transport systems lies well beyond the realms of human capabilities.
Braking distance (without including any decision time) for a 15mph car is 11 ft, for a 30mph is 45 ft. Self driving cars won't change these limits. (well, they may be a little better than humans at maximizing braking power through threshold braking on all 4 wheels, but it won't be dramatically different)
So even with perfect reaction times, it will still be possible for a self-driving car to hit a human who enters its path unexpectedly.
In fact, self-driving cars may actually improve the situation if cars actually start complying with speed limits en masse.
There's a 30mph city street near me where cars routinely go 45mph -- the fatality rate jumps up to 60% at that speed. So just having cars follow the speed limit would go a long way toward reducing fatalities.
There is a fundamental inequity between the operator of dangerous equipment comparing hours and years of life, and the pedestrian who suffers the consequence of that comparison. The USA auto industry is built on this inequity, which is why no one ever talks about it.
Lives, dollars, wasted time are all fungible.
Your new, inequity point adds externalities to the discussion. That's fine, but those are also measured in lives, dollars, wasted time.
But I shouldn't pick nits; you were speaking in generalities! So when I said "any transportation goal" and you disagreed, you hadn't actually thought of a particular exception to my universal statement. And you still haven't thought of one. Are your contributions to this discussion offered in good faith?
2. Auto plant needs radiators. Truck delivering them takes longer to get them there. Costs $x more. Car costs more. Car buyer has to work longer to buy car. Car buyer wastes y hours of life.
There are lots of alternatives that don't involve you spending more time driving.
While it's true that goods will cost more to transport if it takes longer, that higher charge is amortized across many products in the truck, so is a very small portion of the finished product.
So if a radiator fits in a box 16x24x6" or 1.3ft^3 and a 40ft truck holds 2400 ft^3 (subtract 20% since it won't be a perfect fix, so 2000 ft^3, so you can fit 1500 of them in the truck.
If a truck+driver costs $100/hour, that means each radiator will cost 13 cents more.
Or, another way at looking at it -- all of the parts that make up a car aren't going to be bigger than a car (sure, some space is lost to packaging, but there's a lot of empty space in a car), and 6 - 10 cars can fit on a car carrier truck, so each car will end up costing around $25 more.
Though since we're talking about urban speed limits, and there aren't many urban car manufacturers, slow urban speed limits won't affect the price of cars.
But what about an even lower speed? If 15 mph is good, then 5 mph is better. And if 5 mph is better, 1 mph is superior once more.
I think we can agree there is a point where slow becomes too slow and the 'sufficient planning' becomes an unreasonable burden. So given we aren't operating off the notion that slower is inherently better, then there is some equation giving us our optimal point. What if that point is 45 mph instead of 15 mph?
In short, how do we argue that 15 mph is better than 45 mph that can't also be applied to speeds lower than 15 mph?
Often speeding just gets you to the next intersection, or red lighter, quicker, where you then have to wait longer for traffic, or a green light.
(edit: just to be clear, I'm not in support of speeding, however I do value good logic.)
No one is saying reduce speeds everywhere. But in an urban context with lots of pedestrians, these speeds matter, and urban traffic is generally so stop-and-go and congested that drivers rarely sustain the top speed, and reducing it doesn't actually affect travel time by all that much.
However comforting that logic may be, it is unusable in the real world. If you value lives infinitely, then you will never ever do anything that risks your or somebody else's life in order to gain on any other need. You routinely engage in things that are not the safest possible option in order to fulfill other needs, ranging from food or water acquisition through mere entertainment. Therefore you place a finite value on your life. Don't feel bad, so does everybody else. It is possible to determine the value placed on life, I believe there are studies that show the value is more stable than you might think, and balance things appropriately.
It may be uncomfortable thinking, but, again, unless you literally never take even the smallest risk in the pursuit of other goals, you are already thinking this way. You just haven't lifted it up to the conscious level yet.
I'd say you're almost right about nobody asking for infinite value, but I'd say it's more like nobody who has pulled this up to the conscious level is asking for infinite value. People who have not examined the belief are quite prone to speaking as if life's value is infinite... but their own actions inevitably belie that claim. Once examined, it becomes rationally obvious that life is not infinitely valuable (including your own), but, well, if humans automatically accepted and believed all rational things they examine without emotional consequence the world would be a very different place.
http://www.nytimes.com/2011/02/17/business/economy/17regulat...
For evaluating safety regulations relative to the cost of inplementation, NHTSA values the risk of loss of human life in relation to the market value of risk reducing products, and the safety they provide.
They extrapolate from the take rate of airbags and their cost and effectivity, to how much value the average American places on their own life. IIRC on the order of $5 million.
FYI, juries do not look kindly on companies that implement this in liability suits. The data is that if a corporation writes down a $-figure for a (statistical) human life, it anchors punitive fines at a higher level.
Universally adhered to lower speed limits in urban environments would be great.
Conduent offers a turnkey solution for this. They provide and manage speed cameras: https://www.conduent.com/solution/transportation-solutions/r...
Most of them are empty but people unfamliar with the place will usually slow down.
The vast majority of people just go however fast they feel comfortable (considering conditions, etc) regardless of the speed limit.
Mixed traffic speeds decrease safety.
Raising speed limits so that you don't have the % of people who comply with the letter of the law traveling slower than the people who go however fast they're comfortable usually improves safety.
Unless your goal is to increase ticket revenue or appease the "think of the children crowd" there's no point to lowering speed limits. It doesn't do much to affect traffic speed. To do that you have to modify the road or do something to change the traffic flow.
Self driving cars will improve safety because they'll result in political pressure to raise speed limits to match reality and they'll make dynamic speed limits more practical.
The trick is to make people feel like driving fast is unsafe, without actually making driving any unsafer.
Sure, but city streets are already mixed traffic. There are pedestrians, bikes, vehicles parking or turning, etc. It's not reasonable to raise the limit to what people want to drive and just ignore all the other users of the street.
Also, the optical narrowing mentioned in a sibling comment is quite effective. They've done that on a few streets near me via things like sidewalk bulb-outs at intersections, and swapping the parking lane & bike lane (so it goes curb-bike-parking-drive, rather than curb-parking-bike-drive). Everyone drives more slowly on those streets now - myself included.
For more, see https://www.vox.com/the-big-idea/2016/11/30/13784520/roads-d...
Exactly. That is why we should ban all cars anywhere there is people, and they should be limited to highways. Because safety is most important, right? Right???
It’s very simple, put speed cameras on every corner, and fine in terms of day wages. E.g., one week of your income as fine for going x% above.
Several countries are doing parts of this already, or moving towards it.
(IMHO) jamitons would dissolve if people would leave a flexible buffer between them and the car in front of them and focusing on minimizing braking, rather than driving up to their bumper, brake, wait for moving, accelerate, brake, repeat. The lag due to reaction time and ac/deceleration exacerbates the "viscosity" of traffic flow. If most drivers focused on "staying fluid" rather than hurry-up-and-wait, traffic ought to improve. Like fluidized beds.
If a car can't observe any potential hazards that might impact it using different threat models it should drive more slowly. Blowing down a narrow street with parked cars on both sides at precisely the speed limit is not a good plan.
And then he started me on watching for suspicious gaps in the parked cards along the side that could indicate a loading bay or a driveway or an alley or a hidden intersection. And so on though multiple categories of collision hazards, and then verbally indicating them to him while I was driving.
And the reason for that exercise was to drive home the point that if there's a vehicle or a person that could get into my lane, it's my job as a defensive driver to be aware of that and be ready to react. Which includes making sure I could stop or avoid in time if I needed to.
I don't know how driving is taught now, but I would hope a self-driving system could at the very least match what my human driving instructor was capable of.
But they should be willing to walk in front of it in an in-spec performance regime. There's some really good Volvo commercials along that line, with engineers standing in front of a truck.
Apologies for going off topic here, but I'm curious about this. I've tested every car I've ever owned and all of the recent cars with all-round disc brakes have outperformed this statistic, but I've never been able to get agreement from other people (unless I demonstrate it to them in person).
I'm talking about optimal conditions here, wet roads would change things obviously but each of these cars was able to stop within it's own car length (around 15 feet) from 30mph, simply by stamping on the brake pedal with maximum force, triggering the ABS until the car stops:
2001 Nissan Primera SE
2003 BMW 325i Touring (E46)
2007 Peugeot 307 1.6 S
2011 Ford S-Max
I can't work out how any modern car, even in the wet, could need 45 feet to stop. In case it's not obvious, this is only considering mechanical stopping distance, human reaction time (or indeed computer reaction time which is the main topic here) would extend this distance, but the usual 45 feet from 30mph statistic doesn't include reaction time either.
I learned about this feature when reading car forums for my car and finding threads from people who were rear-ended when they accidentally triggered this feature by slamming on the brakes when they didn't intend to come to a complete stop.
Which is utterly stupid. Braking is the natural reflex, but not always the right one. I've been in more than one close call (think left turn on incoming traffic) where the correct response was not to floor the brake, but floor the gas.
> people being taught to pump brakes, which increases braking distance in cars with ABS.
Which is a mechanical turk version of what the ABS is doing under the hood.
http://www.government-fleet.com/content/driver-care-know-you...
http://www.brake.org.uk/facts-resources/15-facts/1255-speed
Car and Driver did a test with sports cars and profesional drivers and came up with 142 - 155 ft from 70mph, while my first reference quotes 245 ft (around 40% less, so extrapolating, their stopping distance from 35mph would be around 20 feet).
https://www.caranddriver.com/features/rocket-sleds-the-best-...
The average car on the road is not a sports car with performance tires and is not stopping on a clean, dry track. So I don't think it's a stretch to assume that an average car on average roads with tires optimized for tread life would be 40% worse than a $100K sports car with $400 tires that are optimized for grip rather than lifetime.
The 45 feet from 30mph is a common figure (the UK government's Highway Code uses it as well).
The cars I tested are normal cars, but I concede I had good tyres and I always choose smooth roads to test on. Once you include:
1. Driver ability
2. Vehicle quality
3. Road quality
4. Prevailing conditions
5. 4 passengers plus luggage
I guess you can explain the difference.
How did you measure that? Because plugging these figures into a uniform acceleration calculator, 50km/h to 0 in 4.47m requires a deceleration of 2.2g but "Analysis of emergency braking of a vehicle"[0] experimentally measured very best case deceleration as barely scraping 1g (with ABS at 80km/h, significantly lower at lower speeds or without ABS).
[0] https://www.tandfonline.com/doi/pdf/10.1080/16484142.2007.96...
Cameras are not accurate enough though, their dynamic range being terrible. Wonder how humans would fare if forced to wear goggles that approximated a lidar sensors information.
Theoretically self-drivers will always see everything that is relevant, unlike a human driver. And theoretically a robot-driver will always react more quickly than even a hyper-attentive human driver, who has to move meat in order to apply the brake.
From an ethical standpoint the interesting phase will only start then. It‘s one thing to bring a small fleet of high tech (e.g. having LIDAR) vehicles to the road. It‘s another to bring that technology to a saturated mass market which is primarily cost driven. Yes, I assume self-driving cars will eventually compete with human driven ones.
Will we, as a society, accept some increase in traffic fatalities in return for considerable savings that self-driving cars will bring?
Will you or me as an individual accept a slightly higher risk in exchange for tangible time savings?
I believe the claim is 38 multitudes better than humans, significantly better than marginally.
> accept some increase in traffic fatalities
No. And the question is more about "some" than "some increase"
> accept a slightly higher risk in exchange for tangible time savings?
Texting while driving and even hands free talking were becoming laws in many states before smart phones -- and my experience is that many people readily accept this risk and the legal risk just to communicate faster. The same can be said for the risk of drunk driving -- it's a risk that thousands of Americans take all of the time.
Some airlines do not allow pilots to manually fly above 3k feet, nor allow first mates to land manually[0].
0: https://www.wired.com/story/boeing-autonomous-plane-autopilo...
There is a (prevalent) happy path where autopilot and autoland can get you the whole illusion "but the plane flies itself" - but the whole system is built around humans catching and handling all and any exceptions.
A USAF C-54 made a complete transatlantic flight (takeoff and landing included) on autopilot back in 1947: https://www.flightglobal.com/pdfarchive/view/1947/1947%20-%2...
Modern Cat IIIc (zero-zero) autopilots handle everything except taxiing and takeoff. And I think the takeoff thing is more political than technical.
Sounds silly when compared against old tech.
Accidents happen, best we can do is try to prevent them.
It's impossible to fully remove the issue. Vehicles still need to react to and reduce accidents, but they will absolutely never eliminate them.
Although the actual logistics of your proposal might be challenging (child comments point out that some speeds/distances might be impossible to solve) your instinct is a correct one: the people designing and deploying these solutions need to have skin in the game.
I don't think truly autonomous cars are possible to deploy safely with our current level of technology but if I did ... I would want to see their family and their children driving in, and walking around, these cars before we consider wide adoption.
also that testing does happen in the case of every av program i know of. closed obstacle courses with crap that pops out forcing the cars to react. look up gomentum station.
† i did it for you. http://gomentumstation.net/
IOW, setting this up as some kind of quality standard gives unjustified cover ("Hey, our own CEO risked his life to prove the car was safe!") if AVs fail on the open road, because the requirements of open and closed tests are so different.
Just like they take the repair mechanics on the first test flight after a major repair of a big airplane.
We need law enforcement to be able to keep pace with advances in technology and let's face it, they're not going to have the money to employ data analysts with the required skills. Do we need a national body for this? Is there any hope of a Republican government spending the required money to do so? (no)
On the other hand, I'd wonder if increasing NTSB scope this much would drastically decrease the average quality of NTSB's work. Scaling ain't easy.
Bonus: Job creation to replace the jobs lost to automation
For instance, take a look at:
https://www.ntsb.gov/investigations/AccidentReports/Pages/HW...
"The Tesla was equipped with multiple electronic systems capable of recording and transmitting vehicle performance data. NTSB investigators will continue to collect and analyze these data, and use it along with other information collected during the investigation in evaluating the crash events."
I mean, how is this different then all of the other accidents that occur every day? Yes a self-driving car is involved, but do people really think autonomous cars aren't going to be involved in fatal accidents?
Of course they are...but I've always thought that autonomous vehicles only have to be like 10% safer for them to make tons of sense to replace human drivers.
We want to know what happened, how it happened, how we could have prevented it, how likely it is to happen again, what assumptions or overlooked details lie at the heart of this.
The required level of expertise, effort and precision here are higher than in a regular traffic accident. Moreover, the required skill-set, knowledge base, and willingness to work in a new area here make this an exceptional case.
Finally, the outcome of this will be much more than liability in a single case. This could set the precedent for an entire field of industry. This could be the moment we find out self-driving cars are nearly a pipe-dream, or it could be the moment we kill self-driving cars at the cost of millions of preventable traffic accidents. This investigation just might determine a lot, again, that makes it exceptional.
The latter needs to be addressed by government. The former can be addressed by concerned parties using their political connections to get existing law aggressively enforced.
I guess we'll find out how competent they are at this, but I think they did a surprisingly good job with the previous Tesla investigation: https://dms.ntsb.gov/pubdms/search/hitlist.cfm?docketID=5998...
I don't think self-driving cars and their sensor data, even if they rely on the operator to explain what the car “remembers”, fundamentally shift the landscape.
Agree, Kumail Nanjiani (comedian) has a great rant on twitter about exactly this, ethical implications of tech-
> As a cast member on a show about tech, our job entails visiting tech companies/conferences etc. We meet ppl eager to show off new tech. Often we'll see tech that is scary. I don't mean weapons etc. I mean altering video, tech that violates privacy, stuff w obv ethical issues. And we'll bring up our concerns to them. We are realizing that ZERO consideration seems to be given to the ethical implications of tech. They don't even have a pat rehearsed answer. They are shocked at being asked. Which means nobody is asking those questions. "We're not making it for that reason but the way ppl choose to use it isn't our fault. Safeguard will develop." But tech is moving so fast. That there is no way humanity or laws can keep up. We don't even know how to deal with open death threats online. Only "Can we do this?" Never "should we do this? We've seen that same blasé attitude in how Twitter or Facebook deal w abuse/fake news. Tech has the capacity to destroy us. We see the negative effect of social media. & no ethical considerations are going into dev of tech.You can't put this stuff back in the box. Once it's out there, it's out there. And there are no guardians. It's terrifying. The end. https://twitter.com/kumailn/status/925828976882282496?lang=e...
It is scary, big tech orgs have no incentive or motivation to even consider ethical implications, whats worse is the American consumer has shown repeatedly that you're OK to do really shady stuff and as long as it means a lower price product/service for the consumer. We're in a kind of dark age of tech regulation and he's right it is terrifying.
Though that would of course depend on how insurance even looks for SDCs. Maybe big companies like Uber will self-insure.
"[Uber] said it had suspended testing of its self-driving cars in Tempe, Pittsburgh, San Francisco and Toronto"[1]
1: https://www.nytimes.com/2018/03/19/technology/uber-driverles...
I was resumed shortly after.
27 Mar 2017 (1 year ago)
https://spectrum.ieee.org/cars-that-think/transportation/sel...
"The Uber vehicle was reportedly driving early Monday when a woman walking outside of the crosswalk was struck.
...
Tempe Police says the vehicle was in autonomous mode at the time of the crash and a vehicle operator was also behind the wheel."
That's a very bad look; the whole point of self-driving cars is that they can react to unexpected circumstances much more quickly than a human operator, such as when someone walks out into the road. Sounds like Uber's platform may not be up to that standard yet, which makes me wonder why they're on public roads.
On the other hand, it sounds like it happened very recently; I guess we'll have to wait and see what happened.
This tendency to buzzword out of obvious truths (NN's are programs for example) derails rational discussion.
https://news.ycombinator.com/newsguidelines.html
Doing these things derails discussion much faster than an obvious comment, so would you please (re-)read the rules and abide by the spirit of this site when posting here?
AI (short of AGI) is never guilty; it's creators and operators, OTOH...
Perhaps there will be unexpected 'issues' meaning that such data will have been lost in this case.
1. Uber really had a tool that did this IRL: https://www.theverge.com/2018/1/11/16878284/uber-secret-tool...
E.g. Brazil was at odds with WhatsApp over their message encryption — Brazil wanted WhatsApp to stop encrypting messages so that it would be possible to subpoena chat logs. If Brazilian police raided a hypothetical WhatsApp Brazil office, would it not be prudent for Facebook to cut off data access from that office? Especially given that the nature of technology means that once police have an employee’s laptop and can compel them to enter their password, they have access to essentially all of the company’s data worldwide.
But let's extrapolate. Say one day there are 20 self driving car companies. Should they be required to share what they learn so the same mistakes aren't repeated by each company or does the competitive advantage outweigh the public benefit from this type of information sharing?
They seem to be adjusting quite a bit in the last 1-2 years so that is the definition of learning. Otherwise, just showing how companies were once dumb is usually not very useful in knowing how adaptable they are.
I agree that we’ll know better looking back how they react to this event.
Should "not killing pedestrians" be a competitive advantage? No. It should be mandatory for every vehicle.
Therefore,we should make sure that everything works towards this goal, including sharing data!
The argument I've always heard is that an autonomous systems will outperform humans mostly by being more vigilant (not getting distracted, sleepy, etc.) rather than using detectors with superhuman reaction times. Obviously, whether or not this outweighs the frequency of situations where the autonomous system gets confused when a human would not is an empirical question that will change as the tech improves.
And, like, once the systems are mature and hardware has evolved and so forth, I think that's right. It will. It might today under certain circumstances, like if it gets a really unambiguous sensor input (or it might not).
But I've never heard anyone actually associated with a driverless car program assert that their vehicles have superhuman reaction times today, and the vehicles drive extremely cautiously. I think it's likely that due to their difficulty in understanding their sensor readings, if you look at total time necessary to make a course correction from the point when an obstruction first could be noticed by an alert human driver, driverless cars are not winning and may be substantially losing in at least some cases.
From what I've seen I wouldn't trust autonomous cars to "understand" all situations. I would trust Waymo cars to understand enough to avoid hitting anything (at a level better than a human), at the risk of being rear-ended more often. Everything I've seen from Tesla and Uber has given me significantly less confidence than that.
I agree with the sentiment though. This has been a major selling point for this technology, but it has not been sufficiently demonstrated yet.
(It certainly didn't predict the crash "seconds before", as the car accelerated into the imminent colission, until <1sec before collision.
Something that is endemic in HN discussions of driverless cars is commenters who dramatically overestimate the dangers of humans driving. Like an order of magnitude or more. You see it all over this thread, in which people imagine that killing one pedestrian in like let's say 10 to 20 million miles driven (with those miles overwhelmingly done in unchallenging conditions) constitutes "vastly safer than human drivers" rather than "vastly less safe than human drivers."
Driverless cars simply don't have the mileage yet to prove they are safer than driverful cars. But there's also no indication they are less safe, based on casualty rate so far.
There is an indication that they're less safe! They have (very conservatively) 5x the number fatalities per mile driven! Now, look, the error bars on that estimate are of course massive. It is plausible that they are much safer and they just rolled the dice and got unlucky. But this is data, as long as you include the error bars.
Toyota has enough UX data that they added "brake assist", turns out that a lot of accidents happen when the user stomps the brake, but then releases, just never presses it hard enough to come to a complete stop, or was trained before anti lock brakes.
https://www.youtube.com/watch?v=grcuorbrYxA
Another scary thing about braking is that most cars probably don't have their brake fluid changed often enough; most owners and even dealers treat it as a lifetime interval, when it may be closer to an annual thing interval.
That's an invalid conclusion to draw from this accident.
There were 34,439 fatal motor vehicle crashes in the United States in 2016 in which 37,461 deaths occurred. This resulted in 11.6 deaths per 100,000 people and 1.16 deaths per 100 million miles traveled.[1] As far as I know, the instance this article is about is the first death in an autonomous vehicle accident, meaning that all the 2016 accidents were humans driving.
Why is it that you see one death from an autonomous car and conclude that autonomous cars aren't ready to be driving, but you see 37,461 deaths from human drivers and don't conclude that humans aren't ready to be driving?
I admit that there just aren't enough autonomous cars on the road to prove conclusively that autonomous cars are safer than human-operated cars at this point. But there's absolutely no statistical evidence I know of that indicates the opposite.
[1] http://www.iihs.org/iihs/topics/t/general-statistics/fatalit...
EDIT: Let's be clear here: I'm not saying autonomous cars are safer than human drivers. I'm saying that one can't simply look at one death caused by an autonomous car and conclude that autonomous cars are less safe.
I'm not sure I see anyone here making that conclusion. I think you're the only one who's brought it up.
For one, I personally sure as fuck do not think humans are ready to be driving.
> That's an invalid conclusion to draw from this accident.
The conclusion was not invalid at all. Other self-driving car companies have driven more miles than Uber has and they have done so safely. Uber has even taken its cars off the road, so even Uber agrees that their self-driving cars are not ready for the roads yet.
It is also important to take into account what Uber is like when it comes to safety and responsibility. They have knowingly hired convicted rapists for drivers, they have ridiculed and mocked and attempted to discredit their paying customers who have been raped by their employees/drivers, they have spied on journalists, they have completely disregarded human safety on numerous accounts. A company with a track record like Uber's probably not should be granted a license to experiment with technology like these self-driving cars on public roads.
They just aren't responsible enough.
Look at the post I'm responding to, and the section I quoted.
> Uber has even taken its cars off the road, so even Uber agrees that their self-driving cars are not ready for the roads yet.
Uber avoiding a PR nightmare should not be taken as Uber thinking their self-driving cars aren't ready.
> It is also important to take into account what Uber is like when it comes to safety and responsibility. They have knowingly hired convicted rapists for drivers, they have ridiculed and mocked and attempted to discredit their paying customers who have been raped by their employees/drivers, they have spied on journalists, they have completely disregarded human safety on numerous accounts. A company with a track record like Uber's probably not should be granted a license to experiment with technology like these self-driving cars on public roads.
I'm not defending Uber as a whole. I think in general they're a fairly typical rent-seeking middle man whose only real innovation has been figuring out a way to break the law ambiguously enough to get away with it (admittedly, a law I don't agree with). I don't even necessarily think autonomous vehicles are a good thing. I just think that if you're going to criticize autonomous vehicles, safety isn't the criticism I'd level against them, because it's not backed up by the evidence when compared to how unsafe human drivers are.
https://www.newscientist.com/article/2095740-tesla-driver-di...
The fact that human drivers aren't particularly good isn't really relevant, beyond setting a correspondingly low bar within the null hypothesis.
This all ties to regulation allowing these vehicles to drive on public roads, because that regulation was permitted due to the above hypothesis and hopeful expectations that it would be true.
Obviously, I haven't seen the entirety of the data set to know fatalities per car-mile. Which would be the relevant statistic here. I also didn't see such a number in your post, which I'm assuming means you are probably not aware either. But simply providing the numbers for the null hypothesis doesn't do anything.
But that's not really my point. My point isn't that self-driving cars are safer, it's that merely looking at one car accident doesn't prove that self-driving cars are less safe.
https://www.forbes.com/sites/bizcarson/2017/12/22/ubers-self...
so, 6 million miles driven and one fatality is significantly worse record than human drivers. So far.
If self-driving cars are between dumb drivers and normal drivers, and self-driving cars take dumb drivers off the road, it may be a net positive.
Let's say we have 9 normal drivers and 1 dumb driver. The 9 normal drivers cause 0 accidents per year each, and the 1 dumb driver causes 15 accidents per year. Let's say autonomous cars cause 1 accident per year. If autonomous cars replace all the drivers, there's 10 accidents per year instead of 15.
Obviously these are hypothetical numbers. All I'm saying is that if you only have one of these numbers, you don't know which kind of driver is safer.
https://techcrunch.com/2017/11/27/waymo-racks-up-4-million-s...
Nevermind that the initial rollouts almost always involve locales in which driving conditions are relatively safe (e.g. nice weather and infrastructure). Nevermind that the vast majority of autonomous testing so far has involved the presence of a human driver, and that there have been hundreds of disengagements:
https://www.theverge.com/2018/1/31/16956902/california-dmv-s...
Humans may be terrible at driving, but the stats are far from being in favor of autonomous vehicles. Which is to be expected given the early stage of the tech.
That's fine, we don't have to consider just fatal accidents:
https://www.theverge.com/2018/1/31/16956902/california-dmv-s...
> One result of the sharp increase in GM’s number of miles driven is a plethora of accidents. The auto giant’s autonomous cars were involved in 22 fender benders over the course of the reporting period (and two more in 2018). That’s one crash for every 5,985 miles of testing.
I didn't say that.
I'm not saying autonomous cars are safer, I'm saying that this accident doesn't prove autonomous cars are less safe.
> That you pulled out the IIHS stats without apparently bothering to look up the number of reported self-driving miles driven seems to suggest that you assumed that the fatality rate was obviously skewed against human driving.
I brought up the IIHS stats to show that merely reporting an accident in isolation from a full statistical model doesn't prove anything.
> That's a very bad look; the whole point of self-driving cars is that they can react to unexpected circumstances much more quickly than a human operator, such as when someone walks out into the road. Sounds like Uber's platform may not be up to that standard yet, which makes me wonder why they're on public roads.
> On the other hand, it sounds like it happened very recently; I guess we'll have to wait and see what happened.
How in the world do you see that as an assertion of proof.
This metric was often decried because it had poor statistical significance. It would however be nice to update this metric according to this death.
Maybe now this metric would indicate that IA are more dangerous than human, will be interesting if the perception of this flawed metric will evolve in reaction...
I'm curious to see that, do you have a link? That would actually be a valid comparison.
This is Elon musk bragging before the first autopilot accident : https://www.telegraph.co.uk/technology/2016/04/25/elon-musk-...
This is a counterpoint with some fact: http://safer-america.com/safe-self-driving-cars/
And concerning the crash, last update seemed to point driver’s fault... with autopilot... don’t know if final ruling is already out:
https://www.washingtonpost.com/news/the-switch/wp/2017/06/20...
People still need to look both ways when coming out from behind cars.
I want more details.
Some of these accidents are unpreventable by the (autonomous) driver. If a pedestrian suddenly rushes out into the street or a cyclist swerves into your path, the deciding factor is often simply the coefficient of friction between your tyres and the road.
The autonomous vehicle and the human attendant might have made a glaring error, or they might have done everything correctly and still failed to prevent a fatality. It's far too early to say. It's undoubtedly a dent to the public image of autonomous vehicles, but hopefully the car's telemetry data will reveal whether this was a case of error, negligence or unavoidable tragedy.
Like my comment says, let's wait until we have facts before passing judgement.
But from this distance, things don't look good for the technology or the future of autonomous trials on public roads.
The evidence that due care was not taken will pretty much be the existence of the fatality.
There surely will be video, from the car itself and also perhaps third party security or traffic cameras. The level of carelessness we are going to have to see in order for a jury to blame the victim will be pretty high.
As for a fatality proving that due care was not taken, I agree. It doesn't tell you who failed to take due care though.
In law you split that into criminal and civil. Very unlikely there will be any criminal charge, hence no criminal liability but it’s possible.
Civil liability is more interesting, but if the deceased’s estate brings suit it opens up a can of worms in the discovery process. Every internal communication, prior incident, etc...
At the end of the day it’s not much different than any other vehicular fatality case, but there will be a defective product component (again nothing unheard of in the case of vehicular fatalities). At the end of the day the car will be insured for accidents, so we will probably learn more about the insurance coverage for self-driving cars.
When it comes to human drivers we deal with probabilities, but for some reason people want absolutes with autonomous ones.
Most drivers drive too fast. (Me too!) Most drivers have not killed anyone. All drivers who have killed someone, were driving too fast at the time. Many drivers will disagree with me, but they are simply wrong about appropriate speeds, and we will all be safer when robocars are driving for them.
I would say that if you claim that a massively complex system like public traffic, with thousands of participants, many of them badly or completely untrained, can be organized in such a fashion that any and all accidents can be prevented, the burden falls on you to show how.
Which is a lot of high level reasoning and inference with information from a variety of sources which aren't on the face of it strictly related to the driving task.
As long as self driving cars represent an improvement over human drivers, I’m ok with them having a non-zero accident date while we work out the kinks.
How do we handle this for autonomous vehicles? Do we just fine/sue the company that made the vehicle/developed the software? Do we send imperfect human developers to jail because they made a mistake, even if in the grand scheme of things they have saved lives compared to humans being behind every action made by a vehicle?
A big part of the public image for autonomous cars is increased safety, any deaths at their hands starts raising where and how to place the blame - a subject I think very few are prepared for right now, which is likely part of why Tesla explicitly states autopilot needs a human driver present right now, and why Google has been extremely cautious with operator-supervised tests up until recently.
But is it any surprise that Uber had the first self driving car to kill a human being? You can cook in an orderly, careful way, or you can turn the kitchen into a disaster zone and expect other people to clean up for you.
I know which I prefer when it comes to human lives.
And, medicine takes tens of thousands of lives unintentionally, if I recall correctly.
Medicinal testing kills tens of thousands?
But no, medical testing doesn't kill tens of thousands, practiced medicine does. I may have read your post wrong an figured you meant medicine in practice separately from military testing. There's research to suggest mistakes are the third leading cause of death in the US [1]. My point wasn't so much that killing random bystanders is okay, but that there's a level of unintended death in both, and not just in testing. Society decides what's acceptable, and if for miles driven the number of deaths that self driving vehicles cause is lower than the number of deaths caused by human drivers per miles driven, well... I'd say that's a fair way to look at it.
There's no way for autonomous driving to become a reality without people dying?
I think we could achieve autonomous driving without needless deaths along the way. It might take longer, but I'd say it's worth it to, I can't believe I have to make this argument, avoid killing people.
Lastly, the whole point of the human operator is to be the final safety check.
You're right that we have no idea of the cause until the data is analyzed (and the human operator interviewed). Yet, my first thought was, "Of course it'd be Uber."
This only true for the uninitiated, never let it get to that. I drove for Uber/Lyft/Via in New York city so I can experience and study these situation. These sort of accidents are preventable. The following is the basic:
1.) Drive much slower in areas where a pedestrian and cyclist can suddenly cross your past.
2.) Know the danger zone. Before people "jump into traffic" or a cyclist swerve in front of you, they have to get into position, this position is the danger zone.
3.) Extra diligence in surveying the area/danger zone to predict a potential accident.
4.) Make up for the reduce speed by using highways and parkways as much as possible.
It helps that Manhattan street traffic tends to be very slow to begin with. Ideally I will like use my knowledge to offer a service to help train autonomous vehicles to deal with these situation. It has to be simulated numerous times in a closed circuit for the machine to learn what I've learn intuitively driving professionally in NYC.
Around the bars, people stumble into traffic all the time. Any driver, automated or human, would need to anticipate that.
So essentially everywhere? I have seen pedestrians walk unexpectedly into traffic on high-speed 4-lane divided roads with no crosswalks.
> 2.) Know the danger zone. Before people "jump into traffic" or a cyclist swerve in front of you, they have to get into position, this position is the danger zone.
What "position" are you referring to? There are places where you can account for or predict pedestrians. There are also places where you cannot, such as when someone walks into traffic from behind a tall parked vehicle, where you have no chance to see them in advance.
And that'ss with humans behind the wheel: lazy, distracted, slow to react fleshbags that we are.
I'm not sure that a truly unavoidable accident would occur even once a year in a fictional world in which all drivers were perfect and had millisecond reaction speeds.
What is obvious however, is that these situations are so rare as to be irrelevant. In practice accidents are avoidable by the driver of the vehicle - or at least avoidable to such an extent that it's not worth considering the other cases.
Also: although I personally don't object to some rational victim blaming I think it's a little distasteful that we're already speculating about how this must be the victims's fault, when there's simply not enough evidence to make that kind of determination yet. Let's not forget that part of the privilege of being allowed to participate in traffic implies a responsibility not to kill people even when they behave unexpectedly.
For some statistical perspective: if human drivers had as many fatal accidents per mile as uber has, then the average male driver would kill 1 person in his lifetime (men drive more). Clearly: that's absurd; people may cause too many accidents, but not nearly that many - and that's being rather charitable to uber's self-driving vehicles, since they have back-up drivers and thus don't count complicated traffic situations, and safety drivers, and thus may well have caused more accidents by themselves. So going purely by the unusual-ness of such an accident with so few miles, I'd say the initial assumption must be that this is likely a bug in uber's car, even if I'm sure there were contributing factors.
Edit: I guess it's not surprising wikipedia has stats on the influence of alcohol on fatalities, but it tops out at 4 times the legal limit - at which point human drivers are still safer than this (sample size of one...) uber record so far. :-/
Of course not. I'm not saying they're common, merely that they exist. I do not agree with the idea that accidents would go away if drivers were just trying harder, though. There are legitimate unavoidable accidents, and there are also limits to practical human driving.
> Also: although I personally don't object to some rational victim blaming I think it's a little distasteful that we're already speculating about how this must be the victims's fault
To be really clear, I am not blaming the victim here. I have no idea what happened. I'm actually very inclined to blame Uber, though I recognize that's just my personal bias against them.
Near my house there is an arterial I often cross where a large bush on the corner of the intersection obscures the view to the left from the stop sign roughly 10 feet back. I don't just stop at the sign then YOLO through the intersection, I stop, then creep forward, first looking for pedestrians or bicyclist that might step out from the bushes, then at the edge of bushes I stop again and look again both left and right for cars and bikes or pedestrians in the far lane before proceeding across. I often have to stop another time between the sign and the bushes for the pedestrian or bicyclist that just emerged, if I was focusing on getting across the arterial and beating cross traffic I would have killed every one of them.
The idea that cars should drive past tall vehicles at 5 mph is slightly ridiculous. I have never ridden in a car with someone who constantly adjusted their driving speed based on the cars parked on the road. Choosing a reasonable speed? Of course. Extra care at obvious obstructions? Absolutely. Slowing traffic to 5mph because a van happens to be parked? No.
Until there's a detailed report it's really hard to say if it was preventable or not - but I think regardless the optics are bad and this is going to chill a lot of people's feelings on self driving whether or not that is an emotion backed by data.
This holds iff the control input you apply is only braking. Changing the steering angle is generally far more effective for the "pedestrian darts out from hidden place onto road" situation. It's far better to sharply swerve away to ensure that there's no way the pedestrian can get into your path before your car arrive there than it is to stand on the brakes and hope for the best.
Indeed, the faster you're moving, the more you should consider swerving away over braking -- take advantage of that lethal speed to clear the pedestrian's potential paths before he can get into yours.
Yes, this intentionally violates the letter of the traffic laws (and might involve colliding into a a parked or moving automobile on the other side of the road) and also involves potentially unusual manoeuvring on a very short deadline; but it's far better to avoid a vehicle-pedestrian collision even if it's at the cost of possibly busting into the opposing lane / driving off the road / hitting a parked car. Decently experienced drivers can do this, I can do this (and have successfully avoided a collision with a pedestrian who ran out between parked cars on a dark and raining night), there's no fundamental reason that computer-controlled cars can't do this.
...so you probably won't find any support for it in this crowd.
> there's no fundamental reason that computer-controlled cars can't do this.
Sure, there's no fundamental reason you can't swerve but you need to identify the thing in the road you're trying not to hit before you can decide whether to swerve or brake. Self driving cars can't yet reliably ID things well enough to take the only evasive action they know (stopping).
I agree with the thrust of your comment though, and think that a change in the law may be required to allign incentives for machines as well as people to drive humanely.
With self-driving cars this really seems to call for mandatory investigations by a third-party with access to the raw telemetry data. There’s just too much incentive for a company to say they weren’t at fault otherwise.
I mentioned in another comment that something I use to try to improve my own driving is watching videos from /r/roadcam on reddit, and trying to guess where the unexpected vehicle or pedestrian is going to come from.
Here's an example of a pedestrian suddenly appearing from between stopped cars (and coming from a traffic lane, not from a sidewalk), and a human driver spotting it and safely stopping:
https://www.youtube.com/watch?v=wYvKPMaz9rI
Why can't a self-driving car do this?
"the whole point of self-driving cars is that they can react to unexpected circumstances much more quickly than a human"
This statement isn't really true. Do you think Uber is investing in this because it makes their passengers safer? No, they are pretty much immune to any car crashes with a human driver, the risk and insurance hit is assumed by the driver. They are doing this to save money. They don't have to pay a driver for each trip. The safety aspect is just a perk, but Uber will be pushing this full force as long as the self-driving cars are good enough.
I'm more concerned that Uber will "lose" the video evidence showing what kind of situation it was, and we'll never be able to know if a human would have had ample time to react.
There's enough discrepancy here that I'm not sure what happened until we see the onboard data (unlikely to be released to the public) or a statement from a trusted public official like the Tempe police chief.
I find it crazy that so many people think it is. First off, by definition like 49% of the drivers are technically better than the "average driver".
Second, just like with "human-like translation" from machines, errors made by machines tend to be very different than errors made by humans. Perhaps self-driving cars can never cause a "drunken accident", but they could cause accidents that would almost never happen with most drivers.
Third, and perhaps the most important to hear by fans of this "better than average driver" idea, is that self-driving cars are not going to take off if they "only" kill almost as many people as humans do. If you want self-driving cars to be successful then you should be demanding from the carmakers to make these systems flawless.
Average means something else, technically.
And I strongly doubt "average [X]er" would be interpreted as a median by most people. (Except when median and mean overlap)
The quote is "by definition like 49% of the drivers are technically better than the "average driver"." It's obvious to anyone who's not pedantic that they're referring to the median.
As for an example of a mode average: "The average participant's favorite polygon was a triangle."
That's not the original use. That's a response to the (unattributed) idea that self-driving cars would need to be "better than the average driver". In the original context of people giving thresholds for self-driving cars, I disagree that median is what is meant.
> As for an example of a mode average: "The average participant's favorite polygon was a triangle."
But that mode might be 5 out of 40, and I'd just call that a lie. In a lot of distributions the mode just gives you the biggest or smallest number. And it's affected a lot by bucket size; if I measure "how long did you sleep last night" with a resolution of seconds, the mode is probably going to be 0, which is hilariously misleading. Mode, much like a broken clock, might pick a suitable number sometimes, but all the factors you need to check to see if it's suitable basically render the idea of 'mode' redundant. Just use those factors to pick your number. Mean and median are pretty reliable in being useful. Mode isn't.
Of course, to impute a categorical variable in a data set with empty values, for example.
No, that’s not the definition of mean, that’s (something like) the definition of median.
In a skewed distribution (which driving skill may very well be), the mean can be very far from the middle. If a relatively small number of people are extraordinarily bad at driving, most people are above average. If a relatively small number of people are extraordinarily good at driving, most people are below average.
Why this nitpick matters: if the leading point of your thesis depends on is quantifying how many people are above average, you should really know if the average is at the 10th, 50th or 90th percentiles (or somewhere else entirely).
I agree that the _goal_ should be 10x or even 100x better than the best human drivers, but better than average would result in net good and it's hard for me to see an argument in which that's not true because it's less net suffering/death.
>If you want self-driving cars to be successful then you should be demanding from the carmakers to make these systems flawless.
We're still talking about tons of metal moving at very high speeds around other vehicles. That's just not possible. Should my self driving car handle a catastrophic tire failure at 90mph better than me? Absolutely. Is that still a situation that's likely to result in a crash regardless of the best inputs on a compromised system? Yes. There will _always_ be situations in which fatalities can occur with cars or any other kind of vehicle no matter how well they're engineered.
Having seat belts is obviously statistically safer, but when they were first introduced, many people loudly protested that they'd rather be thrown clear in an accident.
It gets worse when a person is sitting in what appears to be the driver's seat. If the car is stopped and that person is looking toward another passenger or down at a phone, nobody will expect the vehicle to begin moving. Making eye contact with the person in that seat is meaningless, but people will infer meaning.
Even on crosswalks. If I just strolled out on a crosswalk without looking, and waiting for drivers who were paying no attention, I’d be long dead.
http://www.nytimes.com/1998/04/05/automobiles/behind-the-whe...
Who exactly do you think is granting Google, Uber, etc. approval for trials like this on public roads? It's going to be some bureocrat with zero ability to gauge what sort of safety standard these companies' projects have reached.
There are no standards here... what were you expecting would happen?
Uber cutting corners and playing fast and loose with legislation? Unheard of!
Here's hoping they get hit with a massive, massive wrongful death lawsuit.
They're on public roads because they've decided that deploying their tech is more important than safety.
I don't see any other explanation. We know that it's pretty much impossible to prove the safety of autonomous vehicles by driving them, so Uber (and almost everyone else) have decided that, well, they don't care. They'll deploy them anyway.
How do we know that? The report by RAND corporation:
https://www.rand.org/pubs/research_reports/RR1478.html
Key Findings
* Autonomous vehicles would have to be driven hundreds
of millions of miles and sometimes hundreds of billions
of miles to demonstrate their reliability in terms of
fatalities and injuries.
* Under even aggressive testing assumptions, existing
fleets would take tens and sometimes hundreds of years
to drive these miles — an impossible proposition if the
aim is to demonstrate their performance prior to
releasing them on the roads for consumer use.
* Therefore, at least for fatalities and injuries,
test- driving alone cannot provide sufficient evidence
for demonstrating autonomous vehicle safety.[1] "even with these methods, it may not be possible to establish the safety of autonomous vehicles prior to making them available for public use"
More to the point, the report notes that new methods to determine the safety of self-driving cars are required.
Which the industry is not exactly falling head over heels trying to develop.
Realistically no one will trust “new methods” and establishing their relevance is really difficult. I would imagine that most of these companies are running lots of simulations, because why wouldn’t you? But how many people will see that and trust it more than data gathered on the road?
No, the whole point of self-driving vehicles is that firms operating them can pay the cheaper capital and maintenance costs of the self-driving system rather than the labor cost of a driver.
I'm very curious to see how they'll investigate this and who will be determined to be at fault (person behind the wheel or Uber). It will likely set a precedent.
Semi-autonomous cars have drawbacks.
If anything, airplanes prove that machines doing most of the work and humans stepping in only when necessary is a proven model.
https://www.usatoday.com/story/travel/columnist/cox/2014/08/...
Airplanes have far less traffic to deal with. And the points at which they deal with traffic (e.g. takeoff and landings) is completely controlled by humans, including many humans outside of the plane.
They're used in low visibility conditions with relatively calm weather, but don't work well in bad weather.
This is preferable, because a child, blind person etc needs to be able to use the autonomous vehicle alone.
This is not how many people drive in non-autonomous cars? Commuting is on auto-pilot for many of us; you don't really remember how you got to work and I see people tying their shoelaces, calling, eating sandwiches, reading the paper, chatting on whatsapp all the time. Sure it's all illegal (in a lot of countries), but people are bored and drove that road 1000x.
No, that is not how people drive in non-autonomous cars. The phenomenon you're referring to is related to how sparsely information is stored to memory when doing a routine task (even when you were fully concentrating). Just because you can't recall an activity in detail does not mean you were not paying full attention.
However, these are test vehicles. The driver's full-time job is to be focused on the road and what the car is doing. They shouldn't have any misconceptions about the need for them to stay focused on the road. Now granted, even then attention will lapse occasionally, just like it does when you are actually driving. But I don't think that being physically in control of the wheel is necessary for maintaining focus at the same level as a good driver. Driver's Ed instructors need this skill. Also some back seat drivers I know are quite good at maintaining strong focus on the road regardless of who is driving :)
http://www.econtalk.org/archives/2015/11/david_mindell_o.htm...
However, in my opinion, the driver and by extension his employer Uber must be held responsible in this case. Uber have a license to test their self-driving cars only if there is a human driver in control at all times. Regardless of the crazy things their software decides to do, the human is there as a person of ultimate legal liability.
It's also plausible that the pedestrian was at fault. Under some circumstances it would be impossible for a human driver to avoid collision with a pedestrian, this may be no different.
Hopefully with self driving vehicles we can start to move beyond this attitude, especially when there should be ample camera footage available to help dispel the usual claims that the person struck "came out of nowhere" or whatever.
I think it's time to point out the obvious, and require that autonomous cars apply the brakes first, and THEN require driver intervention.
And that they be a whole lot quicker to err on the side of braking.
Cameras getting fuzzy ?
Slow down.
Your ML algorithms are showing lower confidence measures for how they classify nearby objects and trajectories ?
Slow down.
nearby vehicles slowing down and you don't know why?
Slow down.
This is inexcusable.
It's not that simple, you're assuming the car even had some indication that something was wrong. For all we know the car's vision was showing high confidence it saw an open road.
Slow down?
(Disclaimer: I'm a bike advocate, so I may have a different perspective on some of this than most.)
Our car-based transportation system is far and away the most dangerous thing any of us accept doing on a daily basis. 40,000 die a year.
But when cases come to court, everyone on the jury has in the back of their mind "that could have been me if I lost concentration at the wrong moment, or made one bad judgement, etc etc."
So penalties are comparatively light for traffic fatalities. Big punishments are only meted out if the case is so egregious -- repeated drug use, flagrantly reckless behavior -- that the jury can be convinced that the driver is different from them.
In other words, drivers don't get punished for doing something dangerous, because everybody on the road is doing something dangerous. They get punished for doing something more dangerous than the norm.
In this case, there's no question that the "driver" is different than the jury -- it's a computer. Now the symmetry that made jurors compare themselves to the accused is broken.
The result, and what self-driving car advocates don't get, is that self-driving cars don't just have to be safer than human drivers to be free of liability, they need to be safe period. In a trial, they don't benefit from the default "could have been me" defense.
That's a HUGE requirement. In fact, it's probably impossible with our current road system. It won't just take better self-driving cars, but better roads and a major cultural change in our attitudes about driving.
As a bike advocate, I welcome this shift, but I also see how deluded many of the current self-driving projects are. Software moves fast, but asphalt and mentalities move slow. We're not years away from a self-driving transportation system, we're decades.
And this trial is just the beginning of that long story.
The chance someone will cause a grave accident and the chance someone is insolvent are not independent variables.
You get some real horror stories in law school about people trapped in burning vehicles with no remedy for the surviving family.
https://en.m.wikipedia.org/wiki/World-Wide_Volkswagen_Corp._...
One way you can solve for this is strict products liability. Normally we ask if there's a defect in design or negligence. That produces costly litigation where a lay panel reviews a bunch of technical engineering documents with no special training, with highly varying outcomes. Instead, just have the manufacturer pay some statutory compensation whenever their product causes harm. The social harm of the thing will be borne by its purchasers.
It's hard to do that with car accidents though, because driver error contributes so much to outcomes, it's often weird to punish the company.
But we could vastly simplify the auto insurance system and improve the safety of these vehicles by forcing them to just warranty against harm once drivers are out as a factor. (Insofar as a warranty is a promise not to do harm with a fixed penalty up front based on breach.)
It would be fitting if such a statutory regime were the earliest primary laws about AI, because that would officially make a requirement that ai does not harm humans into literally the first law of robotics.
Arguments about how relatively safe self-driving cars will be are beside the point. A self-driving car is something that can kill you. The fact that death by self-driving vehicle is especially rare makes it especially scary.
A low accident rate is not a good thing, psychologically. Only when a risk is comfortably common can we comfortably begin to ignore it.
On even the slimest cause, plaintiff's council always tries include the manufacturer in the case. Anyone with deep pockets that they can pull in.
Although, as far as I know, were the jury to become aware of this, a mistrial should be declared.
If they kill fewer people, they can be run by a joint venture of Satan and the Mafia for all I care.
For example, what if instead of 100 people per day dying from human-driven car accidents(where 95 of them are car drivers/passengers, and the other 5 are pedestrians/bicyclists), self-driving cars only kill 30 people per day, but 28 out of 30 are pedestrians/bicyclists?
I can see banning autonomous vehicles in city centers.
I had been assuming that self driving cars would be a 'reset' for our decision to value cars more than the countless lives they end. Self driving cars could take the irrationality and emotion out of driving. It would be America's "stop the murder of children" moment (The Netherlands rallying cry to reform auto traffic in 1970's).
But now I see that it can, and probably will go the other way. These giant companies could push to further restrict the rights of cyclists/pedestrians around roadways, vilify the 'stupid' people who made a reckless decision that resulted in an accident, and push our country even further into car culture.
We ban accounts that do this. Please read https://news.ycombinator.com/newsguidelines.html and follow the rules when posting if you want to keep commenting here.
You're in a bubble if you think a world hyper connected to social media that's in the same world scared that autonomy is about to kill millions of jobs in one jab and would NOT over-react to a single accident.
From the article it's impossible to know if the cyclist was at fault, so I won't jump to conclusions, but we can't, especially as the creators of the future of autonomous vehicles talk so dissociatively about human life. Either we go out into the public roads with very high certainty that OUR software and OUR hardware won't kill someone, or we wait till we get there.
Public opinion on human driving is doing just fine with human drivers right now, so we can and should take our time to get to the future we want or risk a backlash and a guilty conscious.
On one hand, the "just one isn't so bad" argument is rational. In terms of fatalities per mile, self driving cars already seem to be safer than humans as a class. Therefore it would be wrong of me to say that this tech does not belong on the road when I believe I should be allowed to drive.
On the other hand, you said it best. If I ever wrote code running on a machine that killed somebody I don't know how I'd continue in my career. Hell, I work at Google very far from Waymo but if one of our cars struck and killed someone I'd feel some personal responsibility.
So ... I guess it's complicated.
There were over 40,000 motor vehicle deaths in 2016 in the US [1] - more than a hundred daily. While loss of human life is always a tragedy, individual fatalities are absolutely worth it if we can begin to reduce that count sooner than by proceeding with excessive caution.
It would be morally unacceptable to delay development merely to avoid a guilty conscience.
[1] http://www.nsc.org/NewsDocuments/2017/Fatality-estimates-Jun...
There is no reason self-driving cars can’t be tested in private. Companies can hire pedestrians to interact with the cars, and the software can go through the same certification process that buildings and vehicles currently go through.
It’s a false dichotomy to say that you can either have self driving cars or minimally safe and accountable development, but not both.
EDIT: Here is a link to a thread replying to a parent which is now auto-collapsed. https://news.ycombinator.com/item?id=16620968
Does "sufficiently tested" mean you get the end-product a year later? What if the end-product saves 10,000 lives a year? In that case what cost is morally acceptable to get there a year faster?
You can't have the certainty to make such statements without "sufficiently testing" the product.
Can you point me to where I made a statement? I only see questions.
They can't even share what they learn with each other effectively.
Again, why not just test them in private and hire people to be pedestrians?
And get these employees to do what exactly? You obviously can't tell them to put their lives at risk to test what happens if they walk onto a car's path, or if they ride a bike on a car's blind spot wearing dark clothes at night, or if they fall from a motorcycle in front of a car after hitting a raccoon.
The dichotomy here is that private courses are inherently orderly and the real world is inherently chaotic.
How is it not OK to have consenting people do these things but OK to have random people on the street participate in exactly the same tests?
If all you're testing are scenarios that are known to be safe for the employee in question, then what exactly are you gaining from that testing?
> How is it not OK to have consenting people do these things but OK to have random people on the street participate in exactly the same tests?
Are you seriously saying Waymo’s (for example) testing practices are worse than just testing unsafe software in public?
Anyone with half a brain would hopefully quickly realize that you don't actually need a live person to lay down on a skateboard. Heck, you can simulate a much more risky jaywalking scenario with a mannequin on a dolly than with a living person.
Waymo is deploying its fleets on public roads too, which suggests to me that they think that private course tests can only get you so far.
No, it just suggests that building and operating huge private courses that realistically emulate daily traffic situations in cities is much more expensive than just (ab)using the "real" public infrastructure paid for by tax dollars for your beta-testing needs, and that Waymo (just like Uber) takes full advantage of this chance to privatize gains and socialize losses.
Ignoring these facts is just as intellectually dishonest.
Are you really asserting your statements without any fact whatsoever?
The PP was insinuating that people with leraner's permit (and, by law, an experienced driver in the passenger seat) or people who just got a license (and hence were, literally, tested) are "insufficiently tested".
The PP was responding tho the claim that letting "insufficiently tested" systems on the road with the goal of letting them improve is irresponsible.
For the response to have any merit, you need to cite accident statistics for people with learner's permits, or new drivers.
I don't think it's even a very tough argument that these are at best basically limited filters on actual driver skill. I've known people who literally went to another city for favourable conditions for their driver test. I've known of people who passed having driven not much more than a few hours in their lives.
Nowadays if you want to be the driver in the passenger seat for a learner you need to do a somewhat more difficult test and be older.
Also, I'm really not talking about this specific case but I think it's particularly relevant that in this case there was a qualified driver able to take over for the autonomous vehicle, which is actually more supervision than a 14 year old with a learner's permit has.
The post I'm responding to, from my perspective, has a flippant attitude towards the deaths caused by human drivers.
There's definitely going to be some osmosis but I think you're pretty vastly overstating it (and including several years in which your ability to even comprehend what traffic is is going to be severely limited).
Certainly in the long run we would want to test in the real world. The argument is whether or not we are there yet. Other commenters have made a compelling argument, again based on the statistics, that we are not there yet. Do you disagree, and if so, based on what facts?
What I did say is that with deep learning it simply is not possible to "test" in a simulated environment to any level of certainty.
You can use simulated environments with test subjects to help develop such as system (and should do so). But you will never be able to adequately test such a system in this environment.
Why? The state space of traditional robotics and engineering problems is far more constrained. So much so that you can't even compare the fields in my opinion.
https://medium.com/waymo/waymos-fleet-reaches-4-million-self...
If anything, we are morally obliged to double down.
Self driving cars can also be tested in real life environments with safety drivers behind the wheel whose own driving shortcomings are unlikely to coincide with software error. This is believed from the OEM's own telemetry to have prevented several accidents with early generation Waymo (Edit: and apparently one was present in this vehicle and was unable to prevent the accident)
Of course, there's a PR benefit of taking the drivers out of the car at this stage of their development, and it appears this has taken priority...
I'd like to believe this technology will save 20000 lives per year. But today is a time for humility and reflection.
Because we have the driving records of all self-driving vehicles. Based on the evidence, their accidents per 100,000 miles driven is far superior to the average driver. The law of large numbers tells us this is highly unlikely to be the result of luck.
We do? Because one of Uber's apparent reasons for testing in Arizona rather than California (FWIW, Waymo has since moved from AZ to CA) is that there are no requirements to disclose driving/performance records, as there is in California [0].
The most we know at this point is that Uber has claimed to have driven 2+ million miles [1], which means it has a fatality rate of 50x the rate of human drivers (~1.18 per 100M miles). Yes, it's not fair to extrapolate like that, but this is just to point out that you need to cite evidence for "[self-driving vehicle] accidents per 100,000 miles driven is far superior to the average driver"
[0] https://www.dmv.ca.gov/portal/dmv/detail/vr/autonomous/auton...
[1] https://www.forbes.com/sites/bizcarson/2017/12/22/ubers-self...
If you think rushing to market, in some self-inflicted arms race for fully autonomous vehicles is morally acceptable to you, AND human lives are lost in the process, then I find that fascinating.
> US has 1 traffic fatality per 85 million miles driven. Uber's self-driving cars have driven ~3 million miles & killed one person
> While loss of human life is always a tragedy, individual fatalities are absolutely worth it if we can begin to reduce that count sooner than by proceeding with excessive caution.
Wait, when did I agree to be a utilitarian with you? Did I vote utilitarianism into power?
if excessive caution causes more death and more suffering, why would you be for that?
For that matter, how confident are you that you truly understand the secondary and tertiary consequences of your choice enough to make a utilitarian choice?
You can bring your hypotheticals to bear all you like, but this is a serious problem.
Generally speaking, it is easier to pinpoint X thing led to Y tragedy than to quantify the tragedies that would have happened without X. It is incredibly hard to show what did not happen but "should" have. Even if you can get it straight in your own mind, good luck proving it to the world.
People are living longer and child mortality is down. We don't really celebrate that. We just complain about overpopulation and how we are destroying the environment.
When Iraq lit oil wells on fire on its way out of Kuwait, they were expected to burn for months and be a global environmental disaster. When crack teams from around the world converged and put them out far faster than expected, it was not celebrated with the same degree of fervor that it had originally been decried as a disaster.
Y2K was also predicted to be a global catastrophe. It was quietly prevented and gets remembered as "Those fools who ever thought this was a big deal!"
If self driving cars could reduce mortality compared to current rates and we were confident of that, waiting until it is perfected before releasing it means accepting ongoing deaths that don't have to happen. It's not uncommon for people to feel that new tech needs to be perfected while missing the fact that it may be a big improvement over the current status quo without being perfected.
I don't really know anything about self driving cars. I have mixed feelings about replying to your comment because I don't actually know how I feel about this particular issue. But the thought process you are putting forth is both common and not very pragmatic.
I guess a pithy rebuttal would quote some saying about the perfect being the enemy of the good. But I think that leaves out a lot of important points.
At least doctors have the Hippocratic oath.
Sure: start by reading Spinoza's TIE.
Raises the question though: will stakeholders be willing to pay the upfront financial costs to obtain the level of quality and assurance that other engineering disciplines provide? We'll see.
I agree, but there is already a lot of software in a car that can kill people in case of an error. In particular the airbag have multiple sensors and a complex heuristic to decide when the have to open, which ones, full or partial strength, ... https://en.wikipedia.org/wiki/Airbag#Operation An airbag that doesn't open can kill someone. An airbag that open unnecessary can kill someone. From Wikipedia:
> From 1990 to 2000, the United States National Highway Traffic Safety Administration identified 175 fatalities caused by air bags. Most of these (104) have been children, while the rest were adults. About 3.3 million air bag deployments have occurred during that interval, and the agency estimates more than 6,377 lives saved and countless injuries prevented.
So, there is already a lot of programmers that have to be careful because an error can kill someone in the car (or the plane, or a medical machine, ...) I guess it's more easy to ignore the people killed by airbags because we are use to them and we imagine that they have a simple sensor (something like a light switch. So in case of a failure nobody is guilty.
With a self driving car, the first problem is that it is new and scary, and the second problem is that in an error that creates an accident we are use to blame the driver. But now we don't have anyone to blame (perhaps the programmer).
As long as it kills fewer people than would otherwise have died due to negligent human drivers I'd say it's already there.
>From viewing the videos, “it’s very clear it would have been difficult to avoid this collision in any kind of mode (autonomous or human-driven) based on how she came from the shadows right into the roadway,” Moir said. ( https://www.sfchronicle.com/business/article/Exclusive-Tempe... )
Not saying the software wasn't partly at fault but it doesn't look that clear cut. It sounds like better sensors could have helped.
If the hardware is not capable of capturing sufficient dynamic range to be able to see into the shadows while driving in daytime/nighttime in this location, one must ask why not? That seems obviously negligent.
If the car can see into the shadows (say either with lidar or sufficiently sensitive hardware) then a self driving car either just failed to identify and predict the object and subsequently killed a pedestrian, or it detected them and killed them anyway.
If it killed them anyway, it may be that it was physically impossible for them to avoid the detected object, or forensics (if properly performed) would show that the collision was avoidable.
All of these scenarios seem pretty inconsistent with the hypothesis of the police chief knowing what the duck he's taking about, and I question what the hell he's doing releasing such a statement before the evidence is in...
> Tempe Police Chief Sylvia Moir said that from viewing videos taken from the vehicle “it’s very clear it would have been difficult to avoid this collision in any kind of mode (autonomous or human-driven) based on how she came from the shadows right into the roadway."
This seems like an unfortunate accident, but it's not at all clear that the car was at fault or that it could have done anything to prevent it. There was even a human behind the wheel and they didn't react in time either.
As a driver, there are certain heuristics, e.g. did my traffic light just go red, or is the pedestrian signal/traffic light in the same direction about to go green.
The answer is to do what denser European cities do, which is to have pedestrian crossing signals have their own turn while all vehicle lights are red.
I'd be curious whether "walking outside of the crosswalk" turns out to be material in the analysis by NHTSA/NTSB. Were the crosswalks big enough and close enough that a pedestrian would have normally walked inside one? Was the interchange a regular 90-degree crossing of two roads, or was it something more irregularly shaped? What is the annual rate at which pedestrians get hit at this intersection by human drivers? Do self-driving cars need more examples of "pedestrians walking outside the crosswalk" in their training sets?
"The California Vehicle Code says you can actually cross any street as long as you aren't a hazard to vehicles. It is also legal to cross mid-block when you're not between two intersections with signals." [1]
The article points out that LA has a different rule. Uber's going to need to geofence locations and use different rulesets for different cities.
[1] https://www.scpr.org/news/2015/04/14/50992/how-do-you-cross-...
https://www.azcentral.com/story/news/local/glendale/2014/09/...
> According to Arizona law, pedestrians are supposed to cross within marked crosswalks, or at unmarked crosswalks at intersections. An unmarked crosswalk is the location where two roadways intersect but no marked crosswalk is present, for instance in a residential neighborhood. Jaywalking is technically crossing "between adjacent intersections at which traffic-control signals are in operation." In this instance, pedestrians shall not cross at any place except in a marked crosswalk.
But why would state-by-state (or city) geofencing be needed for this situation? Seems like pedestrian safety would be prioritized regardless of whether the pedestrian seems to be committing jaywalking. Is there a situation in which the AI shouldn't attempt to brake if a large object appears in the street in front of it? And how big of a factor would local laws play?
May be not. Regardless of the street crossing law in a specific city, it is a bad idea to kill pedestrians.
So we are all in a lose-lose. I'm anticipating severe backlash from Congress this week, if not today.
we don't need active testing on our streets. automakers would have dared to test ABS or "safety" systems in such a manner for the simple reason of liability.
Interesting way of putting it. Kinda echos the "privatize the profits, socialize the losses" sentiment from the financial crisis.
The ironic part here is that Uber already has a strong reputation for skirting the law and not caring. Their response and the following audits will be worth following.
This assumes that the reputation costs are comparable in weight to the acts that they commit. Based on the past year and what I hear about Uber's growth[0], I'd argue that their reputation and their valuation are quite uncorrelated, but it's hard to know the truth.
0: https://www.forbes.com/sites/miguelhelft/2017/08/23/despite-...
Rather this assumes reputation costs times risk of having those costs while undertaking some action is comparable to gain from this action.
And theoretically the only people who can regulate banks are the banks themselves.
Funny how you hear that argument every time a horrible industry is trying to keep profits high by eternalizing costs: meat packers in 1900, tobacco in 1950, oil and gas till today.
I wonder whether food companies (like the meat packing industry in Chicago) said the same thing before the FDA was created in 1906; i.e. that reputation will just regulate the industry to not sell mislabeled products, or spoiled or adulaterated food.
The only possible costs would be if they're made to pay a price in a court of law, which seems unlikely since the woman who was killed had a bike with her, and it's unknown if she was on the bike at the time, but she's been described as a pedestrian outside of a crosswalk. Clearly Arizona authorities just don't care about safety.
We don't really know if there will be any safety benefits at all, and while I believe we might get to that point, it's a bit farther away than some tech-giants would like us to believe.
If you listen closely to who say what, you'll notice that a lot of entrepreneurs claim that we will have full AI within a couple of years, while a lot of boring engineers huff and puff and are generally pessimistic.
I'm sure the there are great savings involved in self-driving transport - especially long-haul , so it will happen sooner than later, but this will initially mean more death - not less.
For better or worse, I don't think a self driving car will be commercially successful until it shows it is at least as safe as a human driver. More likely it will have to demonstrate that it is significantly safer. It might be the case that whichever company first launches doesn't correctly gauge the safety of their car, but I definitely think the goal is to be safer from day one.
Nobody is saying the car itself was at fault. People are saying (justifiably) that the people who put the car out on public roads with faulty engineering are at fault.
> I don't think a self driving car will be commercially successful until it shows it is at least as safe as a human driver. More likely it will have to demonstrate that it is significantly safer.
I agree. And the incident under discussion illustrates that the technology has not yet reached that point.
Furthermore, "commercially successful" is not the first objective that needs to be met. The first objective is "safe enough to be allowed on public roads". The incident under discussion illustrates that the technology has not yet reached that point either.
Automation will indeed do some harm to a certain group of people. The question is who should be responsible for bailing those people out (if at all). Is it society (aka, the govt), or the companies reaping the benefits of said automation?
The problem is that we should be 100% sure that there will be less death and not allow the self driving cars on public roads before.
I don't think we have the real numbers, how many Km each car drive and how many times the human had to get involved to prevent a big crash. If there are laws and checks in place so the actual numbers are reported then I want to see them, I read all self driving topics here and never seen this laws.
Until we have the real number the fact that self driving cars will save more lives is just a hope for a far away future.
https://www.cnbc.com/2017/05/23/alphabets-self-driving-waymo...
Self-driving car companies will have massive net positive externalities on the public, to the tune of hundreds of billions of dollars per year when fully deployed.
An electric rail network on the other hand would have real positive externalities, but isn't as easily privitazable so won't be built.
False comparison. First of all DARPA has invested a lot more than that in all sorts of related AV technologies (e.g. LIDAR, AI, robotics). But the real question is when were those "few million dollars" invested. Seed investments generally are much smaller than what companies end up being worth.
In our high tech system, taxpayers generally take on the riskiest stage of very early development, where it takes billions of dollars in bets spread over a very large area over 10+ years. And you're right, there are some socialized benefits, but the statement stands that the costs are socialized as well.
Imagine if you told Google's earliest investor that their investment was a tiny percentage of what it's worth today, and they enjoy the benefit of using Google now, so it's fair that Google's later investors kept all the equity.
I was responding to your link about the Grand Challenge.
> Seed investments generally are much smaller than what companies end up being worth.
Of course. Are you really going to make someone else go point out the hundreds of millions in dollars of seed-level funding by Google, by Uber, etc., to demonstrate the obvious point that the DARPA Grand Challenge is terrible evidence for the claim "autonomous vehicle development was funded by taxpayers".
> so it's fair that Google's later investors kept all the equity.
People who invest in Google get to keep the equity they they purchase, but they don't get a claim to the equity of companies that exist because of Google's search product. Likewise, when the government invests in public goods, they don't then get to claim everything that is built on top of the public good.
Indeed, the government provides the rule of law, without which almost no modern economic development could take place. But that obviously doesn't mean the government has claim to all economic value. Likewise, none of us could work without eating, but that doesn't mean we owe all our income to farmers.
If they used the right liscensing for the information that they created then they might.
Again you make a false comparison. Nobody claimed that early stage investors are entitled to "all economic value". The statement stands that costs are socialized. Silicon Valley is greatly subsidized by early stage taxpayer investment.
> Are you really going to make someone else go point out the hundreds of millions in dollars of seed-level funding by Google, by Uber, etc., to demonstrate the obvious point that the DARPA Grand Challenge is terrible evidence for the claim "autonomous vehicle development was funded by taxpayers".
"Autonomous vehicle development was funded by taxpayers" is a factually correct statement. You again fundamentally misunderstand the distinction of when investments are made vs. quantity of investment. Earlier stage investments are riskier; Google et al did not start pouring money in until the technology started showing some promise after many years of taxpayer investment. As is commonly the case with our high tech system.
I guess we should credit Andrew Carnegie with pre^12-seed funding since he founded Carnegie-Mellon and a lot of relevant robotics research has taken place there.
Again, I am not discussing all of DARPA's activities, I am talking about the Grand Challenge you brought up, which I continue to maintain is minuscule compared to a million other sources and thus is terrible evidence that "autonomous vehicle development was funded by taxpayers" in any non-trivial sense.
> Nobody claimed that early stage investors are entitled to "all economic value".
You misinterpret my analogy. The point is that your approach, if taken seriously, would mean the government would have a claim on every single bit of economic value in the US, not that it would have full ownership of all of it.
> The statement stands that costs are socialized. > "Autonomous vehicle development was funded by taxpayers" is a factually correct statement
Ha, yes, and we can also conclude that autonomous vehicle development was funded by Carnegie, Roomba, and my buddy Alex who runs a robotic delivery startup.
The simple fact remains that taxpayers have made significant and critical investments in nurturing AV technology, like many other technologies that Silicon Valley has commercialized once they bore fruit. Your fundamental premise that investments that are "minuscule" in size are necessarily minuscule in significance is trivially wrong.
A good example is pharmaceutical research. NIH budget is ~$30B and arguably a fraction of that is pure drug development.
The top pharma companies (ignoring VC investment in start ups) is over $70B.
There are of course situations were externalities fail to be privatized (e.g., traffic congestion), but I don't think auto accidents is one of them.
http://www.nytimes.com/1997/02/11/us/jury-decides-simpson-mu...
To state the obvious, I am not a lawyer.
The theoretical difference with criminal charges is that criminal charges deal with harms to the difference sovereign (ie the state and the public) that need to be punished regardless of the wishes of the immediate victim. e.g. you're not allowed to settle a murder charge out of court.
For instance, there are accidents that cause deaths all the time; but they would only be prosecuted as manslaughter if someone were doing something considered inherently unsafe and unlawful.
If you run a red light, and kill someone in the process, it's possible you could be prosecuted for manslaughter, though even then you might not be if there could have been extenuating circumstances (sun in your eyes, etc).
In this case, there would only be a chance of a manslaughter charge if Uber were somehow being particularly negligent. In this case, it sounds like there was a human operator in the car, though the car was in autonomous mode. It's possible that the operator wasn't paying the attention that he or she should have been, or it's possible that even the human operator didn't see the pedestrian in time. It mentions that the pedestrian was crossing outside of any crosswalk, so it's possible that this was a tragic accident of them trying to cross a street with traffic without having been sufficiently careful.
On the other hand, it mentions a bicycle, but also says the victim was walking, which I find odd:
"Elaine Herzberg, 49, was walking outside the crosswalk on a four-lane
road in the Phoenix suburb of Tempe about 10 p.m. MST Sunday (0400 GMT
Monday) when she was struck by the Uber vehicle, police said."
Local television footage of the scene showed a crumpled bike and a Volvo
XC90 SUV with a smashed-in front. It was unknown whether Herzberg was on
foot or on a bike.
Unless the operator was being particularly negligent, or there was some serious and known problem with the self driving car but it was put on the road anyhow, I doubt any manslaughter charges will be filed.
Remember, Arizona has explicitly been encouraging the testing of self-driving cars, so I expect that testing a self driving car, with an operator to take over in cases which it can't handle, would not be considered unlawful or extremely negligent. Maybe what the operator was doing could be, but we'd need more information before it would be possible to tell.
The police clarified that the victim was walking her bike across the street: https://twitter.com/AngieKoehle/status/975824484409077760
Criminal law applies to criminal acts. It's possible for do something wrong that "injures" someone (physically or in some other way) without committing a crime.
Criminality generally requires wrongful intent. If you simply screwed up while otherwise obeying the law then you haven't committed a crime (negligence can be a crime, e.g. if you are more careless than a "reasonable man" would be this is in itself a form of bad intent -- so swinging swords around in public places while blind-folded isn't OK).
Also, tort law has a different standard of proof -- it is resolved based on preponderance of evidence rather than proof beyond reasonable doubt, so there's a lower bar than for throwing someone in jail.
Oh, and it's harder to prove criminal charges against nebulous entities. (Who's the criminal in this case? Uber's CEO? The head of the software team? The person who gave uber a permit to test their crap on public streets?)
Uber's overall conduct might approach the point of "criminal enterprise" at which point RICO statutes might be invoked. Not likely though.
While the woman's death is tragic, I hope this doesn't set us back on whatever progress made in autonomous driving.
EDIT: removed "While this a serious f*up by Uber and"
Did Uber have authorization to operated automated in AZ?
2) Testing or operation of self-driving vehicles equipped with an automated driving system on public roads with, or without, a person present in the vehicle are required to follow all federal laws, Arizona State Statutes, Title 28 of the Arizona Revised Statutes, all regulations and policies set forth by the Arizona Department of Transportation, and this Order.
3) Testing or operation of vehicles on public roads that do not have a person present in the vehicle shall be allowed only if such vehicles are fully autonomous, provided that a person prior to commencing testing or operation of fully autonomous vehicles, has submitted a written statement to the Arizona Department of Transportation, or if already begun, has submitted a statement to the Arizona Department of Transportation within 60 days of the issuance of this Order acknowledging that:
a. Unless an exemption or waiver has been granted by the National Highway Traffic Safety Administration, the fully autonomous vehicle is equipped with an automated driving system that is in compliance with all applicable federal law and federal motor vehicle safety standards and bears the required certification label(s) including reference to any exemption granted under applicable federal law;
b. If a failure of the automated driving system occurs that renders that system unable to perform the entire dynamic driving task relevant to its intended operational design domain, the fully autonomous vehicle will achieve a minimal risk condition;
c. The fully autonomous vehicle is capable of complying with all applicable traffic and motor vehicle safety laws and regulations of the State of Arizona, and the person testing or operating the fully autonomous vehicle may be issued a traffic citation or other applicable penalty in the event the vehicle fails to comply with traffic and/or motor vehicle laws; and
d. The fully autonomous vehicle meets all applicable certificate, title registration, licensing and insurance requirements.
-----
[0] "Executive Order: 2018-04 - Advancing Autonomous Vehicle Testing And Operating; Prioritizing Public Safety" - https://azgovernor.gov/file/12514/download
How do you know that? From my initial reading of the police report, the woman was jaywalking and ran out into the middle of the street. I'm not even sure a human or autonomous driver would have been able to stop in time. The only tech that might have been able to is Waymo's since, based on the video they released, they have a radar lock on every pedestrian within a mile of the vehicle and they do predictive tracking to determine their position. Even then, it might have still not stopped in time.
Unless it is a pure capitalist move.
Is every programmer in the world partly responsible for every death caused by every programming bug ever? Should we blame all civic engineers every time a bridge collapses?
I am an individual, not "part of a system". Simply being a driver doesn't make me a reckless or dangerous one. For example, 10K of those deaths involve DUI, and I am certainly not part of that system. I'm sure 95% of drivers never kill anyone.
(back of the envelop, assuming 300,000,000 drivers driving for 60 years of their lives at a 40,000 per year death rate)
Massive misunderstanding. As a driver your actions form a part of the traffic around you, even responsible drivers are "attached" to the system and put pressure on it. Maybe one day somebody rear ends you and you are knocked forward into a crosswalk, or you break suddenly to avoid hitting a pedestrian and cause an accident behind you. Or maybe your very safe driving leaves enough room for somebody to take a foolish risk cutting in front of you and lose control of their car. Collectively society accepts that getting places quickly is worth the cost that we pay in human lives. And each road user agrees to the small risk of death/injury beyond their control in order to get somewhere.
Moreover, it is illegal to drive a car safely. If you try it, you will be pulled over. You may be fined or lose your licence. Try it and see. So yes, being a car driver does make you a dangerous one.
I wonder how the issue developed when the automobile was first introduced. I remember something about needing to have a guy walking in front of the vehicle waving a warning flag, but apparently that didn't last long.
Found it - https://en.wikipedia.org/wiki/Red_flag_traffic_laws
Or, alternatively, we do indeed have a cap to the amount of resources we are willing to expend to save a life - lives are not of infinite value.
That's how it is profoundly different.
Every regulatory system in the world has to consider these things, explicitly or (more commonly) implicitly. See e.g. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1633324/ for the application of the idea to healthcare, or http://www.nytimes.com/2011/02/17/business/economy/17regulat... for the public policy implications for environmental regulation.
Or more relevant yet, this "guidance on valuing reduction of fatalities and injuries by regulations or investments" by the Department of Transportation: https://www.transportation.gov/sites/dot.gov/files/docs/VSL_...
Unless Python.
Of course, the US department of transportation should have set up proper certification for all of this. They could have easily done so because they can arbitrarily choose the certification costs.
https://en.wikipedia.org/wiki/Motor_vehicle_fatality_rate_in...
Not that 37,000+ is a great number, but I don't think many of the detractors here are arguing that Uber et. al have a perfect record. Just that it's possible that progress is being made in a more reckless way than necessary. Just because space flight is inherently difficult and risky and ambitious doesn't mean we don't investigate the possibly preventable factors behind the Challenger disaster.
edit: You seem to be referencing the worldwide estimate. Fair, but we're not even close to having self-driven cars in the most afflicted countries. Nevermind AI, we're not even close to having clean potable water worldwide, and diarrhea-related deaths outnumber road accident deaths according to WHO: http://www.who.int/mediacentre/factsheets/fs310/en/
That's not a particularly convincing argument, given that (so far), Uber's self-driving cars have a fatality rate of 50 times the baseline, per mile driven[0].
Having to wait an extra ten years to make sure that everything is done properly doesn't sound like the worst price to pay.
[0] Nationwide, we have 1.25 deaths per 100 million miles driven. Uber's only driven about 2 million miles so far: https://www.forbes.com/sites/bizcarson/2017/12/22/ubers-self...
https://www.androidheadlines.com/2017/12/ubers-autonomous-ve...
Presumably a bit more since December.
https://en.wikipedia.org/wiki/Motor_vehicle_fatality_rate_in...
Fluctuates just over 1 per 100 million.
(1/2+ million) / (1+/100 million) ~ 50-
Let's say that halving the death rate is what we can reasonably expect from the first generation of self driving cars. Every year we delay that is 15,000 people dead. This woman dying is a personal tragedy for her and those that knew her. However, as society we should be willing to accept thousands of deaths like hers if it gets us closer to safer self driving cars.
What's your evidence for why this is a reasonable expectation? The fatalities compared to the amount of miles driven by autonomous vehicles so far shows that this is not possible at the moment. What evidence is there that this will radically improve soon?
No, you couldn't have characterized Uber has having "infinitely better" fatality rate than the baseline, because that would have resulted in a division-by-zero to calculate the standard error. Assuming a frequentist interpretation of probability, of course; the Bayesian form is more complicated but arrives at the same end result.
It's true that the variance is higher when the sample size is lower, but that doesn't change the underlying fact that Uber's fatality rate per mile driven is empirically staggeringly higher than the status quo. Assigning zero weight to our priors, that's the story the data tells.
I would be very interested in a 3rd party (government or private) create a rigorous test (obstacles, weather conditions, etc...) for self driving vehicles. Becoming "XXX safety Certified with a Y Score" for all major updates to the AI could help restore confidence in the system and eliminate bad actors.
Identify the objects in pictures.
We take our biological vision systems for granted, but it seems one autopilot system couldn't identify a semi crossing in front of the vehicle...
The driving schools also have theoretical tests where one has to identify objects in a picture, interpret the situation, and propose the correct action. Of course, these tests are on a higher level: "this is a car, this is a pedestrian" vs. "you're approaching an intersection, that car is turning left, a pedestrian is about to cross the street, another car is coming from there etc."
Not to mention the road and track tests a driver has to pass which include practicing controlling the car in difficult conditions: driving in the dark, evasive actions on slippery surfaces and so on.
Edit: In my opinion it's insane to allow autonomous vechiles on the roads without proper testing by a neutral third party.
Then, you can start to identify situations where the driver's actions were outside of a predicted acceptable range, and investigate what happened.
Additionally, if you have a large pool of equipped vehicles you can identify every crash (or even more minor events, like hitting potholes or road debris) and see what the self-driving system would have done.
The realistic problem is that Uber doesn't give a shit. As such, deployment will never be optimized for public safety. It will be optimized for Uber's speed to market.
I think you're severely underestimating the path that something like this would have to take. The certification itself would be under so much scrutiny and oversight that it would take years for that to get done. Unfortunately, the technology is far more readily available and easy to get working than the political capital required to create a certification for this.
So much talk about LIDAR and other sensors. Why nobody talks about obvious idea of Road Object Message Bus? ROMB is a protocol where each road object (a traffic light, a sign, a car, a bicycle, etc.) transmits info about itself. A car could broadcast its direction vector, intention to turn, any non ROMB moving object it sees. A traffic light could broadcast current state and when it is going to change. That information would greatly enhance overall security, especially during rain and snow conditions, when even LIDAR fails.
Self-driving is such important (just after eliminating combustion engines) that we could upgrade existing cars with cheap ROMB boxes. Vehicle GPS tracking system costs about $30. ROMB box would cost about $60. Let's say that from 2027 all cars have to have a ROMB box to enter a downtown ...
Let's say your car ROMB received info about the white truck, while your car cameras and vision recognition systems see just a cloud and any truck in 100 m range.
ROMB purpose is not to replace cameras or LIDARs, but to extend gathered info.
And I think that the comment is very relevant to the current discussion. It talks about how to increase the safety of self-driving cars.
Please answer why it is not relevant in your opinion. Because from my point of view it looks like you don't read it before downvoting.
BTW HN is not complaint with GDPR. After some time I cannot delete my comments.
https://youtu.be/GzrHNI6eCHo?t=100
Using https://github.com/commaai/openpilot , which is cool but not on public roads.
Given that all the main car companies are keeping their technology private I don't see how open-source systems are supposed to keep up without people doing this.
I think you can also contrast this to the many more people who decide to drink alcohol and then drive.
Sure, but drinking (more than a little) and driving is already illegal, so they're not exactly comparable.
Because at some point, it's literally not possible to test something that's ultimately intended for use on public roads anywhere but public roads. Or, to put it in terms that would be more familiar to developers, "you can test all you want in QA, but prod is the ultimate test of whether your code really works".
As mentioned downthread, the problem here is that Arizona basically gave companies full leeway, without having to demonstrate that they had done sufficient tests internally before putting the cards on public roads. Apparently they're not even required to report data on accidents to the state, which is absurd.
I wouldn't be surprised if Uber were ultimately found to be cutting corners, but Arizona is also responsible for rushing to allow self-driving cars on their roads without doing their own due diligence first.
I'm stunned that when I drive in Texas, there don't seem to be regulations over which vehicles can have flashing lights and with which colors. Yet in these other cities you have experimental robots that are designed to blend in with other cars as much as possible. NO! Make them stick out.
My thoughts exactly. It is wrong. We have a chance to do this properly, openly and together, but instead our governments are letting private companies put our lives in danger.
The largest real difference seems to be the speed of communication nowadays, the rate at which public opinion can be manufactured and globally spread.
In this case have a machine replacing a human and using non deterministic algorithms to do so.
Automated driving ONLY makes sense on specific, well mapped, sensor enabled, pedestrian free roads. Let’s focus on that use case before we let these things take over our towns and cities.
Because the for profit company has a say in government and lawmaking, the person getting run over doesn't.
Some phase of testing will necessarily be of public roads. The fact that they started that phase prematurely doesn't somehow mean that "private for profit companies" should never test on public roads.
Boeing tests new airplanes in the public airspace too, over public areas.
Uber abides by no such testing standardization.
Implement a way for the highway operator to affect the cars on it: make way for emergency vehicles, adapt to infos the operator can aggregate and transmit, change the speed of cars to the slowest one around etc.
Once the majority of people have self driven cars and those have been proven in this controled environment you can think about allowing them on open roads. Or not. Or adapt those open roads to what self driven cars need.
sure, makes sense to me.
But God, that would cost a fortune.
It would. Like an infrastructural job. I mean, it would be like railways for individual trains.
But that's kinda-sorta the job expected from government and what taxes are for: paying for things the individual would have problems doing.
I mean, this is the way cars have always operated. It is somehow any different to let them "test" there brake system, there acceleration systems? Car companies have always been able to do whatever they want, who cares about the deaths.
For some reason, when it comes to roads, we all just accept a certain amount of deaths each year. Over a million people are killed every year around the world, up to 40,000 in the US alone. That is acceptable according to every car driver.
Self-driving cars need this all the more, since most of the hard and scary stuff is the integration of many systems and how they deal with unexpected scenarios. And worse still -- in a traditional car, you had an expert test-driver like my friend who could, say, take evasive action if the brakes didn't work.
But in these cars, the driver is the system-under test.
Yes, and so do these guys. They might be somewhat risk averse, but lets not pretend they have in the past made sacrifices that have cost lives (Pinto, etc...) and there are constant recalls going on.
All of them, I mean you took a poll? that must have taken a while.
My understanding (as a non-US resident) is that parts of the US are built around the assumption you drive.
If someone wants to pretend it is not acceptable while still driving, that is just silly.
Yes, this is 100% true. Arizona included.
It's difficult, usually impossible, to buy-out of driving (or being driven) in most places in the USA. Ask DART why rapid transit ridership in the Dallas area is falling despite a growing population and a billion-dollar annual budget.
https://www.washingtonpost.com/news/dr-gridlock/wp/2018/03/1...
> Police said the vehicle was northbound on Curry Road when a woman, identified as 49-year-old Elaine Herzberg, crossing from the west side of street, was struck. She died at a hospital, the department said.
No mention of her being a bicyclist. My gut instinct is to accept that the victim was indeed a pedestrian -- with the assumption that they'd have that detail cleared up by the time they have her identity ready to release to the public.
So, she approached from the left. That would make it significantly less likely that she just suddenly appeared in front of the car, right? It's not a narrow street.
It's possible she was walking her bicycle across the street, but that still looks pretty bad for Uber. That street is a straight shot with clear visibility.
(Still need more info, though.)
https://www.google.com/maps/place/N+Mill+Ave+%26+E+Curry+Rd,...
[0] The caption for this photo says the crack in the road represents "burned out flares at the location where a woman pedestrian was struck and killed". Not sure if that is exactly where she was hit, or if that's where her body landed, or if the flares were just near the location.
http://s3.reutersmedia.net/resources/r/?m=02&d=20180319&t=2&...
The article says "as soon as she walked into the lane of traffic she was struck".
The police sergeant also "said he believed Herzberg may have been homeless".
These three things may together indicate that even a human driver might have struck and killed the lady if she entered into the roadway unsafely. I'm not sure how everyone (especially on Twitter) can be jumping all over Uber yet until the investigation is complete.
Sad of course, either way, but not yet enough evidence to assume that the algorithm+driver combo was any less safe than a human driver alone.
And a mistake in a broader sense - was it an avoidable accident? If it wasn't this barely feels like news, other than as a milestone.
Also... I don't see how her being homeless has anything to do with it.
well, higher than average rates of substance abuse in the homeless population. other than that I'm not sure how it could be relevant or why the article mentioned it. we wouldn't know without an autopsy but being drunk could explain walking out into traffic.
It's unfortunate that this happened. But we do need take into context the situation and all factors involved.
I can think of a number of ways it may be relevant. In addition to what I said in this discussion, I left a comment about that here:
I know more than average about homelessness. A high percentage of homeless people have serious health problems. If she was homeless, she may have already been in very poor health. That may have contributed to her death.
I can think of potentially other contributing factors, but I am leery of coming across like I am badmouthing homeless people. They are subjected to enough hostility and prejudice and I am usually trying to advocate on their behalf.
[1] https://www.theguardian.com/technology/2016/jul/01/tesla-dri...
This case is in fact the first autonomous test vehicle caused fatality.
There were at least several Tesla autopilot related fatalities and injuries, but I would seriously not put those into self driving bag.
"Tesla driver killed while using autopilot"
"The Tesla driver killed in the first known fatal crash involving a self-driving car"
"It's been nearly a year and a half since Joshua Brown became the first person to die in a car driving itself."
"Tesla bears some blame for self-driving crash death, feds say"
This is in the site guidelines: https://news.ycombinator.com/newsguidelines.html.
Seems like an attempt to shift the blame off the autopilot system and onto the driver.
> A Tesla spokesperson said the car in the Chinese wreck was badly damaged and unable to transmit data. “We therefore have no way of knowing whether or not Autopilot was engaged at the time of the crash,” the spokesperson said in a statement.
I've heard the general report, and I find it interesting that a supervisor was at the wheel, and they, as well as the car, failed to prevent the pedestrian from getting hit. I read they were not in a crosswalk and I'm sure the specific details of the accident matter greatly. It makes me wonder if a human driver with no autonomous system would have faired any better since someone paid to be alert and a car designed to be alert both failed to prevent the accident.
I'll reserve final judgment until more details are out.
would you drive 40 mph down a city street with trucks and buses parked head to tail on both sides of the road?
I expect we'll soon see tech companies do the same in order to favour their autonomous automobile businesses at the expense of pedestrians and cyclists.
If you want to get up on the history you can grab a copy of "Fighting Traffic," by Peter Norton. He goes through all the efforts cities made in the 1920's to limit cars, and how the motor lobby formed in opposition to this and little by little reversed the tables to the situation we find ourselves in today.
Indeed it has been known for a long time. This is an opportunity to reverse bad decisions made a century ago, regardless of how quickly self-driving cars actually are adopted.
What does the statistics say? How many miles have the self-driving cars driven, and how many deaths were they responsible for? How does it compare to a human driver?
When a million humans drive a car for a mile, and 1 of them results in a death, it's easy to pinpoint the blame on a "random drunk/distracted driver". How'bout start thinking of the self-driving software as a combination of all human drivers, with just a much smaller odds of having a drunk/sun-blinded/distracted driver than an average human driver.
As of December 2017 Uber had driven 2 million autonomous miles[1]. Let's be generous and double that, so 4 million.
The NHTSA reports a fatality rate (including pedestrians, cyclists, and drivers) of 1.25 deaths per 100 million miles[2], twenty five times the distance Uber has driven.
You probably shouldn't extrapolate or infer anything between those two statistics, they're pretty meaningless because we don't have nearly enough data on self driving cars. But since you asked the question, that's the benchmark: 1.25 deaths per 100 million miles.
[1]: https://www.forbes.com/sites/bizcarson/2017/12/22/ubers-self... [2]: https://en.wikipedia.org/wiki/Transportation_safety_in_the_U...
- Uber autonomous: 33.3 deaths per 100 million miles
- Waymo: 0 deaths per 100 million miles
- National average: 1.25 deaths per 100 million miles
Of course, the Uber and Waymo numbers are from a small sample size.
But there's also the bayesian prior that Uber has been grossly negligent and reckless in other aspects of their business, in addition to reports that their self-driving cars have had tons of other blatant issues like running red lights.
It seems reasonably possible that an Uber self-driving car is about as safe as a drunk driver. DUIs send people to jail - what's the punishment for Uber?
Comically, that’s why OP said not to do that.
Comparing dissimilar things is actually worse than not comparing at all since it will increase the likelihood of some decision resulting from the false comparison.
Is the data sufficient to say if Uber might eventually arrive at a usable self driving vehicle. Plainly no. It's not sufficient to answer this question one way or another.
Is the data sufficient to indicate if Uber is responsible enough to operate an automated test vehicle program on public roads. Maybe.
There still needs to be an investigation of cause, but if the cause is in a autopilot failure, or the testing protocols preventing a failing autopilot from harming the public, then the question is what the remedy should be.
I agree that you have to use data available to make the best decision possible.
There may be methods to account with all of the problems of comparing two different measures, but it requires a lot of explanation.
But extrapolating one measure into another is wrong without those caveats. That’s the comment I replied to. So in no situation would the method I replied to be useful for what reasonable question is asked.
The purpose of the extrapolation was to get a (flawed) approximation to that answer. By itself, it doesn't say much, but all we can do is parse the data points available to us:
- Uber's death rate after approximately 3 million self-driven miles is significantly higher than the national average, and probably comparable to drunk drivers.
- Public reporting around the Uber's self-driving program suggests a myriad of egregious issues - such as running red lights.
- The company has not obeyed self-driving regulations in the past, in part because they were unwilling to report "disengagements" to the public record.
- The company has a history of an outlier level of negligence and recklessness in other areas - for example, sexual harassment.
Providing this likely wrong number anchors a value in people’s minds.
It’s actually worse than saying “we don’t know the rate compared to human drivers because there’s not enough miles driven.”
Your other points are valid but don’t excuse poor data methods hygiene.
Even now you are making baseless data on its face because you don’t know the human fatality rate per 3M enough to say is “significantly higher.” Although I think it’s easier to find enough data from the human driver data to match similar samples to Uber. But dividing by 33 is not sufficient to support your statement.
I haven’t seen data on the public reporting. That seems interesting and would appreciate it if you can link to it.
https://www.nytimes.com/2017/02/24/technology/anthony-levand...
Let's assume that the chance of killing in any two intervals of of the same number of miles traveled is the same. Let's say that the threshold for self driving cars being "good enough" is the same death rate as human drivers.
If we assume Uber is good enough, then they should kill people at a rate of at most 1.25/100,000,000. The waiting time until they first kill someone should fit an exponential distribution. The probability that a death would occur in the first t miles is 1 - e^(-lambda t) where lambda is the rate of killing people, is 1.25/100,000,000. I.e. 1 - e^( -(1.25 / 100,000,000) x 4), which is 5 x 10^-8.
If Uber has only a 5 x 10^-8 probability of driving "safely enough" they should lose their license at the very least.
Edit: Oops, 4 != 4,000,000. It should be 1 - e^( -(1.25 / 100,000,000) x 4,000,000) which is about 0.049...
Still, I think we can ask for better than a 5% chance of being the same as a human driver.
(also replaced stars with 'x' because HN was making things italic)
Edit: Actually, maybe I'm wrong in assuming (a) the majority of miles driven are interstate miles, or (b) that the majority of existing miles logged by self-driving cars have not been on the interstate. Would love to see some data if anyone has it, although I suspect Google, Uber, et al. are reluctant to share data at this point.
How often did the human have to take control?
Therefore, I think it would be more fair to consider all miles run by all autonomous vehicles all over the world in the denominator.
It is for the same reason that we want to consider all miles driven everywhere, not just those in Arizona.
Additionally, our self-driving car statistics we tend to see today also tries to ignore the presence of test drivers and how frequently they intervene. As long as they can intervene, the safety record of self-driving cars is being inflated by the fact that there's a second decisionmaker in the vehicle.
EDIT: Oh, and human driving statistics are also bad because a lot of accidents don't even get reported, and then when they do get reported, a lot of it is through different insurance companies, that's before we get into nobody centrally tracking "miles driven", and that's why most statistics for human driving safety are more or less just an educated guess.
Just a guess, but I doubt having a test driver that can intervene will help safety statistics for autonomous vehicles much. I think we’ll find test drivers will usually be unable to notice and react fast enough when they are needed.
But yeah, I'd agree there's a lot of difficulties expecting a test driver to immediately take over in a sudden event like a bike entering the roadway.
Also, if someone had posted a poll a day ago as to which company's self driving cars were likely to be the first to kill somebody, I think the vast majority of people would have predicted Uber. I don't think that's a coincidence.
Note that they already know their cars will need help driving still, which is why they've assembled a call center for remote drivers to take over their cars in Arizona. Of course, those remote drivers likely can't intervene at nearly the speed of an onboard safety driver.
Actually, the time to "overeact" was even earlier, but most of self-driving cars ignored any criticism. So now you have stupid car companies cutting corners and killing people so they can be "first" on the market or whatever.
You experience it sometimes where you just happen to glance in the right direction, as a driver, and avoid horrible things from happening. Or hesitate to go through an intersection when a light turns green and some person is running a red. You and your body somehow knew but it can't be explained.
Computers don't have that, whatever it is.
I was driving a long roadtrip from Texas to California, split into a couple segments over several days. At one point, my adrenaline and heart rate suddenly spiked. I felt freaked out but could not see a reason for it. I checked all my mirrors, traffic was busy but seemed to be moving along normally. A few moments later my vehicle was rocked by a semi-truck blowing past, traveling much faster than surrounding traffic, and missing my vehicle by what seemed like an inch.
The roadway was curved slightly, so I think the semi was in a blind spot when I was actively searching for the problem.
It's interesting that a subconscious process could alert me to a problem, in this case it didn't help me resolve it, but at least I was alert and looking. It had never happened before so there was a bit of confusion as well (why am I suddenly freaking out?) - but now I know how to pay attention if that feeling happens again.
Sure maybe it happens and there is nothing really going on but my point is there is something about our sub-conscious that cannot be implemented with computers.
Please petulantly downvote all minority opinions!
I cycle along paths like this from time to time myself, and I always assume that I should let a turning car pass in front of me. As soon as I see the turn signal I will either fall back or even pass into the car lane to overtake on the other side.
Of course that's in part because I know I would lose the fight anyway, but also because I think I am not actually supposed to be "overtaking" them on the right side, so they have the right of way before me.
What do the rules say in this case?
[0] This is based on this link posted elsewhere: https://www.google.com/maps/@33.4370667,-111.9430321,3a,75y,...
This is the insanity of (most) bike lanes, that you have a lane to the right of a right-turn lane that can go straight. And that's why I don't use most bike lanes. They're trouble.
Perhaps there's a better metric to look at, but I'd like to see number of deaths caused per miles driven.
If Uber's self-driving cars are killing more pedestrians than a human driver would, we have a huge problem, but I'd be willing to bet they're at least an order of magnitude safer in this respect.
And to say that "it's not their fault" and "it could have happened to anyone" will be cold comfort to those who just watched their car mow down an innocent, and there was nothing they could do to stop it.
Sympathy is not a zero sum game.
There's typically 1 death per 100 million miles driven.
Uber's only driven a couple million miles.
People will always make mistakes, the benefit with self driving cars however is that mistake should only be made once. Whatever bug caused it to be considered at fault can be patched and will never occur in that exact circumstance again. Meanwhile with humans you could have that same case occur over and over again.
With how much money people can win in lawsuits nowadays, all it would take is a handful of cases to totally destroy a manufacturer of self driving cars.
Did you even read the first 5 words?
>It'll likely never get passed
Can you stop trolling my posts? You aren't adding much of anything.
I would guess it's inevitable that in the medium/long-term, there will only be one or two companies developing and selling self-driving tech to every car manufacturer. Small players won't have enough data, everyone will be too afraid of the risk, the amount of government regulations surrounding self-driving cars will increase. More regulations will protect people but it will also mean it's much harder to start gathering the data you need and testing your product.
> The NHTSA reports a fatality rate of 1.25 deaths per 100 million miles[2], twenty five times the [4 million miles] Uber has driven.
So they should have driven around 75 million more miles before getting their first fatality, in order to remain even with humans. Not to mention they've been driving on the clearest/sunniest roads in only a few cities.
Of course a sample size of one is not enough data, but I'd say we should err on the side of "we have a huge problem".
There should have been so many safety precautions in place that nobody should have died from this yet.
I think the rate of overall vehicle crash deaths in the U.S. is less than 2 per 100 Million vehicle miles. Pedestrian traffic is a fraction of that.
AI drivers have killed 1 human having driven a mere ~5 million miles (not even counting Tesla autopilot). Disengagements occur at a rate of 1 per 5k miles.
The evidence strongly, strongly points to self-driving cars being more dangerous than humans today.
If there was any whiff of shortcutting, poeple need to go to jail and Uber should be bankrupted by penalties and lawsuits.
And then I learned Facebook is going into self-driving cars.
Sigh.
Recently I've seen a lot of comments defending autonomous vehicles from a statistical standpoint, but even if these cars have the potential to be safer on average, it scares me to think that the software driving a car is less regulated than its airbags or seatbelts. Especially considering the auto-industry's tenancy to ignore safety issues without external pressure [0].
Is my information outdated? Have there been efforts to review or regulate autonomous driving software in the last year?
Not hitting pedestrians should be close to the top in priority, so while failures of the software are certainly expected during this kind of test, this is one of the worst kinds of failures that should have received a lot of attention before ever putting the vehicles on a public road.
The safety driver failing makes me wonder how well qualified they are, or if there is an issue with staying alert for long times without actually driving the car.
From studying many bicycle-related police reports, I was dismayed at how often I read the phrase "a pedestrian riding a bicycle."
Sad event, but it's likely a human driver would have killed her as well. She was on the road at night without lights, jaywalking without attention to the on-coming traffic.
One could argue the car was driving faster than it's headlight distance safely allowed. If so, humans are going to be quite frustrated with how slow autonomous cars drive at night.
Personally, as a biker, I get slightly annoyed by pedestrians who seem to think I have little control or awareness. I ride on the road where practical, and, if on sidewalk, I go much more slowly and sometimes just get off and walk if the area calls for it. Generally, bikers are much more aware of their surroundings than drivers (both out of self-preservation and an unobstructed, elevated view).
Of course, the best solution is fully separated paths for walkers, bikers, and cars. And, in this instance, it looks like there is ample room for building that, with no street parking to complicate things (and the sidewalk also looks wide and sparsely used).
Unfortunately, this will inevitably set the field back by at least 10 years with stricter regulations. We've seen the same with pharma and aviation, where innovation was slowed drastically and is only realistic for big companies. This is especially true here because if self-driving vehicles get banned from public roads, companies like Waymo will be forced to develop bigger private "test tracks" that approximate cities to have any hope of building deployable technology, and they will face the same fundamental limitation – the data will come from an approximation/model and not the actual location with real-life scenarios.
Although there's no doubt that regulations are crucial to prevent such mishaps, I do hope lawmakers can find the right equilibrium on the tradeoff between safety and innovation.
It is important to know whether our machine is now doing worse/better than our counter-part and this means a lot for our self-driving initiatives.
I've heard Uber is around 3 million self-driving miles.
So Uber would be 30x worse than humans.
Note that this is far too little data from which to draw any conclusions, since the sample size for fatalities involving self driving cars is one and the number of miles driven is many orders of magnitude less.
[1] http://www.iihs.org/iihs/topics/t/general-statistics/fatalit...
[2] https://www.theverge.com/2017/11/28/16709104/waymo-self-driv...
It's unfortunate, but unsurprising, the first pedestrian killed by an experimental self-driving car allowed on public roads was in Arizona.
We need to discuss how the developers self-driving cars will be held accountable for the crimes they commit. There is no reason the person who programs or the person who makes money from a self-driving car should be held less accountable for a crime that if committed directly by a person would almost certainly result in jail time. You can’t replace yourself with a computer program and then choose to take only the benefits and not the responsibilities.
The executives should be held accountable, not the developers.
The person who profits the most should be held the most responsible. But the separation of roles between the executives and the developers is likely to mean that no one gets punished at all.
Do you want software development to turn into healthcare, where every developer needs millions of dollars of malpractice insurance? Because shit like this will turn it into a healthcare like system real quick.
It's up to the product (the collective of individuals that deliver the product) to address and mitigate the risk it creates, that's not solely on the shoulders of individual software contributors.
If A writes a generic computer vision algorithm and open sources it, B integrates that into a "is this a bomb or not" product with a white paper outlining its failure rate in a specific situation, then C sells that product to D who uses it in an entirely different situation and E gets blown up... who gets sued? It definitely should be somebody, there should certainly be a liability and incentive to avoid such a liability but I it probably lies somewhere in C-D space, not A-B space.
How else to interpreted that? When a single bug can cause loss of life, and given that this in a thread about Uber, it’s hard to draw other conclusions. By all means though, offer another perspective on how regulating industries with significant number of lives on the line can’t manage regulation. While you’re doing that, I’d point to the aerospace sector which seems capable of both innovation and regulation.
For example: https://en.wikipedia.org/wiki/Boeing_737_rudder_issues
Although the NTSB investigated the accident, it was unable to conclusively identify the cause of the crash. The rudder PCU from Flight 585 was severely damaged, which prevented operational testing of the PCU.[3]:47 A review of the flight crew's history determined that Flight 585's captain strictly adhered to operating procedures and had a conservative approach to flying.[3]:47 A first officer who had previously flown with Flight 585's captain reported that the captain had indicated to him while landing in turbulent weather that the captain had no problem with declaring a go-around if the landing appeared unsafe.[3]:48 The first officer was considered to be "very competent" by the captain on previous trips they had flown together.[3]:48 The weather data available to the NTSB indicated that Flight 585 might have encountered a horizontal axis wind vortex that could have caused the aircraft to roll over, but this could not be shown conclusively to have happened or to have caused the rollover.[3]:48–49
On December 8, 1992, the NTSB published a report which identified what the NTSB believed at the time to be the two most likely causes of the accident. The first possibility was that the airplane's directional control system had malfunctioned and caused the rudder to move in a manner which caused the accident. The second possibility was a weather disturbance that caused a sudden rudder movement or loss of control. The Board determined that it lacked sufficient evidence to conclude either theory as the probable cause of the accident.[2]:ix[3]:49 This was only the fourth time in the NTSB's history that it had closed an investigation and published a final aircraft accident report where the probable cause was undetermined.[4]
Second:
In 2004, following an independent investigation of the recovered PCU/dual-servo unit, a Los Angeles jury, which was not allowed to hear or consider the NTSB's conclusions about the accident, ruled that the 737's rudder was the cause of the crash, and ordered Parker Hannifin, a rudder component manufacturer, to pay US$44 million to the plaintiff families.[16] Parker Hannifin subsequently appealed the verdict, which resulted in an out-of-court settlement for an undisclosed amount.
I have no problem "piercing the veil" for egregious issues. I'd have no problem holding a developer liable for failing to secure a project but just continuing on rather than quit. But "Let's just hold all the engineers criminally liable all the time!" is a bad idea and it is not already done for a reason.
Software isn't like civil engineering where you can mathematically prove that a design is sound.
All we have in engineering (non-software) is something like safety factors and confidence, and this is done with (usually) rigorous mathematical models as well as loads and loads of testing to fill in the gaps of mathematics (think unknown constant/parameters, assumptions, etc).
None of this is impossible to do for software. There are systems that enable one to do easy/entry level verification (such as something like TLA+), to much more complicated reasoning (something like COQ). This will allow the system designers to gain confidence in if the system will work and gain understanding about under what scenario they will fail. Contrast this with the existing software landscape, which is mostly, at least from my perspective, just let me write some stuff until things do approximately what I want. Even at the top of the ladder, I feel the tests conducted are "adhoc" at best and with none of the rigours that you associate with traditional engineering fields.
I'm going to cauterize the off-topic debate about the US healthcare system by pointing out that OP was talking about the expense to doctors of malpractice insurance, not about costs to the patients or medical outcomes.
Malpratice liability varies widely by country, but it's a non-trivial expense for doctors everywhere, and significantly higher in states with strong tort liability for doctors.
It's hard to imagine a world with criminal liability (or tort liability) for software engineers that doesn't ultimately end up with a system of insurance for engineers, roughly analogous to the medical malpractice insurance system for physicians.
That healthcare in other countries is able to achieve this in spite of the medical malpractice insurance system points to the fact that such a system is not certain have to have the deleterious effects you confidently assume.
Whether it is a burden for engineers is another question. But the article and the discussion aren’t about the inconveniences faced by the engineers who programmed this system.
Which, as someone else noted, exists and is probably a good idea if you're an independent consultant or possibly a professional (i.e. licensed) engineer who signs off on drawings or other documents for clients or regulators.
But there are many industries where civil liabilities are required. In fact, any software independent consultant is civilly liable for their work, but it’s not specific to software.
IEEE has a section in their member toolkit that goes into why professional liability insurance is needed, https://m.ieee.org/membership_services/membership/discounts/...
The costs aren’t that high or at least they weren’t 15 years ago when I purchased it for less than $1k/year for $1M in coverage. Most people need this even if they think they are safe. If you’re the one who wrote the deployment script that erased $1M in data, it won’t be entirely mitigated that the script made it through qa.
Also interesting is that the engineer who wrote the Uber software is currently liable for criminal negligence, like pretty much everyone else. But you would have to prove culpability. I can’t find any examples of software engineers convicted so it’s hard to tell who goes to jail-developer, qa, or executive.
More info on criminal/civil negligence- https://www.theblanchlawfirm.com/?practice-areas=criminal-ne...
Almost all employees have the possibility of criminal negligence based on their work. For programmers, this could mean that if you fuck up the code for a pacemaker and someone dies, you could go to jail. That’s a big risk and I can’t find any programmer who has been found culpable for someone’s death. This is the current law in the US.
If Uber was negligent in its code, then the programmers could go to jail. They have programmers and they work and assume this extremely low risk.
Now maybe you’re arguing that some special law should or should not exist for Uber drivers.
Moreover it is not clear to me that not holding the companies that create industrial robots that kill people criminally responsible is what most people would consider just. Again, I think it’s just that there is a massive difference in the scale of exposure; there were not enough interested people to have a debate.
As a pedestrian you already run a significant risk of being killed by a car. To the extent that we hold autonomous car makers responsible for these deaths (and I'm not saying we shouldn't), we should hold non-autonomous car makers responsible for the deaths their vehicles cause as well.
Slow your roll. Nobody know why this person died yet.
We have held humans responsible because assuming a correctly functioning car they are performing the most complex and risky task, and are most able to cause problems. Likewise self-driving car software performs a complex and risky task in which failure can have serious consequences.
In this case, the operator was an AI that was negligent and it was unlicensed/unregulated. That's a new scenario. In the human case a person might go to jail for negligent vehicular manslaughter. What does 2 years of jail time look like to an AI? What does a suspended license look like to an unlicensed entity?
For scale: the risk of death from a self driving car will probably be less than the current risk of death from normal cars, and will definitely be less than the risks incurred in the 20th century from cars, buildings, etc.
Self-driving cars are definitely a new and large legal development, but there's no reason to think existing legal principles can't handle them.
As the GP states, the entire reason Uber is testing in Arizona is because their state government completely got rid of reporting regulations which were present in CA; the status quo is decidedly not the same as it is for established technologies.
As for scale, look at the other comments where people analyze the risk posed by self driving cars. Your assumption that the risk of death from self-driving cars is less is not backed up by the evidence.
It’s fine to say that self-driving cars might eventually be better drivers than humans, just like robots might eventually be better at conversing than humans.
There is no reason self-driving cars can’t be be tested in private. Uber can hire pedestrians to interact with them—I don’t volunteer to be their test subject by deciding to take a walk.
Self-driving cars are a new and important industrial development that will require adjustments to policy. They don't require revolutionary new legal principles.
What should be passed (but I can't see how) is a percentage of allowed deaths, at least in the early years, and set it to something like 5-10% of the current rate, reducing downwards to 1% after 20 years.
People will die from self driving cars, and undoubtably their will eventually be a case that is 100% the self driving car's fault. The benefit of self driving cars comes from the mistake being permanently fixed, while with human drivers it can be committed over and over again.
There needs to be some kind of protection on the companies (and obviously the developers, I've never heard someone try to say they should be held responsible before) from lawsuits. Otherwise all it'll take a is a small handful beefore companies will just let it die.
If we are going to be arguing from a utilitarian standpoint, suppose we hold the executives of self-driving car companies as responsible as if they were themselves drivers. Then if self-driving cars truly are safer as you optimistically claim, both fewer people will die from accidents involving them and fewer people will go to jail for those same accidents. Seems like a win to me.
Civil engineering and medical device manufacturing seems to be doing fine, despite having similar principles of engineers' liability.
The idea that software engineers should be held responsible for something that (as far as we can tell so far) was an accident and not the result of negligence or malice is several orders of magnitude beyond the level of liability that civil engineers and medical device manufacturers have.
Nobody said that. The original comment said developers should be held "accountable for a crime that if committed directly by a person would almost certainly result in jail time" [1].
The standards from medical devices and/or civil engineering, with the associated licensing requirements and verification processes, make sense. Even in the case of a careless mistake or strategic oversight, individuals who could have known but nevertheless signed off should be identified, if not explicitly punished.
> Nobody said that.
Well, they quite literally did, because the original comment in this thread was:
> We need to discuss how the developers self-driving cars will be held accountable for the crimes they commit. There is no reason the person who programs or the person who makes money from a self-driving car should be held less accountable for a crime that if committed directly by a person would almost certainly result in jail time. You can’t replace yourself with a computer program and then choose to take only the benefits and not the responsibilities.
I guess you can quibble about the difference between "accountable" and "liable", but that's not a discussion that's particularly interesting to have here, especially given OP's other comments in this thread which make it quite clear that this is what they had in mind.
If the developer was negligent or malicious in their duties, why not prosecute them?
Whith so few self driving cars that number sold be zero. If you can't assure safety with a few cars whith a human as backup, you should not be in the streets. And it's not the first dangerous accident of an Uber self driving car where Uber was at fault.
Something does feel wrong about punishing them when the decision to put the car on the road in the first place was almost certainly not their own.
Though I agree Uber should be held accountable for it and it shouldn't be a token fine since the whole point of punishing an accident like this is to discourage them from occurring in the first place.
This sort of accident orchestrated by a group of people probably won't be gracefully handled by our legal system.
Look, I'm all for developers and (software companies in general) to be considering the ethical implications of the work they do, and the moral obligations that they take on as a result of it. However:
> We need to discuss how the developers self-driving cars will be held accountable for the crimes they commit. There is no reason the person who programs or the person who makes money from a self-driving car should be held less accountable for a crime that if committed directly by a person would almost certainly result in jail time. You can’t replace yourself with a computer program and then choose to take only the benefits and not the responsibilities.
This is a bad mentality to take with postmortems for software failures in general, at least from the outset. You need to look at the underlying factors that contributed to the issue, not simply looking for a person to assign blame to. It's possible that negligence is the underlying cause, but not necessarily - and even if negligence is a cause, what were the other cultural factors that led to the negligence happening, without being caught somewhere else in the pipeline? It's tempting to look to assign blame, but if you do that, you'll actually miss out on the systemic improvements that would be necessary to prevent similar incidents in the future.
But moreover, this is a bad outlook to take here, because this wouldn't be criminal behavior if committed by a human. From the best we can tell, given the details available so far, it's an accident, and it's very rare for criminal charges to even be considered in accidents like these, unless it's a hit-and-run.
I can assure you that a human who is driving carelessly would be held criminal liable. Why do you assume an accident that was severe enough to have resulted in a persons death—the car didn’t just scrape them because they ran across the street—is not due to a reckless programming?
Do you see evidence that the car was "driving carelessly"? That's an honest question - from the reporting so far, it doesn't seem clear what the underlying cause was.
Secondly, this is demonstrably false: most pedestrian fatalities by vehicles do not result in criminal charges. If you don't believe me, look up the stats. Or talk to the countless bikers' advocacy groups that have been lodging this exact complaint for decades: drivers are not generally held criminally responsible, unless there are mitigating circumstances (the driver is drunk, the accident was a hit-and-run, etc.).
> Why do you assume an accident that was severe enough to have resulted in a persons death—the car didn’t just scrape them because they ran across the street—is not due to a reckless programming?
When a pedestrian dies, just because they died, that doesn't mean the driver is automatically responsible. It could have been the pedestrian's fault, or it could have been the driver's fault. Or it could be both. Or it could even be neither (a true accident, with no assignment of blame).
The same thing holds here. You can't assume that this is the result of "reckless programming", and to be entirely blunt, by jumping to that conclusion on the basis of literally no evidence whatsoever (and misinterpreting existing case law on vehicular accidents in the process), you're actually undermining the success of any future efforts to prevent these sorts of accidents in the future, whether or not it ultimately turns out to be the fault of someone at Uber.
I accept your point that this accident might not have led to criminal charges if a human had been responsible. But I don’t waver on my argument that if a human driver would have been held criminally responsible for this accident, then we should we hold the executives (or in extreme cases programmers) of Uber responsible in exactly the same way, whether that be criminal or not.
Finally, with humans and pedestrian fatalities many cases involve drunk driving or sleepy driving. Self-driving cars can’t get drunk or sleepy; they can just have bad programming or bad hardware, both installed by their manufacturer.
If one of those things caused the accident, the developer isn't to blame. Civil engineering has experience tracing liability from mistakes (and incentivizing prevention).
There's a reason sweat shops and slave labor happen more in some places than in others.
There's a reason why high pollution manufacturing moved to China. Because China was more willing to let their people die in exchange for jobs.
I assume this case will also be one of the most well-recorded cases of a fatal car accident in history as well, given the amount of sensors and equiment on-board a self-driving car, along with eye witness testimony from the operator on board.
Can't tell if Uber has just been incredibly unlucky as of late or if just enough of their employee-base is incompetent as to prevent them from just having a quiet year with no large failures.
There should probably be a specific, complex and comprehensive test for autonomous vehicles. Also I'd want to see shared liability, the company are driving it (by proxy through their software) so they should be liable to some extent.
Maybe we should start with a specific, complex and comprehensive test for human drivers first.
And to build on the sentiment of this thread, every driver should finance their own test course to practice on before "subsidizing" their learning on public roads.
Basically what you propose?
As for comprehensive testing - I think the UK car and motorbike tests are quite good, not comprehensive, but they demonstrate a general ability across a range of skills. Humans can be expected to act relatively consistently (in such things), the test must be more comprehensive when treating a system that you can't expect to have innate consistency.
Humans can solve these problems due to life experience, not just driving experience. In other words, I think we're gonna need fully-conscious AI to solve self-driving.
The only way self-driving tech will reach production is if the input space is restricted, which is a significant-but-not-groundbreaking iteration on what we've been doing for decades with airplane autopilots and self-driving monorails. Sure, we can have self-driving cars on specifically designed freeways, but nothing more.
But at this point what makes it better than a train?
Take the train, and you have no vehicle with which to go from the destination station to your endpoint. Also, you could travel at any time instead of being held to the schedule.
People would still get hurt and die directly due to software that cannot be perfected, but the gains to society as a whole might be worth it.
2. Who do you blame when a faulty algorithm eventually kills a person? Who do you get mad at? When a drunk driver kills your family member, you can go to court and look at their face and look at the faces of their family members. When that happens with a self-driving car you'll just be looking at corporate lawyers who will shrug their shoulders at you and say "lol sorry our dumb machine killed your daughter".
Slowly with many AI-driven vehicles on the road, we can "optimized" for a better performance on safety and other issues.
Current experiments and production data from human controlled vehicles have not been encouraging.
What does NP-hardness look like for self-driving car tech? non-deterministic polynomial in: number of objects? number of lanes? time steps in the planning horizon? action/observation branching factor? These things are bounded in practice.
Not saying that the computational problems aren't hard. But ending the conversation at "NP Hard" throws away too many nuances.
> Elcock said he believed Herzberg may have been homeless.
I do not think that Herzberg possibly being homeless adds any meaningful information. On the contrary, I feel this may subtly support unconscious prejudices.
In this sense, it is a possibly relevant detail of the story.
Can't speak for the reporter who asked that, but I can take a guess about why it was asked. The victim's death and identity has been known for half the day. Her being homeless might possibly figure into next-day stories about why we haven't heard much from grieving family members. But yes, you're right, it is a detail that will figure into people's prejudices.
I know the scale is different but since this is the first death I’m curious if the percents fall in line.
[1] is distance the right metric?
About 5400 pedestrians are killed each year in the US. US drivers go 3.1 trillion miles a year. So they kill a pedestrian about every 5.7 billion miles. Last November, Waymo said they had 4 million self driven miles, so well short of statistically expecting to hit a pedestrian. In September of some unspecified year Axios claims Uber had self driven over 1 million miles.
My estimate needs help, the distribution of pedestrian bearing roads and pedestrian free roads likely does not match from my total miles per year number and what carbot testers cover. Also, this may have been a cyclist death, which adds another 800 or so deaths per year.
But, in any event, in rough numbers, Uber appears to have beat their expected time to pedestrian fatality by two or three orders of magnitude.
Which seems unbelievably low. I'm getting these figures from this Wikipedia graph[1]
[0] https://en.wikipedia.org/wiki/Transportation_safety_in_the_U... [1] https://en.wikipedia.org/wiki/File:US_traffic_deaths_per_VMT...
If deaths are broken out by pedestrian vs passenger, I wonder how people will respond if safety skews heavily towards one of those groups with self driving cars.
Their organisation has been shown to be deceitful, their CEO to be devious. I can completely believe they would cut corners or claim safety where they know there is risk. To get to market ahead of the competition knowing that the fate of road users lay in a dice roll.
Technically they had to resort to stealing Waymo's IP to get ahead, and their self-driving cars have previously been shown to behave recklessly.
I do, tentatively, trust Waymo. I've seen enough about the resources they've put into this, the extensive testing they've done, and their safety record so far to at least give them a shot.
In fact, their culture of ignoring rules and common sense might be okay for business development, but with respect to human safety it is just irresponsible and inhuman when it comes to self driving cars.
The issue is that while fewer lives will be lost, it will be a different set of lives than would have been lost without self-driving technology.
Those lives will have their day in court.
Let me make a prediction. Uber will claim that the driver was in control of the vehicle at the moment of the accident. And the "driver" will receive a bonus payment in the coming months.
It doesn't surprise me for a moment that this was Uber (though it might have been a Tesla). From all I've seen and read, these companies are racing unscrupulously, and some have inferior technology compared with others.
Waymo seem to be approaching the problem from a safety direction rather than a pure race for profit.
We need to hold SDs to a higher standard than we hold human drivers. Not 1.5x the standard; 1000. 10,000. 100,000.
And every death must be treated as a serious failure of engineering. These are preventable deaths.
As an aside, I've also been hit in a bike lane in Tempe by a human driver unfortunately, if it were a robocar I'd be living on the beach somewhere off the settlement check.
From SF Chronicle [1]:
<QUOTE> The self-driving Volvo SUV was outfitted with at least two video cameras, one facing forward toward the street, the other focused inside the car on the driver, Moir said in an interview.
From viewing the videos, “it’s very clear it would have been difficult to avoid this collision in any kind of mode (autonomous or human-driven) based on how she came from the shadows right into the roadway,” Moir said. The police have not released the videos.
The incident happened within perhaps 100 yards of a crosswalk, Moir said. “It is dangerous to cross roadways in the evening hour when well-illuminated, managed crosswalks are available,” she said.
...
“I suspect preliminarily it appears that the Uber would likely not be at fault in this accident, either,” Moir said.
However, if Uber is found responsible, that could open a legal quagmire.
“I won’t rule out the potential to file charges against the (backup driver) in the Uber vehicle,” Moir said.
But if the robot car itself were found at fault? “This is really new ground we’re venturing into,” she said. </QUOTE>
[1] https://www.sfchronicle.com/business/article/Exclusive-Tempe...
My understanding is that a LIDAR provides a completely accurate 3D map of the surrounding area around a self driving car. This is in contrast to Tesla's image recognition approach which makes a 3D map from 2D images. So is seems like a pretty giant bug if a self driving car ever crashes into another object with LIDAR, since it should always know the location of external objects.
Please correct me if I am wrong!
So yes, a lidar-equipped AV completely not sensing a pedestrian would be surprising, but you can see how it might have incorrectly classified the pedestrian, or misunderstood the pedestrian's intent.
Last night there was a story on CBS about the first 'self driving bus' going live in San Ramon California. [1] Where the reporter steps out in front of it to see if it will stop (spoiler alert, it does).
And while it was a tragedy, its unfortunate that because it was a 'self driving' car this fatality gets more coverage than the thousands who are killed by 'human driven' vehicles. Bicycle advocacy groups have been arguing for years that better, separated, bike lanes would save lives. Perhaps the companies behind self driving can get behind that effort to protect bike riders from humans and make it easier on their cars.
[1] https://www.cbsnews.com/live/video/20180319105443-california...
Well, it was obviously going to happen sooner or later. It'll be interesting to see what the fallout is. Up until now regulators have been, surprisingly, taking a pretty relaxed approach to regulation of self-driving cars. Hopefully this one accident doesn't impede development on other autonomous car programs too much.
Summary: We spend too much time talking about what decision the car should make in these situations, not enough on how allowing a corporation to make those life and death decisions changes our society.
- philosophical dilemma
- physics-constrained reaction times
- actions that violate the rules of the system
- protected car lanes
- protected bike lanes
- protected pedestrian lanes
- cost
Top reasons for auto accidents today include inclement weather, reckless driving, speeding, driving under the influence, and distracted driving. In theory, most of those could be solved by autonomous driving. But then the list of reasons for accidents would change to whatever new reasons cause autonomous car accidents, such as damaged sensors, programming errors, equipment failure, road hazards, etc.
Even with autonomous cars, we will still need protected lanes, and we will still never implement them, because we don't really care when people we don't know die.
In order to allow transportation to co-exist with pedestrians without collisions, you need some kind of separation between the two. With subways, the protected lane is literally underground, but it does definitely have a protected lane. If you don't go under ground, you can go above ground, like several subways and metros do around the world.
If you don't do either of these, you have to make concessions on the ground level. My personal preference would be tall fences around the roadway, and pedestrian bridges that go over or under the roadway (but both have problems). Another would be to still have the fences, but automate some sliding barricades that would activate when traffic halted, which is somewhat like how train crossings work. We could also implement hybrid methods, like that at Shibuya crossing, for very congested intersections.
A - Human equivalent: self driving car obeys reasonable rules that a human also would eg minimum speed on a highway and kills someone
B - Trolley problem: (very artificial scenario) self driving car has to choose between killing N or M people where N > M and wrongly (in hindsight) chooses N
C - Car fault: self driving car kills someone in a situation where no human would have
As a society we would probably accept A easily but B starts to get shaky. C currently looks completely unacceptable, BUT I would argue that society has to get to a point where even C is ok conditional on the probable result that overall car deaths decline dramatically.
In other words we will have to get to a point where individual deaths are extremely regrettable but the overall death reduction of adopting self driving cars are so undeniable that the individual deaths can be discussed without also talking about banning SDCs in the same breath.
And for long haul trucking applications, freeway lanes at specific times of late night/early morning could be dedicated for autonomous driving.
We know very little about the circumstances here, apart from the obvious tragedy, but given that we learnt from Tesla that you cannot rely on cameras alone, whatever is learnt from this must be shared amoung all the industry and serve as the baseline for future.
I would prefer an outright ban on driverless cars in real world settings (until perhaps each competing car can safely navigate the worst sensor data other companies can offer) but as a minimum we need a neutral clearing house similar to the airline industry.
There are on this forum enough industry insiders, plus a sprinkling of regulators that a global self-regulated path could easily be forged.
I do think Uber should be charged if the car is found at fault. Not a civil case, criminal. We know machine learning algorithms fail all the time (like this). It shouldn't be alright to just ignore this.
Has there ever been a case where a car manufacturer is charged with a crime for a defect that results in a death?
I always imagine I would feel uncomfortable when developing software in certain areas such as this, or e.g. for space rockets containing humans. Errors have a much higher impact than when only developing web applications where the worst case scenario is a tempoarily unreachable website, or maybe some data loss.
Police can't just write "killed by python script" as a cause of death, that would be crazy, and there is no legal framework for cases like that. My guess is that A.I. could be equated to cruise-control, meaning person behind the wheel is responsible.
Maybe someone knows better?
But, to my best knowledge, there are no "corporations" on the road. No matter what company this person is driving for and under what type of contract, this person is responsible for all kinds of incidents.
>>Tempe Police Chief Sylvia Moir said that from viewing videos taken from the vehicle “it’s very clear it would have been difficult to avoid this collision in any kind of mode (autonomous or human-driven) based on how she came from the shadows right into the roadway.
A lot of comments on here are faulting machine learning for this or algorithmic flaws, but unless we see the video it is really hard to blame the algorithms. Based on the quote from the story, it sounds like someone stepped out into the street, not in the crosswalk, at night, directly in front of a car going 40 miles per hour. How could a human or any algorithm cope with that scenario?
So, brace for the incoming wave of similar news. These things will happen, and these things will be heavily publicized. Hopefully it won't cause stupid overreaction to the tune of banning the technology altogether or surrounding it with a bubble of regulations so think as to make in infeasible.
Ultimately they will blame humans, humans are the bugs in their code.
I don't even know what to make of that
What exactly do you mean by "deal with driving"? Drive without a single accident, ever? That's obviously impossible in practice. Drive better than humans, who in 2016 killed 37,461 people in the US alone? I don't see how that would be impossible - human drivers have a limited field of view, slow response times (average time to break is ~2.3 seconds), and are frequently distracted, sleepy, drunk, etc.
This is one of those self correcting problems, and it's going to be fixed way faster than the startups pushing this "smart machines" propaganda are going to like.
Uber has some fucking shitty programmers guaranteed. They murdered this lady. Think about it:
Smart cars basically have 360 degrees of sensors. There’s no way it didn’t detect that lady coming up on the car and the fact that it kills her says to me there was no counter measure. It probably didn’t even slow down until the moment of impact. That's some fucking elementary school shit.
> Chance of Collision? 100%. Maintain Speed
In structural engineering if you designed a bridge poorly and it collapsed killing people you'd lose your ability to practice engineering. The same should be applied.
https://www.google.com/maps/place/N+Mill+Ave+%26+E+Curry+Rd,...
Putting further pressure on what should really be 2 discrete planes of activity (perhaps one in tunnels below ground or in the air, and I don't mean the pedestrians!)
Perhaps now the arrogance of such bad ideas becomes a little bit more clear to the tech set of folks, that in my view were excessively pushing such strange goals upon a public that wasn't clamoring for it.
In this particular case I assume the operator will be thrown under the bus, which is also unfair.
This may well just be bad luck. But I cannot shake the feeling that if Uber started an ice cream venture, they would store their molasses on a hill in a Boston. The only way to get “humanity” associated with Uber would involve an Uber zeppelin.
Or it’s a conspiracy, because nothing is as threatening to Uber than autonomous cars. This is certain to invite more regulatory scrutiny. Just kidding... I think..
And if it can be shown that it was company policy to allow the cars to exceed the speed limit then heads should roll all the way to the top of the company.
Despite the likely outcome, that the engineers behind this took every safety percussions, I think its extremely important, to get the message across, that people with an CS background with such use cases are going to be held to the same standard as for example mechanical and electrical engineers.
https://www.forbes.com/sites/samabuelsamid/2016/12/15/the-tr...
This shift towards autonomous vehicles is utterly unnecessary. They could be simply improving the safety systems of the vehicles to intervene when the driver is about to cause an accident.
Instead they're doing the opposite - expecting the human to intervene when the robot is about to cause an accident.
Knowing that you don't know something is as important as having the facts. To harken back to Cheney, these aren't even unknown unknowns, you're basing opinions on known unknowns.
The bad outcome would be that it turns out that pedestrian safety is too underdeveloped in this system to really be safe.
HUGE difference between consumer self driving car [Everywhere, at all times] and the Machines as a Service [Geo, time, weather-fenced operations][1].
Waymo appears to be at the head of the pack; Sacha Arnoud, Director of Engineering for Waymo gave a talk a few weeks ago at MIT[2] and gives a good idea. We are about 5 years out until these start rolling out as MaaS (machines as a service). Probably 10+ years for level 4 highway operations for consumer models according to Frazzoli.
[1]Emilio Frazzoli, CTO of nuTonomy https://www.youtube.com/watch?v=dWSbItd0HEA [2]https://www.youtube.com/watch?v=LSX3qdy0dFg
Yes, we get it. Fatalities will happen sometimes, they are unavoidable sometimes, and what matters in the long run is if we can achieve a significant overall reduction in fatalities.
But my god, a person was just killed by a computer. Can't we have some compassion and humility?
Let's, as a community, set the standard for how we will react to these events. Let's make sure Uber releases detailed data on what happened, whether they were at fault or not. Let's hold the media accountable for their reporting. Let's mourn the loss of life and think about how we can solve these problems.
But for christ's sake, please stop posting the same thing everybody already knows, which gets posted on HN whenever a self-driving article comes out.
Can't we do better?
---
EDIT: I clarified what I meant here [1]
Wasn't trying to say this thread should be all about mourning. This is HN so we should talk about technology, even when a person died. I'm pointing out one specific argument that gets repeated over and over in place of a substantive discussion, and I think we can do better.
I haven't even read any of the comments on this story yet, and this type of pro-self-driving-car response is what I expect to be in the majority. That and anything anti-Uber :)
It is as if HN users expect the news headline to be: "an event occurs which is statistically speaking non-anomalous".
If one doesn't feel we have enough data to draw conclusions yet, then sure, fine. But a person just died. This is a newsworthy story and it deserves our best attention and discussion, not repetition of something obvious ad nauseam.
When the sensational-to-informational ratio is so large, we always get this.
Is there a term for this tendency? Curious to read the research on it.
Alan Kay pointed out that Kahnemann's slow-system vs. fast-system also applies here.
Do I understand you correctly that you'd be okay with the comments if the were just prefixed with some "thoughts-and-prayers" genuflection? And if so, why not say it directly?
It's not the lack of compassion, it's the lack of substance.
And I suppose it strikes me as lacking compassion and/or humility to come and use HN as a platform to recycle the same obvious idea, when the story is about a death. It would be different if it were about a tire blowing out.
Everyone knows how these statistics work on a macro level. That's not worth commenting here on HN, and especially not on this story.
I just think we could do better.
>Everyone knows how these statistics work on a macro level. That's not worth commenting here on HN, and especially not on this story.
If statistical comparisons were obvious, and people didn't overreact with unjustifiable countermeasures, then indeed it wouldn't be important to make the point. But the world we live in is one where nuclear deaths are scarier than heart disease deaths, and shark attacks more than auto collisions.
I think you're committing the fallacy of "my social circle doesn't make that mistake, therefore it's not worth pointing out."
I can agree that users should be more careful about making the common point when others have made it, though -- I had a reply on this thread about "Uber flouted the $150 license" that I deleted when I saw others made a more informed response to the same point.
The fact that Uber's self-driving car program has a worse deaths-per-miles-driven metric than human drivers is anything but obvious to me.
After an incident like this, it's natural to question whether anything needs to be done policy-wise. And, I don't see any reason why that discussion can't take place here in the comments section.
It seems like you want HN users to use the comments section to express condolences? Or, to just have an empty comments section?
Would you care that statistically self-driving cars have killed less people than human drivers?
For example it's clearly excusable to be angry / depressed / looking for someone to blame when a personal tragedy happens to you. But that doesn't mean you should replicate that behaviour when a tragedy happens to someone else. We should distance ourselves from subjects where we're emotionally compromised, not guide our decision by the example of emotionally compromised people.
The only strange thing in this thread is the emotional detachment many are showing and one could probably argue that's the exact kind of thinking that will lead to more cases like this.
How you would react when emotionally distraught shouldn’t be the model for how you act all the time.
This is in the site guidelines: https://news.ycombinator.com/newsguidelines.html.
I don't think there are enough data points right now to make such claim, but please correct me if I am wrong.
What do we do differently based on the answer to that question?
I realise that the point you're making is about emotional turmoil and not really statistics, but it looks like so far Uber aren't doing so well in terms of driving safety:
https://news.ycombinator.com/item?id=16620736
So yeah, if it turned out that Uber were sending cars out into the world without sufficient engineering or testing* and they killed someone (especially someone I know) then I would be understandably furious.
* In order to get to market sooner? I don't trust Uber and I wouldn't put this past them. Nothing I have seen of the Uber self-driving car program makes me want to be anywhere near one on a road.
Shit, of course. But I'd like to believe that I'll realize that banning self-driving cars is not going to make things any better, just like I'd never propose banning cars in general. I'm not sure how it would help here to imagine it's my daughter; it's not as if we imagine it being our daughter in every traffic accident we hear of, since that would be crippling given the sheer number of them...
This death is a tragedy. There were 37,461 such tragedies in 2016. Every single one of them matters equally. It is only by studying and analysing them in aggregate that we can prevent future tragedies. Seat belts, air bags, crumple zones, anti-lock brakes, median dividers and rumble strips are tremendous acts of compassion. Statisticians, scientists and engineers have made an incalculable contribution to human welfare.
no. there's a reason our community has the reputation of being heartless. it's because it's true.
What is true is that large internet forums suffer from systemic effects that compound into problems that no one has yet figured out how to deal with.
what in the world did you think i meant?
you can try and pretend all you want, it ain't my fault if the mirror ain't a pretty thing to behold.
People are being killed in accidents constantly. We aren't going to be in a constant state of mourning over it.
This is implicitly a discussion about autonomous vehicles, and the very currently relevant and heated topic of wether we want these things on our streets.
There are about 260M[1] cars in the us in 2015, so perhaps more today. That means there are ~4e-7 deaths per car per day. On the other hand, we have 1e-3 deaths per autonomous vehicles per day if we naively use the death of the person in this story. That means so far, autonomous vehicles have a rate of killing people a ~2500 times that of normal cars.
Of course, it's one data point, but out the gate, it's not looking so strong.
[0] https://www.theverge.com/2017/2/21/14687346/uber-self-drivin...
[1] https://en.wikipedia.org/wiki/Passenger_vehicles_in_the_Unit...
How many self-driving cars are there on the roads right now? And what's fatality rate? I' quite sure it's higher now than compared to human-only driver.
> they are unavoidable sometimes
Were autonomous cars not designed to prevent such situations?
>Let's, as a community, set the standard for how we will react to these events
Yes and let's punish non-conformists!
> which gets posted on HN whenever a self-driving article comes out.
I honestly have no idea what that comment is.
NHTSA data measures number of fatalities by the hundreds of millions of miles driven -- the latest rate being 1.18 fatalities per 100 million miles: https://www.nhtsa.gov/press-releases/usdot-releases-2016-fat...
Those are not people I want in my engineering community; just impossible to trust they'd make the right calls and have the right priorities in developing cool new stuff.
That this event doesn't necessarily reflect a larger trend is not incorrect--it's just that it's obvious. That self-driving cars could potentially reduce fatalities in the future is not only obvious, it's also somewhat off-topic when the story is that a person just died.
So my question is, when a computer kills a person, can't we do better? Better than stating the obvious, and better than being dismissive.
Is arguably the whole problem with our industry in a nutshell. The more relevant and largely ignored question should be "how this affects human beings".
If you ask a question "through the lense of technology" ("Can we solve this hard computer-vision problem"?) you arrive at different answers to those you would get if you ask the question through (say) the lense of societal benefits or humanity's relationship to technology ("Is it ok for a badly programmed computer to accidentally kill someone"?).
I say this not to criticise the enormous benefit this technology could bring, but rather to provoke us to think about the implications of the technologies that we are creating outside the confines of an IDE.
edit: NM, I can see where you'd see that, even though I argue that both being concerned about the death, and wanting to defend Uber, aren't mutually exclusive. Instead of saying "this story is only relevant to HN", I should have said, "this story is only noticed by HN and the national media". Which is true, as far as I can tell. I can recall few nationwide stories about other pedestrian deaths, even though nearly 6,000 occur every year.
>'How low has the SW development bar gone, if "it's okay" now means "at least it's not directly killing people"?' https://news.ycombinator.com/item?id=16541235
We now see the standard has slipped to: "Is the death of one person relatively so bad? People die every day."
Next a large crowd of people will be killed and the standard will become: "It's not so bad compared to creating skynet or starting thermonuclear war."
I'm not sure what comes after that.
I mean, we two are still capable of discussing it, at least!
But if you think about it, that's not a pattern anyone should be unfamiliar with. History is full of it.
I don't know, I really have no strong opinion here. But I think stuff like this is truly fascinating. There seems no end to where rationalization can go.
Welcome to the ambivalence of reality...
I don't know, however, how that comparison turns out. From the other comments, it looks like Uber's self-driving cars might be less safe than human-driven cars--in which case you and I would end up on the same side of the argument.
You can make the same argument that people die in non-VR surgery all the time so it is no such a big deal.
Lot's of people are talking as if human beings are a resource which exploitation should be secured for the businesses and technical curiosities.
No, people, humans are what matters and it just happens that there are business opportunities and scientific curiosities. No, it not O.K. to kill few people to speed up the development of self-driving vehicles. No, it is not O.K. to have total surveillance so that we improve AI.
We need to start cleaning our own house, or it’s going to be cleaned for us by people with agendas and little understanding of what they’re cleaning!
You have to recognize that your culture and your moral standards are not universal and you are in no position to force them on everyone else. It is also foolish to expect everyone across the globe to react in a certain way to certain things.
On the other hand, death topics is the side of HN that is heavily biased towards what mods feel about it and their culture. They even ban accounts and give out warnings if you happen to disagree with them and react differently. So there is not much sense to even discuss this.
We need to see accountability for any death caused by a car being operated by a computer. We need to see integrity in testing and metrics that show that these are actually safer for humanity. One more death than expected due to software vs human is too many.
I don't remember where I saw it but the process for creating mission critical software at NASA is insane. They analyze everything. It's a completely different and slower process than the one we use for building websites. SpaceX has probably increased the productivity of said systems, but I'd like to see how they develop mission critical software to keep things safe.
The point of self driving cars is to reduce death, if you can monetize that great, but if you can't hit that baseline in some sort of testable way, then you shouldn't be able to put your car on the road.
The victim's life had meaning. To herself, to her friends and loved ones, and all of those affected by her untimely loss. We don't know her story, nor the stories of the over 6,000 pedestrians (a number that's been increasing) killed annually in the US alone as of 2016.[0] Nor the 37,461 killed in US motor vehicle crashes in the same year.[1]
We talk about the technology, because it was involved in this woman's death and because the technology has the potential to drastically reduce the overall number of such accidents. Understanding why she died, and how similar accidents can be prevented in the future, can give society back something out of an accident that only took. If we can learn from her death, anything at all, it behooves us to do so.
Yes, it's cold. It's extremely cold. But it's something doable. Even if these discussions offer little impact on the actual development of self-driving cars, they can still impact the political debate that's only now just starting. For better or worse, there will be those who point to this woman's death as an argument against the technology. Recognizing that her death is not simply one death, but an example of a type of incident that claims thousands each year without self-driving car involvement, is critically important. At times, statistics can dehumanize and cause us to overlook the very real pain depicted as a small part of a number. But at the same time, they also permit us to recognize problems and measures ways in which we can mitigate and even solve them.
0. https://www.npr.org/2017/03/30/522085503/2016-saw-a-record-i...
1. https://crashstats.nhtsa.dot.gov/Api/Public/ViewPublication/...
Not implying that the lack of empathy is in anyway acceptable, but people should know that when machines are involved in such scenarios it is much easier for them to distance themselves emotionally, this is not new, weapons are a prime example of this. Maybe by knowing this dilemma people can react better to such tragedies.
I hope people show more empathy knowing this.
"The fleet of self-driving Volvos arrived in Arizona after they were banned from California roads over safety concerns."
People from different localities drive differently. I hope they use drivers from say, NY, LA, and various small towns to teach the AI.
People that lives in towns with ~ 10k people drive differently that a new yorker on the same town.
EDIT: They might be training with a biased sample
via twitter.com/adamscrabble
The first person killed by a car:
http://www.guinnessworldrecords.com/world-records/first-pers...
How long before someones tries to program a car to "hunt" a victim and hit her when she makes a mistake and it will appear it was a pure accident with no-one at fault?
At least with automated cars, there will be blood.
Perhaps her estate should receive a perpetual royalty.
American's seem to enjoy the "X doesn't kill people. If only more people had X, less people would have to die." argument.
<braces for downvotes>
Although I agree with your statement in general, it sounds like she was already walking her bicycle while crossing the intersection, and the fact that she had a bicycle was a moot point.
So, on any given day in 2016, there were 2.6 fatalities, with less than 1 in the pedestrian/pedalcyclist groups.
[0]: https://www.azdot.gov/docs/default-source/mvd-services/2016-...
Waymo et al has had none of these issues. Time to revoke their licence to test?
I mean, it's Uber. I'm not sure exactly what people were expecting. Their entire schtick is being unethical and poorly-run.
If there’s more to your thought than “the ends justify the means,” I’m curious what it is.
This is like saying the Duterte is a very effective anti-drug crusader if you accept that murdering innocents and drug dealers is in his DNA.
a) what % of disengagements are 'to avoid accidents' vs navigating an entirely safe but complex situations. For ex: a parked UPS truck on a narrow 2 lane rd.
b) the degree of real-world complexity the tests the cars are engaging in (compared to other vendors). For ex: testing cars on simple suburban routes = fewer disengagements.
Not trying to defend Uber here, I'm just trying to understand the distinction between a high-level general statistic and it's real world implications.
There is no reason whatsoever to believe that Uber's horrific disengagement record, which is orders of magnitude worse than competitors often in the same cities like SF, is because they are tackling orders of magnitude harder situations per mile or are orders of magnitude more conservative, and every reason, even before this fatality (the only one so far despite many self-driving programs running concurrently over years), to believe they were just plain worse.
> Not trying to defend Uber here, I'm just trying to understand the distinction between a high-level general statistic and it's real world implications.
Again, it has to be dangerous for someone. It can't be more dangerous yet not dangerous to anyone in particular. That just doesn't make any sense. Uber runs on the same roads and the same traffic with the same basic approach as everyone else, there's no confounder which could produce a reversal.
Exactly. Fool me once, shame on you; fool me twice, shame on me.
It tells a lot about Uber that they weren't willing to fork over $150 for a permit.
People will always relish in their biases being confirmed... even after a rare one-off event with a sample size of 1 in a burgeoning industry.
https://www.theverge.com/2017/2/27/14698902/uber-self-drivin...
Nothing to revoke if they never had a license to begin with!
Local governments have stopped other self-driving companies before with cease-and-desist letters with threats of serious punitive fines. I highly doubt by not buying a $150 license Uber would suddenly have carte-blanche to do what they want, that's not how it works.
If the state gov was really interested in stopping them they easily could have regardless.
> Waymo et al has had none of these issues.
... absence of negative media coverage doesn't prove anything.
I think Alphabet is to blame here too in a sense (or at least the industry as a whole). Why didn't they push for more strict testing rules? It's in their interest too.
[1] - https://www.theguardian.com/technology/2016/dec/19/uber-self...
Maybe self driving industry needs to have some safety rules too to prevent when the unexpected happens.
(Do we count Tesla's autopilot as self-driving?)
...there's a "standard move" when one of their cars kills someone?
I don't think that really came across the way they wanted it to.
Here's a picture of the crumpled bicycle (note dent on the front right of the car): https://twitter.com/daiwaka/status/975767445859287042
The sign it is lying next to lets us locate the exact point of the collision on Google street view: https://www.google.com/maps/@33.4370667,-111.9430321,3a,75y,...
It's right where the vehicle lane crosses the bike lane approaching the intersection. Very unlikely a pedestrian was walking a bike there. Much more likely the Uber hit the bicycle while merging.
EDIT: I take back "very unlikely" a pedestrian was crossing there. There is in fact a very weird "X" pathway in the median that pedestrians are not supposed to be in. I didn't see that until I looked at the satellite view.[1] So possibly she emerged from there and was walking her bike across the street. That still looks pretty bad for Uber though, since that street is a straight shot with clear visibility and the dent's on the right side of the car. Meaning it would have swerved left to avoid a late-detected obstacle rather than swerve right away from her entering the roadway. She also would have literally had to walk straight into oncoming traffic. I still think bicyclist is the most likely scenario and it's odd the PD doesn't mention the crumpled bicycle.
[1] https://www.google.com/maps/@33.4365931,-111.9425027,198m/da... (street view shows a "no pedestrians" sign)
According to the CDC, 5,376 pedestrians were killed in car accidents in 2015. According to the WashPo, 28,642 pedestrians were killed in car accidents from 2010-2015. That gives a rate of 13-15 pedestrians killed per day in the USA. Too much, yes, but not hundreds of pedestrians killed everyday.
https://www.cdc.gov/motorvehiclesafety/pedestrian_safety/ind...
https://www.washingtonpost.com/local/trafficandcommuting/ped...
https://www.npr.org/sections/thetwo-way/2018/02/28/589453431...
The urban fatality rate of all fatalities is 1 per 100 million miles traveled: https://crashstats.nhtsa.dot.gov/Api/Public/ViewPublication/...
As of September 2017, all of Uber's autonomous cars have driven a total of 1 million miles over a period of 2.5 years: https://www.axios.com/ubers-autonomous-cars-have-driven-1-mi...
There's an argument to be made that autonomous vehicles are better for society and safety. But it sure isn't statistical.
I was referring more towards their negligent design than self-driving cars themselves, but if we're going off of that:
In 2015 there were 5,376 pedestrian deaths by a vehicle. There were, at the time, an estimated 263.6 million vehicles registered.
There has been a death so far this year off of a self-driving vehicle, and there are how many on the road at the moment? If you're going by deaths-per-vehicle, autonomous is losing.
I don't dislike all autonomous vehicles, but Uber is showing a lack of care for all but basic safety measures, that seemingly all of their competitors have gotten down thusfar, and alongside of that they're avoiding regulation quite heavily.
We should just really build rails. Self driving cars are dumping billions of dollars onto a problem better solved by rails. Several cities already have self driving trains including London, Singapore and Kuala Lumpar, and they transport millions of people every day without incident:
http://penguindreams.org/blog/self-driving-cars-will-not-sol...
Self driving car research is asinine. It won't even being to solve the transport problem in America and it's round-peg/square-hole tech. It has the cool factor but there are existing tech that are insanely more useful for a fraction of the cost. American hate public transportation (thanks GM) that we'll never see real/good transport implemented in America.
The beauty of cars vs rails is that cars can go more than just a set in stone angle or path defined by a rail. Cars can go in and out of parking lots and dirt roads. Rails would be more automated, yes, but not as versatile.
Autonomous cars aren't a viable tech until the roads are reengineered with rails built in. It's just way too big of a technical problem to solve with computers alone.
Right now the death rate of autonomous cars is far higher than human drivers and it is never going to catch up given that it hasn't advanced technologically recently.
If our rail network was as potholed and badly maintained as our roads, people would die in train accidents every day.
Rails are great for intensive, carefully manicured routes. They are not a substitute for roads.
Then there are the cases of engineers who are distracted and what not, but I'd imagine we're talking autonomous so that (hopefully) wouldn't be an issue.
And as far as dirt roads go, if the self driving cars can't make it through well defined, paved roads what are the odds they are going to accurately navigate some dirt road?
It seems like a much better plan is to separate drivers from other people more rigorously, and maybe mandate people drive the last-mile themselves (with AI help, obviously).
As someone who lived in West LA for years and saw the new line come in, I'll stop you right there. Cities like LA, Mexico-City, San Jose, etc. are not well designed and trying to re-invent them is going to take one hell of an earthquake.
My brother lived off Geary for a few years and went to a few public meetings on the proposed subway line they want to put under it. He was interested in the idea, as the N-Judah is a methadone clinic (last he was there a few years ago, caveats apply). He never saw such NIMYism and such tight resistance by the property owners on Geary to any improvements whatsoever.
You don't just ease regulations or really do anything 'big' in any city. Palms must be greased, and if you do not it is a horrific process that takes at least a decade.
I'm not saying that you are wrong about the free market. However, my brother thought that the most fantastic thing about the new Star Trek movies (esp. the new Wrath of Khan) was not the warp-drive, nor the aliens, but the very tall building in SF. Even in 400 years, there is no way the planning boards would allow such things.
I used to ride a recumbent to UCLA now and then because I had a death wish, and it shocked me that it took 25 minutes and yet people spent 60 minutes in traffic in their cars every day just to get across the 405.
You're right, though, those cities are mostly lost causes, more for the attitudes present in them than the infrastructure. Doesn't mean we can't try to make any new ones better though.
Riding a recumbent to UCLA from 16th is hardcore. Everyone I knew that rode in west LA got hit at least once, if not more.
If there is anything to be nostalgic about the past over, its how civil engineering projects of such scale were a lot more feasible when the complexity of doing them was so much lower.
I invite anyone who buys into this sweeping generalization to walk a half mile in Las Vegas, NV; Tempe, AZ; Houston, TX; New Orleans, LA; or Tampa, FL in July or August.
Walking half a mile in 35 C (95 F) isn't an actual problem unless you have a medical condition or you are morbidly obese. For those very rare 40+ C (105+ F) days, throw an official midday siesta, or just shut everything down like for snow days.
Again with the hand waving as if this was a solved problem. Trust me - if you actually try to walk half a mile on a 90+ degree, high humidity day you will regret it. The issue isn't just infirmity - i.e. those prone to heat stroke or overweight, but one of comfort. If you actually try tooling around in Houston (for example) you'll end up soaked in sweat. It'll be worse if you're dressed for the office in long sleeves and slacks.
That's when you'll discover that more than the public need to adapt to public transportation uber alles. Can I use a shopping trolley for groceries? Sure. Can I cycle to work? No doubt. But what if your employer doesn't have showers available? Mine doesn't. I can't show up drenched in sweat looking like a wet rat and stinking all day of sweat. And that's exactly what'll happen if I ride the few miles I have to work. Even if I take a change of clothes to work I still have no place to shower and neither does anyone else unless they are health club members. There are many externalities you're just hand waving away. As someone who lives in this climate I assure you I'd love to see light rail everywhere but it just won't replace the last mile for people without drastic societal changes.
There's also the fact that bus and rail frequency needs to dramatically increase to entice riders and also accommodate them when they do come. Will people choose a 45 minute bus ride over a 15 minute drive? Particularly when on average you'll spend 7.5-10 minutes baking in the sun or lightly shaded by an un-airconditioned bus stop? Not unless forced to, no.
I mean, I'd love to see you medevac a gravely wounded person from the middle of Arctic tundra to a hospital 120 miles away on your feet.. except I wouldn't, because we have helicopters for a reason. We don't assume you should have to use a helicopter to get groceries though.
Similarly for long haul trucking applications - could the far left lane/carpool lane be safely dedicated to autonomous trucks during the hours of midnight-3AM with some bright blinking signs paid for by the autonomous trucking industry trade group?
There will still be the last-mile problem. Even if the "rail" is just cheap paint, you'll almost definitely never have 100% of roads painted, and paint wears over time or gets buried under snow etc.
Eventually we're looking for ways to add "more 9s" to the reliability and safety. A non-rail rail could add a bunch of 9s to well-trafficked routes.
Self driving cars can't come quickly enough.
Unfortunately there are features of most US cities (lack of public interest, obesity, disability, low density, long commutes, weather) that block our adoption from approaching yours.
Cycling is great, I hope to get along in life without a car, but cars are still great to have as an option.
According to the US Department of Transportation, there are about 5,800 train-car crashes each year in the United States, most of which occur at railroad crossings. These accidents cause 600 deaths and injure about 2,300.
We should require that every traffic participant gets a sensor, and connect all in a (local) network, instead of only relying on machine learning and its vulnerability to the unpredictable situations where accidents occur. Pure autonomous self driving won’t be the final answer here.
Please correct me if I’m not up to date here, but to my last knowledge self driving car machine learning lacks the most important data, that of dangerous situations. I’m surprised that Uber considered it ready for roll out.
Every car has plenty of communication and localization systems today. Nearly every pedestrian carries an electronic device with potential for exact localization. It makes no sense to not use that data, and solely rely on still-not-perfect computer vision. The optimal system will necessarily be a combination of both.
As for standardisation, maybe realizing what an enormous number of fatal and many more forever life-altering accidents we currently consider normal will finally get people off their backsides. I wish somebody would start a media project revealing how insane the current situation is..
The reason that fatalities are 10x higher in Asia is that people drive (and walk!) like lunatics and vehicle fatalities are just shrugged off.
Just be more orderly is basically what I am saying. Also eat more sauerkraut - it's healthy!
Having driven about 2000KM on the Autobahn as a tourist the lane etiquette on the Autobahn is the best I’ve ever seen, and I’ve driven through the U.S. (both East and West coast) and much of Europe. It’s no surprise to me that the Germans can drive on de-restricted roads and have the one of the lowest number of deaths (HIGHWAYS ONLY) in the world.
Maybe some people don’t want to live in dense urban areas, maybe they don’t want to ride on a rail, maybe they want privacy. But looking out the window, I can say American society has soundly rejected the rail as the ultimate solution.
Short of understanding your perspective as a longing personal desire for trains, your conclusion is so far removed from reality that it’s nonsense.
There would be considerable US pushback against such a system, sadly.
You can't say people reject something they don't even know is a possibility.
Every sci-fi movie of the last 50 years has solved that problem: you don't use trains; you put the cars on the rails (by grouping them together into roller-coaster-looking setups.)
> maybe they don’t want to ride on a rail
Same here: cars on rails can, in most conceptions, signal to the rail system to switch them onto an offramp that dumps them back onto a road.
Basically, here, "rails" are serving as safer replacements for highways. Every road that isn't a highway would still be a road. You're just enabling people to, effectively, "have their car take the train", rather than having to leave their car at the train's parking lot.
I guess the obvious problem is that it would require simultaneously modding both private vehicles and public infrastructure at impossible scale.
It seems like such an obvious design end but implementation would require communism :/.
“Soundly”? If you mean in the way a child “soundly” rejects vegetables, maybe. But, the American public has never had this conversation. Capitalism and it’s ensuing cultural affects had it for them.
How will that help? The reason that the trains in London and Singapore don't kill people is not that they are on rails, it's because they're underground.
You need to put the trains below ground, or well above!
But there are a couple of really big, really massive problems. We have trillions in existing infrastructure and trillions more in existing property that's been developed over the course of decades. A lot of that development has come with unintended consequences. Just look at Houston after Hurricane Harvey and how their development and zoning practices took the insane rains Harvey deposited and magnified their negative consequences.[0] Decades of short-sighted decision-making took a hurricane and made it worse. Some changes will likely be made, but they won't solve nor likely keep up with a problem that's only going to get worse. That'll happen because the alternative, demolishing huge portions of the sprawling concrete city, isn't economically viable. The city can't be moved.
It's baggage. And there's so much more. Tens of trillions of dollars worth. Then there are existing roads, bridges, and other infrastructure connecting all of it. Designed for cars, and much of it legacy infrastructure that doesn't take into account modern infrastructure design principles. Add cities and towns, some of which are (charitably) in less-than-optimal positions. Towns, strip malls, and other structures that wouldn't exist if it weren't for suburban sprawl. Baggage. And then there's the people. People with ideas on what their lives ought to be like and where they want to live. There's a town, Centralia, Pennsylvania, that is literally on top of a massive underground fire that's been burning for over half a century. There was a massive relocation effort with federal and state funds. The town has no services, its zip code was discontinued in 2002, and it's still burning with all of the potential health risks that involves. There are still five residents there today, who have fought every attempt to relocate them. They will continue to live there until they die, at which point their homes will finally be taken and demolished through eminent domain.[1] If we can't bulldoze a town like that, we sure as hell can't take a bulldozer to normal ones.
Baggage. What it means is that we can't simply replace everything. It can only be done piecemeal, which means new infrastructure must take into account the need to accommodate everything that was developed under the old. High speed rail can be built out in the United States, and I genuinely hope that it is. But it literally can't replace the car in its entirety.
Self driving car research is--compared to the cost of major infrastructure replacement, let alone an alternative that simply isn't feasible--practically a rounding error. It doesn't hurt us, because the option was never a binary choice: self-driving cars OR rail. Self-driving cars give us the ability to deal within existing legacy constraints as they're slowly adapted, modified, and/or replaced. They also offer some long-term possibilities that will allow us to make radical changes in urban design. And yes, those changes are ones can accommodate rail, pedestrian-friendly city design, and other options that can drastically improve quality of life.
0. https://www.npr.org/2017/11/09/563016223/exploring-why-hurri...
1. https://finance.yahoo.com/news/pa-residents-living-above-min...
Cities only spread out over the past century. You start building rail again and it will get drawn back in. But the State has to fund that mode of transportation first.
Ridesharing algorithms will 100% solve the problem of underutilised mass transit. Self driving isnt necessary.
Imagine shuttles that hold 10 people using uber pool algorithms and funded by the city.
People will be able to make their request and a nearby shuttle will pick all the people up in that area. The shuttles could do ad hoc transfers with other shuttles so people wouldnt even have to be going to the same area.
The algorithm would minimize the average ride time for everyone.
The #1 issue with mass transit is lack of convenience. Rideshare algorithms will eliminate that lack of convenience.
The police are saying pedestrian walking outside the crosswalk but here's a picture of the crumpled bicycle: https://twitter.com/daiwaka/status/975767445859287042
You can see it's next to a sign. That sign lets us determine exactly where it happened on Google street view: https://www.google.com/maps/@33.4370667,-111.9430321,3a,75y,...
It's exactly where the vehicle lane crosses the bike line approaching the intersection. Mostly likely the Uber and bike collided when the Uber was crossing into the right turn lane.
EDITed to remove offensive word.
Please follow the site guidelines when commenting here. Adding flamebait into a flammable topic sparks explosions, making this place worse for everyone.
Context: How new technology is introduced on public streets and we have no choice or say on whether we want it or not, even if it endangers us and gets us killed.
> When motor vehicles were introduced they appeared to increase man’s freedom. They took no freedom away from the walking man, no one had to have an automobile if he didn’t want one, and anyone who did choose to buy an automobile could travel much faster and farther than a walking man. But the introduction of motorized transport soon changed society in such a way as to restrict greatly man’s freedom of locomotion. When automobiles became numerous, it became necessary to regulate their use extensively. In a car, especially in densely populated areas, one cannot just go where one likes at one’s own pace one’s movement is governed by the flow of traffic and by various traffic laws.
and
> Even the walker’s freedom is now greatly restricted. In the city he continually has to stop to wait for traffic lights that are designed mainly to serve auto traffic. In the country, motor traffic makes it dangerous and unpleasant to walk along the highway. (Note this important point that we have just illustrated with the case of motorized transport: When a new item of technology is introduced as an option that an individual can accept or not as he chooses, it does not necessarily REMAIN optional. In many cases the new technology changes society in such a way that people eventually find themselves FORCED to use it.)
Source: The Mainfesto: INDUSTRIAL SOCIETY AND ITS FUTURE http://www.washingtonpost.com/wp-srv/national/longterm/unabo...
But at the same time, I don't personally want to go back to an age before automobiles became commonplace. They're too useful to me.
Is that selfish? Sure. But our lives are all shaped by our wants, not by our needs.
Someone above mentioned how this type of dependency problem has happened since the first tool -- I don't think so. The problems I deal with daily are drivers who are staring at their smartphone as they turn, or people simply trying to beat the pedestrian across the crosswalk. I've had moments where my life flashed before my eyes more than I can count, and every single time the driver waves their hand at me like it's my fault. I'd like for them to get out of their mobile uterus so we can discuss this like adults, but instead I'm left yelling at their big metal machine, and they're yelling at the interior. And this is in a "walker friendly" area.
Neither would I, but I think the cautionary value in the discussion of the resulting inadvertent restriction of pedestrian freedom is not to visualize an auto-free world as an alternative but instead to visualize a new set of customs of urban development that aggressively protect pedestrian freedom. In the real estate world, the walkability movement is concerned with this and should achieve greater influence.
It's been been happening since the invention of the first tool. Stone weaponry, agriculture, and so forth, all became largely non-optional after their introduction.
Check again. Bow and Arrows are often considered weapons at the same level of firearms, and so many laws apply.
This is offtopic, but I am reminded of how Slack initially supported IRC gateways and recently discontinued them.
It’s not a percect likeness to this quote’s point. But I think it’s interesting this pattern of initially optional technology, then de facto standardized technology emerges ubiquitously.
I suppose it’s arguable that embrace, extend and extinguish is the purposeful, deliberate manifestation of this pattern, arbitrated by a single entity (i.e. a company with a new technology).
Edit: Jeez an immediate downvote? The comment was barely posted a minute ago :)
Anyway, I just finished reading Kaczynski's first book, "Technological Slavery" (Feral House, 2010) It's an amazing work and it elaborates on this point quite a bit. Recommended.
You might also want to check out his most recent work, "Anti-Tech Revolution: Why and How" (2016)
Here's what MIT's student newspaper had to say about the book:
"Anti-Tech Revolution: Why and How is Kaczynski’s well-reasoned, cohesive composition about how revolutionary groups should approach our mercurial future….. I recommend that you read this compelling perspective on how we can frame our struggles in a technological society." -- The Tech, MIT's oldest and largest newspaper
"batshit crazy" haha. A political classification if there ever was one.
I don’t care how good your models and data are, until you’re able to write an algorithm that can fully handle and learn from a situation it was completely unprepared for, it will always encounter edge cases like this.
This is what happens when we have a society of people saying AI is already here, people like Elon Musk saying AI is a “significant threat” despite "AI" being essentially a black box statistical model in it's current form. From what I’ve seen, the general public thinks we’re 20+ years of AI than we really are.
You need general AI before AI can drive a car, full stop. Otherwise, you need to isolate roads from other human drivers pedestrians, and cyclists. It just won’t be reasonably safe until then.
1.3 million people died in the USA from car crashes in 2017. An additional 20-50 million are injured. You don't need to be perfect, you just need to be better than the baseline.
That said, and to be fair to your feelings, Uber certainly is not better. This is a fact I'm sure lawmakers in Arizona are discussing right now.
https://www.google.com/maps/@33.4369934,-111.9429875,3a,75y,...
I suppose you could have a wall that separates the bike lane from the road except for a short area where cars can cross over, so that at least there's only a small space where the bikes & cars can interact, but that introduces a bunch of new problems (cars that can't see bikes behind the wall when they go to cross over, cars that run into the wall, probably even more swerving to get into the lane because your choices are do it RIGHT NOW or you're stuck/hit a wall).
There are ways to design an intersection so that those rules are followed naturally, and there are ways to miss it. C.f. right turn lanes.
The problem is that the lane markings encourage vehicles to make turns from the non-rightmost or non-leftmost lane. One should not be crossing another lane when making a turn at an intersection. One should merge into the rightmost or left most lane in order to make the corresponding turn.
Or just do what Germany has sensibly done and split sidewalks between pedestrians and cyclists (though, cycling enthusiasts are unlikely to enjoy the experience as much)
However, I do need to mention that when you choose to bike on the sidewalk, you're creating the same type of problem that cars that ignore traffic rules create for bikes. There's a reason why biking on the sidewalk is illegal in most places, and why bike lanes/roads exist.
Agree. I'd do it anyways because sidewalks are 99.999% empty where I live. I could ride for 20 miles and not see a person on the sidewalk. So biking on sidewalks is safe as long as you pay attention to what's ahead. No idea if that's the case in this AZ location.
I might be exaggerating a bit.
I don’t fully think I’m a bike I get right of way. If I see a car reversing ahead I move into the street then prepare to full stop but proceed with caution and watch the cars acceleration to see if they see me. If I get the hint they don’t. I stop. It’s scratch on the bumper for them and life for me I am more than glad to yield.
Which are what? Such a broad statement doesn't seem accurate and the sidewalk shown in the map looks low traffic with excellent sight lines, as long as you're not cycling head down at high speed the danger to pedestrians doesn't seem comparable to danger posed by car driver's ignoring traffic rules.
Riding a bicycle on the sidewalk isn't illegal most places because it's dangerous for the pedestrians, it's illegal because it's dangerous for the bicyclist.
I just find it odd when cyclists out of principal decry riding on the sidewalk when the danger of riding with traffic is well known and America has an abundance of long empty sidewalks in suburbia and around office parks.
Many dedicated cycletracks have similar problems. Here in Austin there are several that I refuse to use because drivers far too frequently turn into my path without looking. I ride in the lane so that I can be seen. It's counterintuitive to most drivers, but visibility is often the deciding factor in bike safety. This would be more obvious to drivers if they spent more time cycling. If a cyclist is stuck using a path with poor visibility, I find paying attention to the cars and using light touches of an air horn when approaching an intersection to help. But I still prefer being visible to that.
More generally, intersections are relatively more dangerous (crashes per person mile or something like that) than straight segments of road for any mode of transportation using the road.
Vehicles, on the other hand, do follow traffic rules with regards to lane usage, signaling, and checking what's behind them. Even if they don't, they're still more predictable compared to pedestrians.
The only bicycle riders willing to get organized and lobby the legislators are the very most serious ones. These are the people who want to race along at 50 MPH.
The resulting laws force pitifully slow riders to be out in the street. Even if you can only manage 5 MPH, you have to be in the street.
Lots of us are too arthritic, too fat, too weak, too inflexible, too limited by lung capacity, too limited by heart capacity, and too limited by cheap steel mountain bikes from Walmart.
So our choices are:
a. break the law
b. annoy drivers and risk death
c. don't ride a bicycle
A better design for that intersection would be to keep the bike lane against the sidewalk and the turn lane inside of that, so that they don't cross, and have a "no turn on red" sign, so that the paths of bicyclists and people turning in cars never cross as long as everyone obeys the stoplight and sign.
I can do it, why can't they?
Sure, drivers should check before crossing a lane, but some percentage don't, so it makes sense to minimize lane crosses.
And even assuming this was simply a case of a bad driver, what policy approach would you suggest to protect the general public against such drivers? Accepting that drivers are fallible and designing our road systems to be robust against that seems a more effective approach than berating those drivers who are particularly unlucky in the consequences of their failures of attentiveness.
Perhaps I should have said "wait for turn indicator" instead of "no turn on red", that's a bit clearer.
(It's hard to see a bike at speed going straight when a car is turning right.)
Here's an image that illustrates the danger: http://www.sfbike.org/wp-content/uploads/2013/02/Right-Turn....
This is addressed by having a "no turn on red" sign as I suggested: you don't have to see the bicyclist at all because the bicyclist shouldn't be in the intersection at all when you're turning with a green turn arrow.
And the majority of deaths in London is due to turning transporters/trucks. They are purposely built such that the operator sits very high above and can't even see a bike or pedestrian unless it's many meters away, only by looking into one of 10 mirrors (but of course they don't). On a work site nobody moves these things unless there is an outside instructor, but on public streets we've decided to just blame whoever died. Or, if that fails, blame the infrastructure, even though we oppose any other kind of infrastructure.
Human drivers are terrible.
There are exceptions, but only if there is signage or road markings. For example, if there are two or more lanes marked for right turns, then obviously you can turn from any such lane.
(A little-known fact about that: most California drivers are aware that you're allowed to turn right on a red light after making a full stop, unless there's a "no right turn on red" sign - but few seem to know that this rule applies to all the lanes marked for right turns, not just the rightmost lane.)
Most bike lane markings change from a solid white line to a dotted line about 200 feet from a corner, to give drivers a hint to merge into the bike lane before turning. But even if the line doesn't turn dotted, drivers are still required to merge into the bike lane unless there is a specific indication otherwise.
However, the majority of drivers seem to be unaware of this rule and turn right from the auto traffic lane, creating the risk of a "right hook" collision, which the law is intended to avoid. I've actually had other drivers honk at me when they were waiting in a line of cars to make an illegal right turn from the auto lane while I made a legal and proper turn from the bike lane.
The San Francisco Bicycle Coalition has an excellent info page about this:
http://www.sfbike.org/news/bike-lanes-and-right-turns/
Laws on this do vary in other states. But at least in California, if you see a driver merging into the bike lane before turning right, it's not because they are a terrible driver, it's because they are following the law and (hopefully) increasing safety for bicyclists.
What's more concerning is that my state (Texas) seems to have a lot of vagueness around the laws.
Well, that's confusing to say the least.
If you still don't see the difference, here's one way you can tell. If you are stopped, waiting at the intersection, if you've merged, then you'll be occupying the bike line while you're waiting.
Yeah, like a line is going to protect me from run amok car drivers who are too buy on their cell phones. Or it's going to stop the bus drivers from using them as pickup/dropoff areas, cutting bicyclists off.
Long story short, I bike on the sidewalks. I can be a responsible biker on sidewalks while heeding to pedestrians. My life is more important than to be biking on a vehicle road.
Please support the cause as a local citizen!
http://amp.abcactionnews.com/2064993596/tampa-will-be-home-t...
http://www.bikewalktampabay.org/featured/new-protected-cycle...
Anecdata: The times I have interacted with one of Uber's autonomous vehicles at this intersection, they tend to hit the brakes pretty hard as they are about to enter the turn lane and I am around 10ft behind them. All of my interactions with them, however, have been during the day, and it was unclear if it was the human driver in control.
Uber ruining self driving cars that can save 20k lives each year and improve our economy and get millions out of poverty and improve everyone's quality of life by being short sighted, financial driven, and bringing the worst aspects of Silicon Valley to the extreme.
I am so sorry to everyone at Google and Waymo who have tried so hard to not let anything like this happen by being extremely cautious and safe. Ughhhh I feel so frustrated, this is going to cause so much backlash.
Travis, Anthony and whoever else pushed for this to go ahead without caring about safety is evil. This action doesn't have a direct correlation of evil like directly murdering someone. But these highly intelligent individuals ignored their conscience and the easy deduction that actions to compromise safety, laws, decency for quick profit are immoral. Whether they admit it or not they KNEW that this was a high possibility and refused to take precautions creating this awful culture. 20k lives a year, and all the benifits that will be delayed for years are on their head.
They won't face any consequences, just convince themselves of absolution of guilt and do nothing to remedy the situation, but I have no respect for them.
Road deaths are ~40k each year in the US alone.
I agree about Uber; smells like shit, looks like shit, tastes like shit, you better believe it's shit to the core.
If the autonomous cars kill 1100 per year, but autonomous car companies fund a bunch of studies saying the benefits outweigh the drawbacks, it'll be politically acceptable.
At the very least, it might call into question whether safety drivers are actually providing any real safety. Localities may start to crack down on using that kind of excuse for self-driving tests.
I don't think the "safety drivers" do much other than provide liability-shifting for the company.
This is in the site guidelines: https://news.ycombinator.com/newsguidelines.html.
Also, where is your P.Eng be any useful to justify the learning coefficients of your AI, or to protect about the cosmic ray bit-flip in the hardware your proven correct software is running on ?
Someone has to sign that stuff as "production worthy or I go to prison".
https://www.media.volvocars.com/global/en-us/media/pressrele...
I think the same thing applies to autonomous things.
So yeah.
Autonomous vehicles are expected to be able to safely navigate a large variety of streets and highways, and possibly trails and countryside in the future, without killing anybody else around or killing the people inside, nor destroying the fixtures and buildings that make up your typical city street. The only problem is cars are inherently deadly, go look up the number of people that have died in automobile "accidents" since the automobile was invented, total it up for just one country. They need to be treated with and afforded the same respect as any other instrument intended to project deadly force, because they do, as do whatever algorithms are controlling the instrument.
Everyone I've known in law enforcement, especially CHP officers, does this by default (though they tend to prefer “collision”) when referring to an event where a car hit something. As one of them explained, “collision” or “crash” is a readily discernable physical fact; “accident” is a conclusion about the mental state of the participants.
The idea that an "accident" isn't the drivers false is an absurd premise. Drivers are obligated to be diligent. "Crash" is a neutral term that doesn't assign liability to anyone.
Sure, you might shoot yourself because you're just careless with your gun. Or you might be also thinking of an argument you had with your spouse, or you might be tired when you're handling your everyday carry on your way to work, or you might get startled, or a million and one other things that might prevent you from singly focusing your attention the gun you're handling.
Machines exist to serve us, not the other way around. If a product doesn't take human habits and flaws into account, it's a bad product, because over a large enough sample size someone will make a mistake.
It's not, because the basic rules of gun handling are designed in such a way that failing to obey one of them will not result in injury. They are:
* Never point a gun at anything you don't wish to destroy * Keep your finger off the trigger until you're ready to fire * Always assume the gun is loaded * Know your target and what's beyond it.
> Machines exist to serve us, not the other way around. If a product doesn't take human habits and flaws into account, it's a bad product, because over a large enough sample size someone will make a mistake.
Modern firearms are absolutely designed to this criteria - it takes a willful act to discharge them. If someone "mistakenly" fires a gun and hits someone, it requires multiple failures and clearly rises to the point of negligence.
You know what system we've come up with to improve car safety that has only one rule to remember? Always wear your seatbelt.
You know what we've come up with that requires zero effort from the driver? Laminated glass. Crumple zones. Airbags. Automatic braking.
This isn't about whether people use a product negligently — put it in enough hands and it will happen. The point is to design products such that the consequence of negligence is not injury or death.
- AI decides between one life lost as a better potential outcome
- AI made no decision and just happened to hurt someone
The idea is that making a decision between killing one person vs killing something more valuable than one person (eg. multiple people) is something that drivers do on a regular basis. Computers may not be capable of correctly evaluating the ethics of that decision.
For example - What if someone steps out into the road and you can either hit them, or run into a wall? The former makes it far more likely the occupants of the car will be safe, the latter saves the life of a pedestrian at the expense of possible injury to people in the car.
It is kind of an interesting question, because people can't generally react quickly and rationally enough in a situation like that for it to matter, but in theory, a computer can. But it also dominates the discussion in a way that likely far overstates how frequently this will actually be a decision a computer has to make.
I think that is what the person you're replying to is asking. Given that this occurred at night and on (very likely) uncrowded streets, I feel like its far more likely the car (and backup driver) just somehow missed the pedestrian.
I.e. did the uber car kill this 1 pedestrian, because it avoided killing 2 other people? Did the uber car opt for the "least killing" in this siutation?
I think this whole trolley car question - while a nice philosophical question - is silly though. I dont think humans would do any better in an emergency situation. In that split second when you suddenly find yourself in the "oh sh-<IMPACT>" situation, do you have time to a) make a deliberate, reasoned & fully-informed decision, and then b) control the vehicle effectively to follow through on that decision? I doubt it. If you had a second or two to think it through and then deliberately make a choice and steer towards that crowd of nuns, you'd probably be able to entirely avoid the accident anyway.
More likely is you'd probably do what I am sure most people do which is gasp, stamp on the brakes, shut their eyes and hope for the best ... assuming you even had time to realise there was about to be a crash before it happened. Some people might swerve, but I suspect they do that instinctively to avoid something in their way, not as a decision to hit something else.
I just dont think self-driving cars will get themselves into situations where they have to chose who to kill in the first place. And even if they did get into those situations, I really, really doubt it would be due their actions, and I'd certainly trust it to avoid the crash in the first place a whole lot more than the average human driver in the same scenario.
Some could argue "Ah but yes computers are so fast that they CAN make that informed decision about who to kill in milliseconds! So they have to make a choice! The question stands!". I'd argue back that in those milliseconds before the crash was inevitable, they'd act to prevent the accident before a human would even know what the hell was going on anyway. After that it just starts getting into a game of who can conjure up the most ludicrous hypothetical situation that rarely - if ever - happens in real life.
For example, if a kid runs out into the road in front of the car, it can drive into the kid, likely with no injuries to the passengers. Or it can swerve and drive off of the road, or into another lane, likely saving the kid but putting the passengers at much greater risk.
As you say, in this or any similar situation the human reaction is probably going to be to instinctively brake or swerve without a chance to consider options or consequences. I feel like a computer will have a few cycles to spare to make decisions like that.
I go back and forth with thinking the algorithms in things like self-driving cars should be public. Anything which must make a choice such as smash into bus or smash into bicycle — the public should be aware of the choices the company chooses to implement. Obviously this example is simplistic but it illustrates my point.
Today I’m definitely leaning towards it should be public and scrutinized. Next month I may be worried this would hamper progress.
This is how counting works.
Just because there was no driver does not keep it from being a crime. Do the engineers go to jail or do the managers or the investors?
Obviously we'd need to know the number of miles driven in each category to get a meaningful comparison, but let's all keep in mind a USA where only one person per day is killed on the roads would be a two order of magnitude improvement over today.
[1] https://en.wikipedia.org/wiki/Motor_vehicle_fatality_rate_in...
Having said that, one concern one has with self-driving cars is that one can't "reach an understanding" with them the same way pedestrians and drivers regularly negotiate how they're going to behave with each other.
If one doesn't know one is dealing with a self-driving car, one might consider certain situations safe, because they usually would be, but the car negotiates and notices things differently than a human.
That being said, let's not over-react. This is a single death. These are going to happen. Right now self-driving cars are essentially teenagers learning how to drive. En mass there is going to be a death here or there, but, also en mass, we're going to have a permanently safer road once they've learned, if we can secure them from cyber attack.
Try to avoid all-or-nothing thinking here. Definitely advocate for regulations, and maybe Uber wasn't safe enough, but people are either going to die from self-driving cars or they're going to die from self-driving cars taking too long because of public outrage.
And there's still no proof that self-driving cars are safer than humans when confronted with novel situations, i.e., the situations most likely to result in injuries and fatalities. If anything, the evidence thus far is that self-driving cars are more dangerous than ones driving by humans.
The proper comparison to be made should be between miles driven from autonomous vehicles vs. those of humans and then looking at the incident rates.
Still not better than your average driver, but even if we start by replacing alcoholics and routinely poor drivers the net impact on safety goes up.
The fact that a computer was behind the wheel is the most important factor in this discussion, because the argument for self-driving vehicles is that they, unlike humans, can react to abnormal situations such as jaywalking.
Thankfully this is not the outcome of each and every jaywalking incident, but it is a possibility with all of them. That’s the risk you consciously accept when you walk in front of moving vehicles that weigh thousands of pounds.
That actually depends on the state. “Arizona law requires pedestrians to cross the street at corners and/or designated crosswalks...[if] it seems unlikely that a driver should have avoided---as in a case where the pedestrian darts right out into traffic, then it may be that the pedestrian will be deemed 100% at fault for his/her injuries.” [1]
I’m sure that Uber will be sued over this, and they’ll probably settle quickly and quietly to avoid mass hysteria resulting from misleading headlines such as this one. But that doesn’t mean the jaywalking pedestrian isn’t at fault. Walk in traffic, die in traffic. It’s a pretty straightforward physics problem.
[1] http://www.zacharlawblog.com/2015/11/whos-at-fault-in-an-acc...
> Arizona law also requires drivers of motor vehicles to pay attention and to drive "reasonably" for all traffic conditions. "All traffic conditions" includes the existence of pedestrians in or near the roadways.
> So, should an accident occur between a jaywalker and a car---if shown that the driver could have/should have seen the person and could have/should have been able to avoid, then without question the driver can be held responsible.
We don't know enough either way to cast judgment on fault yet. But to your original point, about the headline being misleading, we don't know enough to know whether the victim was jaywalking, only that preliminary reports seem to suggest she was engaging in that behavior. But labeling her a "jaywalker" strongly implies that she was cited as such, which of course hasn't happened in this case. Calling her a "pedestrian" is a fair choice at this point, not a "misleading" one.
edit: the headline for this story has since been changed from "Self-driving Uber car hits, kills pedestrian in Arizona" to "Self-driving Uber car kills Arizona woman crossing street", which seems fair to me.
Autonomous vehicles should automatically be 100% liable in every accident in which a person is injured, and it should be paid for by a levy on every vehicle.
A levy is fair because the per vehicle risk is much more uniform than with a driver behind the wheel.
100% liability is fair because the software should be written so that the vehicle predicts the path of all objects within its range and adjusts is speed/trajectory so that there is negligible risk of collision given the worst case behaviour of the object. If an object doesn't show on the car's sensors, then its still the car's fault as the sensors should be built to they do pick up all objects. There is a complete power imbalance otherwise: software cannot be injured in a collision but people can, so all people should be 100% covered to give the software incentive to be the best it can.
Summary: a car is governed by predictable physics, hence there is no such thing as an accident involving an autonomous vehicle, only mistakes.
If the vehicle can only crawl at 5MPH because you never know when a pedestrian might just dive in front of you... that's just impractical for real vehicle operation.
It's one thing to hit the brakes because somebody decided to jaywalk without looking; another to expect there is always enough space to stop in those situations.
For example, how many people do you see walking along freeways? If a person is on a freeway then cars should be slowing down for them. It's a rare enough event that cars could be programmed to avoid all collisions with minimal impact on average speeds.
If a car is around people then it should be going at a low speed. If there are enough people around that it can't get speed up, then that car is effectively in a "shared" or "local traffic" area and should be going slowly.
I'd argue that the effect on average speeds will be similar to having low speed limits on residential streets:a slower period at journey beginning/end but the majority of the journey would be at speed on sparsely populated arterial roads.
Here's another argument: intrinsically safe programming might even increase average speeds? Why do we even need traffic lights and pedestrian crossings with autonomous vehicles? If vehicles were 100% safe, we could do away with dedicated crossings, and there would be no need for vehicles to waste time at red lights, merely slowing down on the relatively rare times when it is necessary.
Assuming Arizona hasn't produced law specifically for fatalities in their public road self driving test program... The algorithms are just one detail of the total system design, and the design of the system isn't really the issue. People don't get killed by designs or code.
The verification of it's safety, and the decision to deploy that system on uncontrolled public streets will be the issue. People made these decisions, not an algorithm.
Arizona has something similar:
> By legal definition, there are three or more crosswalks at every intersection whether marked or unmarked.
Source: ADOT Traffic Engineering Guidelines and Processes, Section 910.1
Today you can cross in the middle of the street and take a calculated risk that the people approaching down the street will stop or slow down enough to not kill you. If they go ahead and kill you anyway, they'll still be prosecuted because their attention should be on the road, and the most basic assumption behind their driver's license is that this person is competent enough and fit enough to drive what amounts to a deadly weapon without killing the people around them. When you violate that assumption, the fault is probably on you. Obviously there are exceptions, such as when somebody intentionally jumps in front of your vehicle, but crossing outside of the crosswalk to get across the street is not even close to the same level as trying to commit suicide and fault will be found accordingly.
But yeah. We all agree it's not cool to run down pedestrians or bikers or pedestrians walking bikes etc. no matter where they may be.
If you require 100% impossibility of killing anybody regardless of what the vehicle and the person is doing - it is achievable only by making those vehicles nearly useless - such as lowering their max speed to something 10 mph (maybe even lower since it's still possible to push a person who will slip, hit their head on the pavement and die). If the vehicle is moving fast and somebody jumps onto a street, there are physical limitation of what can be done. So, if this technology is to exist there will always be a space where accidents can - and eventually will - happen.
That might be a fair question, except that one death at this early stage of limited realistic trials makes me doubt the correctness of your belief. On the statistics that we have right now, I'm not seeing how autonomous cars are safer.
>Self driving cars aren't going to be perfect.
Shouldn't self-driving cars be held to a better standard rather than compared generously to objectively terrible existing standards? That we think of pedestrians regularly killed in "accidents" as acceptable collateral is already pretty horrific.
Why?
NHTSA reports 1.15 fatalities per 100 million vehicle miles travelled in 2015.
Waymo advertises >5 million road miles travelled. Let's say Waymo + Uber have driven 10 million vehicle miles and killed 1 person. That makes them 10 times more dangerous than human driven cars.
https://en.wikipedia.org/wiki/Motor_vehicle_fatality_rate_in...
2 deaths in over 2 billion miles is roughly 10 times safer than human drivers.
[1] https://electrek.co/2016/11/13/tesla-autopilot-billion-miles...
How about we test it properly before allowing it on public roads?