Hacker News Reader

Comment by rayiner | Hacker News Reader

> In the six seconds before impact, the self-driving system classified the pedestrian as an unknown object, then as a vehicle, and then as a bicycle, a preliminary report from the National Transportation Safety Board explained. While the system identified that an emergency braking maneuver was needed to mitigate a collision, the system was set up to not activate emergency braking when under computer control.

The prosecutor made the wrong call here. This part is absolutely criminal negligence. Putting a “self driving” car out there that doesn’t have emergency braking enabled (apparently because it creates too many false positives) is an unjustifiable risk. Working emergency braking should be the first thing perfected, before the computer gets to control the car.

View on HN

Arizona law is pretty friendly to motorists. Generally speaking, you're not supposed to jaywalk and when you do, you are taking your life in your own hands and the car that hits you is not liable. It's even worse for the pedestrian -- even at an intersection if you walk quickly onto oncoming traffic you do not have the right of way and are responsible for anything that happens to you. It's just not the case that the car is assumed to be at fault in situations like these -- the pedestrian is assumed to be at fault. The pedestrian is not automatically given the right of way and is basically never given the right of way outside of an intersection. This is different than the laws in other states like California.

Situations like drunk driving can make a difference, but in this case that's a hard claim to make. Maybe if the car was swerving out of its way to hit her, you could make that case. But arguing that it wasn't doing enough to stop -- that's a tough sell given AZ law. The car had the right of way, it was not acting maliciously, and she walked right in front of it. According to AZ law, asking the car to make what might be a dangerous last second swerve into another lane or slamming on the breaks to avoid hitting her is not a legal obligation as these are unsafe maneuvers. There might be another car behind them or other cars/pedestrians alongside them so the law in AZ doesn't require these types of high risk actions to avoid hitting pedestrians that walk right in front of oncoming traffic.

Moreover this person was jaywalking at night in an area without street lights, without even bothering to look both ways, across a median, wearing a black hoodie, through a high speed road right in front of a car that had proper headlights and was going a steady speed.

Anyone familiar with AZ law knew that Uber wasn't going to be charged. You may think the law itself is wrong and that cars should be legally required to take last second high risk evasive maneuvers. These are all tradeoffs to which reasonable people will disagree, but changing the law is the job of the Arizona legislature and not of the prosecutor, who made the correct decision, even if it seems like the wrong decision to you.

Minor correction: the area was actually very well lit with street lights. The Uber dashcam video was just poor quality so it gave the impression it was really dark.

https://arstechnica.com/cars/2018/03/police-chief-said-uber-...

I certainly believe the Uber dashcam video was poor quality. But at the same time, the videos on the link you showed are a bit misleading because the holiday lights are not strung up all year, and in any case the collision happened after the bridge, where to the right of Mill Avenue is a big golf course which is going to be dark at night, and to the left are some desolate office buildings that would also be empty and dark. It's just not a well lit area, neither is it an area with a lot of pedestrians, so there aren't many crosswalks.

You can explore for yourself here:

https://www.google.com/maps/@33.4380916,-111.9434265,3a,75y,...

From the article:

"In this nighttime video, posted to YouTube by Brian Kaufman on Wednesday, the scene of the crash can be seen around 0:33"

"Black says in the video as he drives past the point in the road where Herzberg was hit (around 0:33)."

Ahh, OK, so those Christmas lights are gone by then. Yes, if you look, you see a park/golf course on your right and some office buildings on your left. I guess how well lit it is will depend on how many lights are on in the office buildings, plus of course the street lights. But if you poke around the area, you see that the street lights are insufficiently spaced, at least that's my impression. That area north of the river is not a pedestrian area, even along Mill.

The victim was struct pretty much directly under a streetlight, not at the halfway point between two streetlights. And the lights in office buildings are pretty much irrelevant, streetlights way overlight them.

It is irrelevant how different the lighting might be in other places or at other times.

This [1] has video from the dashcam as well as commercial video from news crews immediately on the scene. Keep in mind the news video is heavily lit from all the flashing lights. It's not as dark as the Uber video makes it look, but it's also nowhere near what you'd call "very well lit" especially given that the pedestrian was wearing all dark.

Something that stands out to me on ars clips is that the street lights look like fireballs, even from a distance. I'd guess they have their brightness or some other setting pumped way up. Also looks like the drivers are cruising with brights and/or halogens.

[1] - https://www.youtube.com/watch?v=ufNNuafuU7M

> This [1] has video from the dashcam as well as commercial video from news crews immediately on the scene. Keep in mind the news video is heavily lit from all the flashing lights. It's not as dark as the Uber video makes it look, but it's also nowhere near what you'd call "very well lit"

1. It absolutely is. The specific spot (00:33 in the Kaufman video) is clear as day on a cellphone.

2. Visibility doesn't even have any relevance in the first-place, self-driving systems don't rely on visible-light cameras, there is no obstacle in the middle of the road, there is no fog, there is no rain, the self-driving system had full and perfect visibility all along.

> especially given that the pedestrian was wearing all dark.

Given how well-lit the road is, that would have made them more visible against the background, not less. Again not that it has any relevance.

> Something that stands out to me on ars clips is that the street lights look like fireballs, even from a distance. I'd guess they have their brightness or some other setting pumped way up.

Yeah, sure, who'd think a camera with a finite and middling dynamic range (such as a cellphone's camera) would try to actually capture information at night and thus saturate on bright direct light sources.

> Also looks like the drivers are cruising with brights and/or halogens.

Yes of course, a rando would equip their car with an omnidirectional halogen which somehow magically brightly lights up the roadside but doesn't move with their car. You've certainly cracked the code here.

2. Visibility doesn't even have any relevance in the first-place

It has relevance in showing that the pedestrian was failing to exercise their duty of care and was thus at fault. Jaywalking at night wearing a black shirt is pretty negligent. Jaywalking even in the day is negligent in AZ, but at night, across a major artery, it's crazy.

Unless you are going to argue that they knew it was a self-driving car, and so thought that the car would see them in the dark. But they apparently didn't even know a car was there, so that's a tough sell. They didn't even look if a car was coming.

A lot of people here are having a hard time coming to grips with the fact that a pedestrian has responsibilities and can be at fault in a collision with a car -- e.g. that the pedestrian doesn't automatically have the right of way or a right to be avoided every time they step into a street. I get that this is a tough concept for a lot of people to internalize, but if you're going to be arguing that someone should be charged, then at least the broad outlines of the law in AZ need to be understood.

I don't think people are arguing that a pedestrian has an absolute "right to be avoided every time they step into a street". But many of us (I think) do believe that a driver -- or an autonomous driving system -- does have some responsibility to take reasonable care to avoid accidents, regardless of who has the right of way.

While the concept of right of way may often have a bearing on who is held at fault for an accident, it does not -- or at least should not -- entirely absolve the driver of any responsibility to drive attentively and carefully in all circumstances.

In this case, it does sounds to me as though the pedestrian was at fault -- particularly in the context of AZ law -- but it also sounds as though both Uber and the "safety" driver were seriously negligent in their responsibilities, and their negligence was a major contributory factor in the pedestrian's death. They should be held to account.

That's not a minor correction at all, it's a proper correction of the smearing of the victim.

Furthermore, Uber had specifically tasked the "safety driver" with both "safety driving" and operation & oversight of the automation system, wilfully splitting their focus where other manufacturers (and previous Uber trials) had one person at each post such that the safety driver could actually focus on safety driving.

Even if it was broad daylight, it still would be unlikely that the driver would have been charged.

Read the section on "duty of care" here: https://www.jacksonwhitelaw.com/az-personal-injury/auto-pede...

and note that drivers only have a duty of care to yield to pedestrians who have the right of way. Pedestrians jaywalking never have the right of way. They are only allowed to cross at intersections.

Drivers do not have a duty of care to yield to pedestrians who do not have the right of way, although drivers have other duties.

> Even if it was broad daylight, it still would be unlikely that the driver would have been charged.

I'm not arguing that the driver should have been charged at any point. I didn't do so even back when the event happened.

Here, I'm using "driver" as a proxy for Uber as the entity controlling the car, not the person sitting in the driver's seat, who was apparently reading. Sorry for the ambiguity.

E.g. if a human was driving the car and the car did the exact same thing, as the Uber car did, but it was broad daylight, then they still most likely would not be charged. If you jaywalk and get killed, it's basically your fault, and the only exception would be if the driver was drunk, or speeding, or not obeying traffic signs, etc. Jaywalkers don't have the right of way, and cars are not obligated to yield to pedestrians who don't have the right of way.

In Germany you have to stop when you see somebody on the street, independent of their right to be there in the first place and independend of the fact how many laws they are breaking with it. You have to stop unless say, it is impossible for you to do so because they were jumping on the street suddenly or it is plausible that you didn’t see them.

This is because a persons right to be alive outweighs your right of way at any time and it is none of your business whether that other person acts lawful or not.

Interesting to see how much this general principles differ.

I think we all agree that there is a moral imperative to stop.

But that doesn't necessarily mean that the law should try to perfectly reflect that moral imperative. Laws generally don't work well when they attempt a high resolution of morality, because the law can only approximate justice, and the more complex the approximation, the more arbitrary and ambiguous the law becomes, which is itself unjust.

The specific problem here is who decides whether the driver did "enough"? The driver can say "If I tried to slam on the brakes, I was afraid my car would spin out of control" or "If I tried to swerve away, I was afraid I might collide with someone". They might say "I thought there was a car behind me, and if I hit the brakes, I would injure that person. It was an honest mistake that the car which used to be behind was no longer there." Etc.

Then you have to decide whether you believe them. In an environment where someone is innocent and must be proven guilty, you don't get a whole lot more precision by adding more precision to the law. Or do you drop the requirement of presumption of innocence?

Also, that leaves a lot of discretion to the prosecutor, which may be abused, or imposed arbitrarily, and is ambiguous.

So you are creating a lot of ambiguity in the law in an effort to precisely match the moral outcome.

The other option is to have clear responsibilities. The pedestrian must do X. The driver must do Y. If the driver does Y, he is not charged. When both X and Y are followed, there can be no accident.

But "Must do all you can" is not clearly defined. "Not speeding, obeying traffic signs" -- this is more clearly defined. So in AZ, the law is a little more clear, but at the expense of not being as morally precise. It's a reasonable trade off. It may suffer from an abstract moral critique, but I'm not sure on balance it delivers less justice.

I thought it would have been a much simpler case of "Driving without due care and attention" of which both Uber and the human driver were guilty of.

But you need to define "due care". I gave a summary of the AZ definition of the due care that applied to both drivers and pedestrians. E.g. pedestrian has to make sure to cross at cross walks, driver has to obey traffic signs and yield to those with right of way, etc.

If you use a circular definition of "due care" and make it something like "due care means you must take all action that is reasonable to avoid an accident" you've again ducked the issue of giving due care a well defined meaning.

> But that doesn't necessarily mean that the law should try to perfectly reflect that moral imperative.

Indeed, which is why such laws should and are aimed at road safety.

> the more arbitrary and ambiguous the law becomes

"When you hit a pedestrian with a car, you're in the wrong", it doesn't become more unambiguous than that.

> do you drop the requirement of presumption of innocence?

That's framing the debate. If you hit someone with your car, you're guilty of hitting someone with your car. There is no "presumption of innocence".

> So you are creating a lot of ambiguity in the law in an effort to precisely match the moral outcome.

No. No ambiguity, and no matching moral outcomes, but improving road safety.

> in AZ, the law is a little more clear, but at the expense of not being as morally precise.

But it's not more clear, and moral precision in not a goal.

> It's a reasonable trade off.

Here I should draw up the statistics of accidents involving cars and pedestrians in Arizona and Germany, but I don't think that's necessary.

> It may suffer from an abstract moral critique, > but I'm not sure on balance it delivers less justice.

That's just an incredibly U.S. centered point of view. I don't mind but your parent was talking about the German system. Maybe AZ works in comparison to the laws in other states with a very car-centered way-of-life, but in comparison to Germany it's really just bat-shit insane if the goal is improving road safety, but granted, that might not be the case.

> "When you hit a pedestrian with a car, you're in the wrong", it doesn't become more unambiguous than that.

Even if a pedestrian runs out onto a highway and it's physically impossible for you to avoid hitting them?

> Even if a pedestrian runs out onto a highway and it's physically impossible for you to avoid hitting them?

The rule over here is that a driver must always be in control of their vehicle. Exceptional circumstances such as those you describe (which are completely and utterly unlike those of the Uber case) would act as mitigating factor, possibly down to no fault if there was no way for a reasonable driver to avoid the accident.

But the default state of things is that if you decide to put yourself in control of a multi-tons vehicle, you better be able to handle it or the consequences of your recklessness.

> "When you hit a pedestrian with a car, you're in the wrong", it doesn't become more unambiguous than that.

Such a general principle would be unreasonable. I'm all for putting pedestrians' safety first, but a driver who is following all rules and safety principles can still hit a pedestrian without any fault - the extreme (but not only) example is somebody who decides to suicide right when you're driving by.

In Russia at least for awhile there was a thriving business of insurance fraud where pedestrians try really hard to get cars to hit them. Dashcams became essential safety equipment. Throws a lot of mud in that water.

> example is somebody who decides to suicide right when you're driving by.

a friend of mine had this exactly issue. he was driving at the speed limit, as usual. an old lady just jumped in front of his car.

at first, he got a murder charge then it was dropped when the prosecutors saw on the road cameras that the old lady 100% wanted to die.

In the case of an unavoidable suicide, it gets mitigated up to a point where any course of action is pointless because it would not serve road safety. The outcome would be the same.

The difference here is that even if the pedestrian or cyclist was in the wrong, you're not automatically in the right. Your own behaviour as a motorist stands on itself, and you're supposed to take the difference in size and weight into account.

Although it may not appeal to everyone's sense of fairness and just punishment(although, in fairness, only one of the parties can realistically be mauled or even killed), but from a road safety perspective it leads to more careful driving.

I agree with your reasoning. There are plenty of situations where you can do everything in your power to avoid hitting someone, and still hit them. n of 1 anecdote: While driving down a 40mph road with tall hedges on the side of the lane I was in, at night, a homeless man darted out of the hedges directly in front of me. I jammed the breaks, but still hit him going about 10mph. He got off the ground and ran off before I could even get my hazards lights on. If I'd swerved right, I would've gone through the hedges off a steep embankment into a river, left would've taken me into an oncoming vehicle. I had maybe 25 feet to stop. He was at fault, and I did the best I could possibly do. The law proposed by craigsmansion is insane in this scenario.

Just a small correction. It's not something proposed by me. It's a description of how things are at the moment, at least in the north of Europe.

The difference lies exactly in the details you describe. If an accident happens, you're not in the clear because you had "right of way", but because you, having good control of your vehicle, exhausted all possibilities to avoid the calamity.

Oh my, I had no idea that it was as such in Europe (EU?). In my particular case, there was so little time to "exhaust all possibilities". In fact, I only had the opportunity to attempt a single action, and that is without knowing what the outcome would be. To make matters worse, it wasn't even really a calculated action; I've taken a few combat driving courses in the Marine Corps that contributed to me reflexively making the correct decision. I just acted from the gut, based on my mental state. No matter what the outcome had been, I would've felt like I had no time to decide to react differently.

My impression is that pedestrians don't jaywalk in Germany nearly so much as in the U.S., as there is a cultural taboo against it. People get mad at you when you do that in Germany, or at least that was my impression -- you can confirm or deny.

So it could be that with pedestrians more conscientious, you can put most of the blame on the drivers by default and not have it be an unfair system. That wouldn't be suitable in the U.S. where people are generally less responsible both as drivers and as passengers.

In terms of statistics, yes, Germany has ~7 vehicle deaths per billion km-mile travelled whereas the US has ~4, but again, car ownership is much more common in the U.S. and necessary to get to work or do chores. People drive every day, when they are tired, and since almost everyone has to drive long distances you have a lot more irresponsible people doing it out of need. I also think that the streets are laid out in a less pedestrian friendly manner which makes them less safe.

All of those ambiguities can be resolved by a court trial. The whole point of trying in court is to decide how the law applies in a particular case. The ambiguity is actually a feature because it leaves it up to the individual but clearly requires the individual to try to avoid hitting pedestrians.

As opposed to Arizona where they very clearly value human life less than your right to get to work early.

This is the most detailed and well reasoned comment I've read here. Thank you for your thoughts!

Yeah, Arizona law seems to put drivers über alles.

I'm sure in pretty much any country a driver is supposed to break if there's a chance of collision.

But consider the following situation: An Average Joe drives down the road. Suddenly out of nowhere a pedestrian appears. Joe hits the brakes, but too late, the poor guy is killed.

He just didn't have time to react. There's no Joe's fault, unless it can be shown that he saw the pedestrian, could stop, but deliberately decided to teach him a lesson.

Now if exactly the same thing happened on a crosswalk, a driver would be at fault. A crosswalk, by design, is a place where pedestrian are crossing. In this case the driver should have expected a person appearing on the road and be prepared to stop. Unless it can be proven that a pedestrian has done something nasty, like jumped on the car from a tree.

The driver was watching TV on her phone. She might still be charged.

You're right that the bullet point list in your link doesn't have anything that applies to uber, but there is no carve out in the negligent homicide law that says running over jaywalkers is legal. The prosecutor could have looked at:

* whether Uber was warned that having one operator instead of two would be unsafe (they were) * whether having one operator to monitor the road and the software is inherently unsafe * whether operators were told that the system would never emergency brake and never warn about needing to brake * whether Uber reviewed footage to ensure operators were paying attention, whether they knew that operators weren't paying attention etc

I hope all of this will be in the NTSB report.

> The driver was watching TV on her phone. She might still be charged.

Has that been conclusively established, rather than they were watching the central console where the automation they were also supposed to oversee is located?

Yes, it was in the official police report. The driver was watching "The Voice" on Hulu at the time, and "...looked up just 0.5 seconds before the crash, after keeping her head down for 5.3 seconds while going 44 miles an hour".

I don't know AZ law, but doesn't the driver have a duty to use a reasonable level of caution to avoid hurting others, regardless of who has the right of way? For example, looking at the road while driving might be within reasonable levels of caution.

Where I'm at (Sweden, so far away and possibly a very different legal situation), right of way is not an excuse for negligent driving. You don't get to run people over unpunished just because you have the right of way if it could have been prevented by paying proper attention.

> Drivers do not have a duty of care to yield to pedestrians who do not have the right of way, although drivers have other duties.

That's not what's written there at all.

It's odd to me that the video would be dark because of a poor quality camera. Why would Uber skimp on that?

My impression was that the main cameras system was too complex for the police to be able to quickly extract video at the scene. However, the car had a redundant consumer-grade dashcam that the police were able to take video from.

This prove that the area is well lit currently, but not necessarily that it was very well lit the day and hour of the accident.

It was a very notorious case. I would expect authorities silently improving the lit in the area and taking other correction measures in all those months.

I wish I had a link for you, but there was a youtube video of the same section of road take just a day (I think) after the accident. That video was well lit.

The Ars article above is from the week after the accident.

I actually used to bike that road all the time 3-4 years ago. The bridge section before it could be disturbingly dark, but the point where she was crossing was decently lit, as I recall. It was also somewhere I would never consider crossing, mostly because there was a crosswalk just up ahead.

Still, if I absolutely had to it was definitely an intersection to watch carefully for cars when crossing, but one with fairly long sightlines for that.

Is AZ a significant outlier in this regard? Is AZ different enough from say, California, that Uber was incentivized to test in AZ as a way to mitigate liability exposure in the event of fatal crash?

I was always curious why they were testing there as opposed to somewhere closer to HQ.

Uber was testing in multiple locations, but they originally moved it to Arizona due to less regulation than they were encountering in California: https://venturebeat.com/2016/12/22/ubers-self-driving-cars-f...

I don't know the history too well, but remember Uber refusing to get some licenses in California and opting for AZ which doesn't require the same level of licensing. Others should correct me, but that was the buzz at the time.

I always thought AZ had these laws because historically a lot of retired people are there who have slower reaction times and aren't going to be doing any stunt driving swerves, and it's generally not a pedestrian or bike friendly place. They prioritize cars. When I saw the protestors a while back pouring onto freeways in the east bay, I remember thinking no way would anyone try that in AZ -- the cars would just run right over them.

> stunt driving swerves

You've put a lot of emphasis on how dangerous and crazy it would be to have not run over the pedestrian, but in other jurisdictions learning how to do an emergency stop is required for all drivers before being allowed on the road independently. There are many reasons a car might need to stop unexpectedly (including the possibility of cars in front behaving erratically) so stopping along with maintaining safe separation between cars are basic driving skills expected in many countries. From your description AZ might be an outlier though.

> but in other jurisdictions learning how to do an emergency stop is required for all drivers before being allowed on the road independently.

People learn to parallel park too. Those that don't do it regularly tend to be pretty terrible at it. The fact that people had to check a particular box at the time they got their licenses says little about what they can be expected to be able to do years after that.

AZ has very liberal regulation on self-driving vehicles (and cars in general), but IIRC the main factor is they have almost no oversight while California has some auditing and reporting requirements, including disengagement rates, and the CA DMV releases annual reports.

I'm not sure about AZ laws, but it's starting to seem a lot like even Texas, where I live, has laws that favor the pedestrian more than Arizona does. I may be misinformed, but that is pretty damning if true, considering Texas' population distribution and the necessity of driving. As a permanent pedestrian, though, my day-to-day concern is more on whether or not people actually follow the local driving laws (they don't, a significant amount of the time) rather than the laws themselves.

I think you're right. As someone who doesn't own a car at all, I don't think the issue for accident avoidance is preventative effect of these types of laws, because even in states with pro-pedestrian laws, it's still very rare for drivers to be charged. What actually makes a difference is proper design of crosswalks, speed limits -- things like that.

Yeah. Not to mention the really nice dinners Uber lawyers took the prosecutor to to discuss the case.

[Citation Needed]

I can of course see how they are not criminally liable under Arizona law where the Automotive laws are pertinent... but, what if someone turns off safety features of a device and then someone is killed if those safety features had been on - would that be manslaughter under Arizona law?

They would have a hard time testing their cars like this a lot of other places. No wonder they picked Arizona. Around here (I'm in Scandinavia) if a car hits something other than another car it is always the drivers fault, jaywalking pedestrians or not. It is the responsibility of car drivers not to hit softer things - full stop.

This and traffic in Paris are the real tests of self driving cars.

Yeah, same in the Netherlands, the car driver is at best 50% at fault, typically 100%.

I do want to see self driving cars in Amsterdam though, I think they'd just shut down and have a sob after not being able to move due to being swarmed with bikes well within any sort of safety area.

There was a recent video by a Waymo person who dealt with understanding the world. He went into great detail about how to observe things. One thing he mentioned is that the car also has the goal of getting to the destination. There was an example of a pretty packed school area. The safest thing to do might be to completely stop. The car however slowed down and tried to make its intentions clear. Unfortunately I cannot find the video.. it also went into detail of how to spot that something unexpected might happen.

> It is the responsibility of car drivers not to hit softer things - full stop.

I get the feeling a lot of people on HN have never actually driven a vehicle or at least haven't in recent memory. If you're driving through a street at a perfectly safe and reasonable speed a sufficiently stupid pedestrian is still perfectly capable of creating a situation that causes you to hit them. Nothing is foolproof. I'm sure I could get a truck driver who's doing nothing wrong to back over me if I behaved ignorantly enough.

Absolutes like "always the driver's faults are just stupid, ignorant, poorly thought out, whatever you want to call it but they sure aren't good.

It is every road user's ultimate responsibility to behave in a manner such that nobody else is forced to take emergency action to avoid them. The reason we have specific rules is so that people behave predictably (e.g. stopping at stop signs) making this easier.

I say this as someone who walks a couple miles through the city every day.

In countries like China where the driver has more fault than a pedestrian and many aren’t following the rules (drivers and pedestrians), what winds up happening is that overall cruise speeds are greatly reduced and everyone is constantly on edge. I joke that the cars never really stop for you but everyone just slowly swerved out of each other’s way.

It can work, it’s just less efficient.

> The car had the right of way, it was not acting maliciously, and she walked right in front of it. According to AZ law, asking the car to make what might be a dangerous last second swerve into another lane or slamming on the breaks to avoid hitting her is not a legal obligation as these are unsafe maneuvers

My mother has dementia and until she was put into a secure home she had a habit of wandering. On three occasions she was picked up by police after midnight crossing the road.

Your defense of laws protecting drivers is fine, but doesn't cover outliers like my mother who would have been knocked down at least, killed at worst. Is that ok for laws to protect the driver/AI driver from running over pedestrians who have lost their personal safety faculties?

> but doesn't cover outliers like my mother who would have been knocked down at least, killed at worst

So would it be ok if that self driving car did see the pedestrian walking down the freeway, tried to avoid the collision, looses control and end up running into the car coming the other way, killing all the passengers in both cars?

The car was traveling at high speed down a dual carriage way (i.e. freeway) in the middle of the night.

In a situation like that any sudden reaction by the on board computer is never going to end well.

So in Arizona after running over and killing a careless pedestrian, human drivers just drive on at the same speed with all care and attention and don't slam on the brakes in fear and panic and usually spin?

The car was traveling at 44mph. A speed reduction to even 14mph would have probably made the collision survivable. If your argument is that the car was unable to drop 30mph over several seconds without causing an accident or swerving into opposing traffic, I don't know - maybe the self driving isn't there and shouldn't be on the road at all.

That's a valid point to make in the general case, for times when you have no good option, and hitting the pedestrian in the crosswalk may be the lesser evil.

But in this case, hard breaking would have been plenty fine and with little downside risk (beyond the general problem of excessive unnecessary braking). The car's computer was aware of an obstacle in sufficient time, as would a human driver have been.

> hitting the pedestrian in the crosswalk may be the lesser evil.

I would not go so far as to say that, only because if that had been the situation then the car and it's onboard system would have to take the all the blame.

But in this case, based on the footage I saw the car was travelling down a freeway at high speed and at night.

Now some of the blame might be attributably to the car, but the real cause of the accident was the pedestrian walking down a freeway, so it seems clear most of the blame hast to be attributed to the pedestrian.

> But in this case, hard breaking would have been plenty fine and with little downside risk

Except to any cars that might be following behind that then end up ploughing into the back of that hard breaking car.

No offense but the problem here is your mother walking right out in the street. A car should absolutely not be liable for accidentally hitting someone or having to risk their own life just because someone confused and unobservant walks out right in front of it.

> someone confused and unobservant walks out right in front of it

You can spot such situations pretty easily if you pay attention.

> or having to risk their own life

Reducing speed and not hitting someone cannot be compared to risking your own life. You're turning the argument around and adding unneeded emotions to this.

I would imagine that Arizona law is at least part of the reason why these trials were being done there, and not somewhere else.

Man it must be hell to drive interstate and remember all the differences

The problem is that a false positive causing emergency braking is also dangerous. So you'd be trading one instance of the car not emergency braking when it should against however many instances of it dangerously doing so when it shouldn't. That isn't an obvious call even before you consider that the human driver can cause the car to brake when it should but can't cause it to not break when it shouldn't.

The only real solution is to get the false positive rate down. Which is why they still have human drivers until they do.

You hit the nail on the head. People often jump to reactions without looking at bigger picture.

It is entirely likely that Uber engineers evaluated this scenario, and (correctly) decided that it was safer overall to turn off reactionary breaking - and have that function performed by a human in the driver seat. If they hadn't, these cars might instead have been brake-checking people at 100x the rate.

It's easy to say things like "Uber should have waited until better safety features were available", such as the eye tracking suggestions mentioned elsewhere in this thread. But features like that take time - especially if they're development-only features that would have no place in the final product. Every additional safety feature pushes FSD deployment back.

Globally, 1.25 million people die from car accidents annually. That's over 3,000 people per day. For every day that you delay mass adoption of FSD, you are accruing massive amounts of fatalities that could have been avoided.

FSD does not have to be perfect, it's development will cost innocent lives - but if you're optimizing for minimal loss of life, it's the correct thing to do. Reactionary policies do the opposite of what you intend them to do - they cost more lives in the long run.

Uber was nowhere near ready for live tests. It's that simple. Relying on human intervention for split-second decisions where no actions are necessary for long stretches of time is pure insanity.

This accident alone demonstrates just how flawed the system is. This wasn't even an emergency situation to start.

What's your criterion of being ready for live tests? Under what conditions is loss of innocent life okay?

A self-driving car is ready for live testing under the same conditions in which it could pass a driver's test, which is the minimum standard for anyone to be allowed to drive legally on public roads.

My road test consisted of driving out of a dead mall parking lot (careful to obey the posted 15 MPH speed limit -- this is actually the hardest part of the test!) turn right onto a street, turn left onto a residential cul-de-sac, make a three-point-turn, and return to the mall parking lot.

These cars could absolutely pass that test.

>Relying on human intervention for split-second decisions where no actions are necessary for long stretches of time is pure insanity.

This is literally how all human driving works right now.

Seriously, look around at other drivers while you're on a freeway or interstate sometime.

They're not driving, they're singing along with the radio, or shaving, or putting on makeup, or sending text messages.

They're not Luftwaffe aces with eagle eyes and steely nerves monitoring the fuel mix and oil pressure while scanning the skies for the silvery glint of the sun off the wings of a P-51 that may herald their last few moments on earth. They're bored people doing their boring commute and even when they have their hands on the wheel they're not really paying attention.

They still require some attention all the time. A sense of responsibility and for all our faults we are att least selective about when we aren't paying attention.

Yes, humans suck. But that's still orders of magnitudes better than uber.

Which weights off different peoples lives. I dont see how developing a technology factoring in unrelated people getting killed could ever get past a ethics board. Especially if the victims are unrelated to the technology to start with and didnt volunteer to be test subjects.

People getting killed by drunk drivers didn't volunteer for it either. It doesn't matter how you spin it, the metric that matters (there is nuance, but put bluntly) is "How many innocent lives are lost in traffic related accidents from now until [year X]?".

If, given two scenarios:

1. We develop FSD very carefully. 0 lives are lost during FSD development. Ten million lives are lost by the time that FSD sees 100% adoption.

2. We develop FSD less carefully. 100,000 lives are lost during FSD development due to suboptimal performance before it is perfected. Five million lives are lost due to human drivers by the time FSD sees 100% adoption.

Would you really choose the former option? If your ethics board refuses option 2 in favor of option 1, it is they that are mistaken.

The difficulty is, you don't know the 100 thousand number is lower than the 10-5 million number in advance. An estimate can be made, but humans have a not unreasonable status quo bias in case like that. Come to think of it, status quo bias on risk is the opposite of the planning fallacy.

That is, without proof the new way will be safer in future by a specific proportion, how many excess deaths can the ethics committee sign off?

(You can make a similar, better founded argument for mass train transit. It is about 10 times safer than driving per passenger mile. So, reductions in safety that moves people from car commuting to train commuting (I imagine safety is expensive and cheaper transit would be more popular) might be worth it. No-one seems to advocate them though.)

> 2. We develop FSD less carefully. 100,000 lives are lost during FSD development due to suboptimal performance before it is perfected. Five million lives are lost due to human drivers by the time FSD sees 100% adoption.

How can you know the future in this case, I would not believe Tesla/Uber (the others) predictions , "trust us, let us kill 1 million people in the next 10 years but after that the number will go down to 100, trust us, we software developers are very reliable at predicting things and our mathematical skill are so great we can be sure even when we use NN that are unpredictable. And if we fail then the next startup will appear, promise the same thing and you have no choice then to give them also a check for 1 million lives to not discriminate between startups.

>People getting killed by drunk drivers didn't volunteer for it either.

Its not a numbers game. Those are likely two different sets of people. You are saving some people by calculating in, that others get killed instead. It is different to for example, using an experimental treatment option on terminal patients. That would be, saving some of the dying patients but some might die regardless. The scenario is much more similar to the trolley problems cousin the transplant problem. Having someone else, uninvolved killed to save a larger group of people.

Seeing as the automatic emergency breaks were disabled, it seems the product wasnt good enough to detect false positives and was rushed by cutting the safety measure, accepting that putting it in live traffic the car would likely kill people unrelated to the research. It was a trade off between slower development and killing people.

I dont think you will find any ethics board that will green light you killing uninvolved people to save how ever many people. Of those i can think of who did that, quite a few ended up being wanted for crimes against humanity. We have ethics boards for a reason. Without them the ends can quickly justify the means.

Globally 1.25 million people die from car accidents annually, but a vanishingly tiny proportion of them would have been saved by the proprietary tech of a taxi company that doesn't even operate in many of their regions, even assuming the technology is ultimately capable of delivering a net improvement on commercial driver fatal accident rates.

Uber isn't a philanthropic research endeavour, and it can't rationalise killing people based on lives it couldn't or wouldn't have saved in the event of its technology actually working. The reason they put tech that reportedly hit things every 15k miles and had a near miss every 100 miles on the road as soon as possible has nothing to do with optimizing for minimal loss of life and everything to do with optimizing for a unicorn valuation.

False positive emergency braking seems less dangerous than running over obstacles. Wouldn't the expected accident be a car crashing into the Uber's back? If the false positive rate is so high that the number of such accidents outweighs the lesser risk, then perhaps Uber shouldn't let the cars drive autonomously with that software.

People can die from rear-end collisions as well, though. It's possible that Uber did the calculations based on the rates of each occurrence and concluded it would be more likely for a fatality to occur from a false positive emergency brake than from running over an obstacle.

Personally, I doubt that they really considered that; I believe that their software was rushed, flawed, and totally not ready for real-world driving. But I don't think it's a certainty that false positive emergency braking is always better than running over obstacles, especially if "running over obstacles" is determined to be a sufficiently rare event.

False positives are only a tradeoff when someone is following. The single most important advantage that driving computers have over humans to make up for their inherent deficiencies is that they don't have to split focus. The computer has no need to check the rear view mirror before going all in on the brakes, it should at any time know the current risk of a rear collision and tune thresholds accordingly.

> The problem is that a false positive causing emergency braking is also dangerous. So you'd be trading one instance of the car not emergency braking when it should against however many instances of it dangerously doing so when it shouldn't. That isn't an obvious call even before you consider that the human driver can cause the car to brake when it should but can't cause it to not break when it shouldn't.

Yes, building a self-driving car is hard. But if you don't even have emergency braking working, then your technology is too underdeveloped to drive on public roads. Keep trying on the test track.

Not having an emergency braking system should not have been a no no in this particular test. It was a prototype with probably many parts not working properly, that's why there was a human to check what was happening and act in case of issues. Here the driver is clearly at fault given that he was willfully not paying attention at the road.

There is probably some degree of responsibility on the side of Uber however:

* how this guy was recruted?

* was he trained and evaluated?

* is this kind of behavior quite common in test drives? if so, was it addressed?

* did Uber put in place at least a basic system to keep the driver engaged (like a switch you have to press every 30 seconds)?

Given the specifics I hope this ruling will not serve as a precedent, it's a bit too special.

Lastly, it raises another question: Car automation is unlikely to become perfect within the next few years, and even if it is, the legal responsibilities and liabilities cause issues. So it's likely a driver will be kept in the loop, at least for a decade or two. Given this statement, how do we design automation that keeps the driver in the loop? The driver must be aware of the car surroundings at all times so he can take over in case of an emergency. Consequently the automation must be designed to maintain him active, and naturally on alert (like a "manual" driver), an automation that is like "yeah, do nothing... humm, oops I don't know what I put myself into, please take over, you have 2 seconds to analyze the situation and take the correct actions or someone will die, kiss, goodbye" is not the correct answer.

>Not having an emergency braking system should not have been a no no in this particular test

The car did have a working emergency braking system that the car was born with. Uber disabled that one too.

No you don't understand, Uber didn't have self driving cars on the road, they had autonomous test vehicles. There was never any reasonable expectation that the test vehicle would perform perfectly in all situations, that's why there is a human monitor in the drivers seat. That human monitor is the final failsafe, it's the same with Tesla's Autopilot, and the same with any conventional car. When Rafaela Vasquez volunteered to drive the vehicle, she accepted responsibility for the vehicle. She was watching a television show when the accident occurred.

> " there is a human monitor in the drivers seat"

Any system that requires a human to sit there bored for hours, waiting for seconds of action at any moment, is fundamentally flawed. The bipedal ape component is being used out of spec and the blame for that lays with the engineers who designed the system.

I'm not so sure. If the car was driverless then yes. But they put a human operator precisely to handle these sort of situations.

They could put eye tracking system in place to make sure the operator pays attention! If the driver doesn't pay attention the car should slowly stop.

Expecting people to always do the right thing is a recipe for disaster.

Eye tracking or not, I suspect humans aren't cut out well for that sort of task. At least not without some pretty specialized recurring training.

Waymo is claiming things like 30,000 miles between needed interventions. Even if that's 100x higher than reality, that's hours of watching and boredom waiting for a few seconds of action.

> They could put eye tracking system in place to make sure the operator pays attention! If the driver doesn't pay attention the car should slowly stop.

Except where normally there is a safety driver focusing on driving and an oversight passenger tasked with looking at and checking the automation's feedback, for cost reasons Uber had given both tasks to the "safety driver", forcing the driver to often stop paying attention to the road to check automation warnings & classify events.

This is exactly backwards from reasonable.

Ultimately systems will require no intervention for very long periods of time punctuated by very quick precise actions that most people are terrible at in the best possible scenario but that computers will be very very good at.

If you task the user with doing the right thing in 5 seconds time after thousands of hours of inactivity a sizable portion will react after the entire affair has come and gone and those are the lucky ones. The remainder will act 3 seconds into the crisis and screw everything up.

What you are describing is the worst possible combination of man and machine wherein you let the machine handle the part the human could easily handle and you task the human with the part they are worst at to ensure maximum carnage.

I think the main problem is one of social acceptance, that we will see accidents that could easily be prevented by a human operator but at the same time drastically reduce the amount of accidents where from a human perspective, only pure luck or a reflex has saved people before.

I would agree with you if she were ostensibly paying attention and just didn't react in time. But she was watching a TV show on her phone. This is not the situation you're describing.

As opposed to pretending to pay attention knowing attention would wander without engagement?

The prosecutor has no doubt thought long and hard, and there is no evidence of a bad call here. We don't even have the benefit of hindsight yet.

The driver being criminally liable is a more powerful incentive. People will not buy & drive vehicles that have a reputation for getting their drivers criminally prosecuted. And the government can still regulate for high safety standards on self driving vehicles independent of criminal liability.

> The driver being criminally liable is a more powerful incentive.

Too powerful. The driver should not be punished for correctly using a legally purchased vehicle. We don't apply that kind of standard anywhere else, and for good reason.

Plus, people would refuse to buy self-driving cars even if they were on average safer than human-driven cars because

1. Everyone thinks they're an above average driver 2. Illusion of control

So your proposal would cost lives in the future when self-driving cars actually are safer.

> for correctly using a legally purchased vehicle

The eyes of the driver in this case were on the phone, not the road. That's hardly "correctly using".

The parent was quite clearly in favour of punishing the driver even when they were operating everything correctly, for the purpose of disincentivising unsafe code at the point of purchase. I'm saying that this is a bad proposal.

I have special insight into what the poster was in favour of. He thought that the prosecutor has a better idea than the rest of us, spent some time thinking about it, and has probably made a good decision. As smilekzs notes it is unlikely that the driver was operating everything correctly.

You may note he explicitly said "the government can still regulate for high safety standards [independently of the actions of the prosecutor]".

> The driver should not be punished for correctly using a legally purchased vehicle.

So where's the market incentive to bother making the piloting algorithms less killy? Consumers will choose cars that prioritise speed and the safety of the occupants, with no regard for the safety of bystanders.

This all seems to stem from the assumption that the computer just does whatever it wants (shrug), and isn't merely a tool being used by a human programmer who's supposed to be in control of it. That way lies [killbot hellscape](https://xkcd.com/1613/).

By punishing the company, not the users.

Presumably liability for the company in cases which the computer is actually the legally responsible party?

People will indeed logically and correctly choose devices which wont opt to kill them. In fact they wont buy anything that will ever opt to deliberately kill them in any circumstances.

The challenge is to ensure the machine has a survivable alternative in the broadest possible range of scenarios to avoid making bad choices.

I had the exact same thought.

>> the system was set up to not activate emergency braking when under computer control.

So the parts that mattered the most where disabled? I bet that even the human operator did not know this.

I'm wondering if anyone else responding to you clicked on your profile and knows what you do or has equivalent experience from which to comment.

I think even absent that, its still useful to have the trial, just to get all the facts out, to have the conversation.

No one knows what would have happened had this gone to trial, it would be better to find out sooner rather than later.

I don't disagree, but I've heard these self driving cars are extremely over cautious, and tend to hold up traffic because they're waiting for a safe time to turn across a lane, for example. I imagine this level of caution would needlessly trigger the emergency brake quite often, and likely cause more accidents and injuries than it prevents.

So then shouldn't the answer be not to disable the e-brake, but to conclude that the tech is not ready for testing in public where pedestrians' lives are at stake?

Obviously yes but that wouldn't be moving fast and breaking things. If companies aren't allowed to do that, then their SV competitive advantage evaporates.

> Putting a “self driving” car out there that doesn’t have emergency braking enabled ...

That's why they put in emergency breaking device with set of eyes, hands legs .... and movie streaming device apparently?

Negligence in that case is not bricking employees phone before putting him in the car. But nobody expects this from companies that hire drivers.

What definition would you use for “perfected” emergency braking—“As-good” as the median driver?

I suppose perfect emergency braking avoids hitting all objects that are visible to the sensors sufficiently early, and hits objects with the minimum possible speed if they become visible too late, while never having a false positive.

  if(obstacleDetected()) {
      // emergencyBrake(); //TODO: fix
  }

  if(obstacleDetected()) {
      // emergencyBrake(); //TODO: fix
  }

  /* 
   VolvoEmergencyBrake();  
     Disabled. We don't need this.
  */

Volvo would definitely enable the emergency brake, but only during regulatory tests.

The state also made the wrong call in allowing Uber to operate such a vehicle on public roads and are clearly now trying to distance themselves from that decision by letting Uber off the hook. Both are guilty of criminal negligence but it's not like the state is going to prosecute itself.

> Working emergency braking should be the first thing perfected, before the computer gets to control the car.

Agreed, but that makes it the lawmaker's criminal negligence, not Tesla's.

Reply on news.ycombinator.com