Hacker News Reader

Comment by shajznnckfke | Hacker News Reader

I think it’s useful not to label this person’s job “driver”, but instead “fall guy”. Their job is to sit around all day doing absolutely nothing. But in the rare event where the car fails, they need to suddenly become alert and fix it, or take the blame for the machine’s failure. I think it’s less accurate to describe this transaction as a form of labor, but instead as an indemnity, an assumption of liability, paid for via a wage.

For train conductors with a similar challenge, a solution has been invented requiring them to pass various visual attention challenges that detect if they aren’t alert. Such systems weren’t present here. The system wasn’t designed to work - it was designed to protect Uber by shifting the blame for failure.

View on HN

It is hard to imagine they weren’t aware that their job involved doing nothing (but pay attention to the road).

I’d be inclined to give them a pass if they were looking at the road and just inattentive (because that’s expected, like you say), but I find it hard to sympathize with someone watching a show on their phone, explicitly ignoring their one job.

that's the fundamental attribution error again, attributing poor judgement/negligence to the driver, who's the visible actor, rather than the inherent systemic flaws designed by other unseen actors who hold greater responsibility.

It's an error to attribute negligence to the safety driver of an experimental car who is watching a video rather than watching the road?

Obviously the system should improve to make errors less likely, but that doesn't absolve anyone of guilt in my view.

We can discuss morality endlessly. We have (collectively) agreed to use the law to assign guilt and mete judgement -- this the court case will bear out eventually. What's left is the problem itself abstracted from the particulars of this case. And the best approach to solving this kind of problem, as evidenced by the aviation industry's miraculous safety record, is to consider the concept of assigning blame as out of scope because it shouldn't be the basis for our safety precautions. In this context, blame is something that distracts you from the real cause and can prevent you from looking more closely. Blame is a kind of 'bottom' argument which in practice is wielded to dismiss alternative perspectives and refuse to integrate their ideas.

that's a good point. i'd add that even in the legal system, assigning blame is highly problematic because decisions are overwhelmingly pressured into being purely binary findings, whereas blame is nearly always shared by multiple entities. lawsuits would be much better if the "win-lose" dichotomy was entirely abolished in all its forms.

in this case, the backup driver should shoulder some blame, but not the majority of it, and commensurately, should not face any of the harsh penalties associated with manslaughter.

air lines have reached their current safety precisely because they consider the whole system as a whole and not just focus on the error.

the wet bags behind the autopilot get bored, distracted, complacent, ill or reckless due a moltitude of factors some of which outside their control

it is well known at this point the human modes of failure, and that there was basically no mitigation is a reasonability of the system designer

But holding individuals accountable for their actions.

A better system would have spotted this person was watching their phone not the road, but it would still have held them accountable and (at minimum) fired them!

This flaw is perfectly predictable, given other experience in industrial safety, so why was the better system not built?

A totally fair point, Uber needs to be held to account for their systemic failings.

That doesn't mean that an employee being clearly negligent doesn't also need to be held accountable.

Them being held accountable is one part of a good system.

I think that might have passed if he only listened to the radio, a good enough way to relieve boredom. But streaming video ? That's a punishable offense while driving.

Also, many people work at really boring jobs. If serious errors happens while you watch netflix, you'll get fired, for sure.

But again, sometimes listening to music is permissable.

So how did this worker get through their screening/interview process? When hiring for this job which criteria did they focus on? Why even allow the driver to carry their screen around? Massive failure, I think there is too much FSD coolaid in the valley springs. I am somewhat surprised we had the power to stop/pause them after this reported unfortunate event. Anyone working on FSD today should focus entirely on mass transit vehicles mkay tnx

Would you say the same thing for a chauffeur or truck driver who killed someone because they were using their phone while driving?

That somebody driving a truck or car in the normal way is doing more work than a person supervising a self driving car is the whole point of self-driving cars. They take all of the activity out of driving, but none of the tedium.

I don't think it's realistic to expect people to function well under those conditions.

When some people learn to drive they can take courses with instructors who sit in the passenger seat and have a brake pedal. When the student driver runs over and kills a pedestrian while the instructor is on their phone, is it the instructor or the student who should face liability? Why not both?

I'm fairly sure the instructor is at least partly liable.

In England certainly the person supervising a learner driver is subject to the same laws about alcohol, mobile phone use, etc as someone actually driving- and can be convicted on a charge of aiding and abetting any crime the learner driver commits. This is the case regardless of whether they are a professional driving instructor with their own brake pedal, or a parent supervising their child driving the family car.

In Portugal there is an interesting law regarding this: The instructor/examiner is always liable for an accident during driving lessons/exams, except if the accident resulted from an action where the learner disobeyed a direct order by the instructor. In that case the learner takes responsibility.

no, because the system is not adversarial in that case, even though the actions of the driver, and the potential tragic results, might be similar.

Saying it's an attribution error is circular. It's a conclusion, not a premise.

And I think it's wrong. If my car is on cruise control, that doesn't mean I can stop worrying about my speed. You can't always just blame the company because it's the bigger entity.

I think we can all agree that many murderers end up committing their crimes partly because of a system that gives rise to poverty and the knock on effects of poverty on humans. However, I think most folks also agree that incarceration is an important factor in discouraging the commission of more murders. Not everyone does but even the most lenient countries tend not to let the perp off scott free.

Which is all to say that "the system made me do it" really doesn't fly as an excuse for felonious behavior.

If deterrence didn't work, prison would be pointless cruelty. Paying sustained attention while nothing happens for hours is simply not something people are generally suited for. You can't deterrence them into it.

sure, but that extreme isn't helpful just as making the FAE isn't helpful, since, as others have astutely pointed out, the system was designed, whether through negligence, malevolence or other intentions, to put the driver in a (systemic) no-win situation.

for instance, in the 737 max crashes, the pilots made errors, but the principal blame lies with boeing and their egregious systemic design flaws.

Stepping back, I also find it hard to sympathize with someone watching a show on their phone. But there are two parties to this crime, and only one of them is being charged. Uber is responsible for creating this situation - I doubt this "backup driver" watches videos while driving her own car - yet has managed to escape liability. Hence why it's appropriate to describe her as a "fall gal".

Yeah, the problem isn’t that they weren’t paying attention, or that they weren’t even trying, but that they where intentionally NOT paying attention.

That said, I don’t think this gets Uber off the hook. If I were on a jury, I’d likely say Uber was guilty of manslaughter (barring a real look at the evidence).

> That said, I don’t think this gets Uber off the hook

There is exceptional negligence on their part. The fact that vehicle was just blithely travelling the posted speed limit, even at night, even though the speed violated it's assured clear ahead ability, is a damning point. The vehicle will operate unsafely in it's default configuration.

And for that reason you have a driver there who should have taken over.

Did this person know he should have taken over when the car was respecting the speedlimit but still going faster than it should have been going? I go the speedlimit at night as well. But here in Belgium where there's so much street-lighting it might as well be day

How much slower was the car supposed to go for it's sensors and was it all the guys responsibility even if he knew Uber apparently disabled the auto-braking due to issues and didn't pause roadtests?

> Did this person know he should have taken over when the car was respecting the speedlimit but still going faster than it should have been going?

Yeah, and how was the driver even supposed to know how fast the car should have been driving? I know how fast I should driving because I know how far I can see, but the car had LIDAR / Radar / night vision / etc. If the speed limit is 45MPH and the car is going 45MPH, what reason did the driver have to think that the car couldn't see? Either the driver can rely on the car to "see" or they can't.

Honestly if I’d been given the job of babysitting a self driving car, the same thing could have easily happened to me. I’d get bored out of my mind and unintentionally pull out my phone & start browsing Reddit before I caught myself. It’s a shit, soul sucking job to actively do nothing.

Let's say you were working as a lifeguard at a swimming pool. Would you also unintentionally pull out your phone after a couple of hours?

If you did and someone drowned in the mean while, there would be no debate about who was liable.

Through, lifeguards at a swimming pool in are typically not expected to be attentive for hours. They rotate and switch.

That's the difference between a system designed to keep people alive and a system designed to do the least possible work to avoid liability.

Even if they’re not, you wouldn’t allow them watching a video on their phone

And they're not allowed, and (as other commenters say elsewhere in the thread) there's another person on the floor whose job is to ensure lifeguards' attention doesn't drift off. That's how you design safe systems: defense in depth.

I know that cars is the most probable cause of sudden death/injury so I wouldn't do that. How can you 'unintentionally' pull out a phone?

I'd probably be looking at the road all the time, but just freeze at that moment where I'd need to take over.

> How can you 'unintentionally' pull out a phone?

Probably more like "absent mindedly" pull out a phone.

Absent mindedly is closer. Whatever you call reaching for something out of habit with thinking about it.

Habitually

You can listen to audiobooks -- if your job is to be a backup driver keep your eyes on the road.

But you did not work there. There are people who would very easily do this job. If she knew she couldn't concentrate, she shouldn't have taken the job and the risks involved.

Employees taking jobs they're unqualified for is a known problem. Reasons might include ignorance, hubris, financial need, greed. Employers need oversight systems to ensure employees are performing their jobs adequately, especially in safety critical roles.

We know that you can pay someone as much as you want but you can’t get them to be vigilant for a low probability event. Except, as you say, by giving them some continuous skin in the game, even if it’s illusory.

This person was being paid to pay attention, which is hard enough to get right. What happens when the person owns the car, and is just commuting? And there are millions of them, not just one person?

Mayhem, if it’s a variant on this system instead of something much more sophisticated.

> you can’t get them to be vigilant for a low probability event.

How about starting by simply removing their phone? Isn't that easy? The driver was looking at her phone at the time when she was supposed to be looking at the road. This is a simple violation. The phone should have been banned and the driver fired after a first violation.

I am pretty sure that while what you say might be hard to do for some people, there are individuals out there for whom sitting and looking at the road for hours would not be such a big problem. If she couldn't handle the job, she shouldn't have worked there.

> I am pretty sure that while what you say might be hard to do for some people, there are individuals out there for whom sitting and looking at the road for hours would not be such a big problem

Those people are known as drivers and they’re doing more than just looking at the road. The driving/feedback from the controls is keeping them engaged.

That's all fine, but we also have decades of experience designing systems for exactly this - they are called train drivers. Driving a train can be the most boring job in the universe, in ways that driving a truck isn't - there is very little feedback from the controls, you can be going for hours in a straight line without any need for input or change of speed - so locomotive controls are designed to require constant positive input to keep the driver engaged. This should be the same - the system should require the backup driver to keep confirming/selecting something on the dash, even if it's inconsequential to the operation of the system.

> I think it’s useful not to label this person’s job “driver”, but instead “fall guy”.

Or "sacrificial part" -- designed to break first to protect the rest of the system. The NTSB report (https://www.ntsb.gov/investigations/AccidentReports/Reports/...) is an interesting read, by the way.

Corporate equivalent of ablative armor then?

This is exactly right, and probably the way that liability will shake out with self-driving vehicles in general since it's in the manufacturer's best interest for it to be that way.

"Here's your self-driving car with a 20 page EULA/ToS. Oh, but if it's in self-driving mode and something goes wrong it's not our fault, you must respond (within seconds) and fix it yourself."

Their job is definitely not to sit around and do nothing. Self-driving cars regularly disengage from autonomy (e.g. emergency vehicle, construction zone, erratic bicyclist) and the safety driver needs to be ready at all times. Safety drivers also undergo extensive training around this.

I believe the normal behavior of self-driving cars in these situations is for the car to brake to avoid hitting the obstacle, so the safety driver has ample time to take over and navigate the situation. Didn’t Uber disable the auto-braking because it was oversensitive, choosing not to pause road tests until that issue was fixed?

Uber disabled the OEM (Volvo) auto-braking but not the Uber braking algorithms. IMO, that’s largely a red herring as they could have equally well chosen to base their platform on a car that didn’t originally have a factory auto-braking system.

No it is not; there is no legislation in place that shifts the burden onto technology. The technology is assistive, and the driver is the end responsible.

Would you blame an airplane's autopilot if it crashes? We still have two pilots even though 99% of the time the thing is on autopilot.

She's a test driver, she did not pay attention, and she killed someone. At best, her employer should help and compensate her, given how it was a workplace, on-the-clock accident.

> Would you blame an airplane's autopilot if it crashes?

If the plane crashed due to an autopilot error, yes, absolutely.

This is exactly right.

This job is a wage against a (very nasty) lottery ticket, to sit there to absorb the legal fallout for decisions made far away.

Reply on news.ycombinator.com