Hacker News Reader

Comment by s1mon | Hacker News Reader

According to Cruise's X(Twitter) account [0], the suspension seems to be related to an incident in which a human driver hit-and-run propelled a pedestrian in front of a Cruise AV and the Cruise vehicle couldn't avoid hitting the person. When that incident was first reported [1], it seemed like a situation where if it was a human driver in the Cruise vehicle, they wouldn't be found to be at any fault. However given this suspension, I wonder if there's more to it than that.

[0] https://twitter.com/Cruise/status/1716877217995894934

[1] https://www.sfchronicle.com/bayarea/article/woman-run-autono...

View on HN

The article you link to says there was more to it! The Cruise vehicle, after running over the woman, stopped on top of her with the rear axle/tires pinning her to the ground, trapping her there. And it just... sat there. I hesitate to believe that a human driver would do something like that.

You could, however, argue that staying in place was the right thing to do; continuing to drive to get the woman out from under the car may have injured her further. A follow-up article[0] suggests just that, and emergency crews ended up lifting the car off of her instead of trying to tow it.

If this is indeed why their permit was suspended, it seems like a questionable reason. Some of the other dumb things Cruise cars have done seem much more obviously egregiously bad. But in this case the crash may have been unavoidable (by human or robot), and avoiding injuring her further by not unpinning her may have been the right move.

[0] https://www.sfchronicle.com/sf/article/cruise-driverless-tax...

This is such a wild edge case though. So we think the vehicle should do what, keep driving, potentially causing more damage?

It's highly situation specific, and I'd expect the emergency personal to make the judgement call on how to free the person (potentially by taking control of the vehicle, or other means).

Oh, I agree. You may have loaded the page before my edit where I expanded on my first paragraph.

(I have my comments appear on a several minute delay because I know I sometimes submit a comment before I've made sure my intent was clear, but I maybe I took too long this time. It's a bad habit, and I know I should slow down, but... yeah.)

Keeping autonomous vehicles on the road will result in thousands of edge cases, and without a clear safety benefit (as opposed to Level 1 systems) its better to stop everything and reconsider.

Honestly, I don't think this tech will be able to ever be fully automated without controlling the environment as well (dedicated lanes). There's just too many variables you can't account for, and too much risk.

Is there a way to measure the (potential) safety benefits without putting these vehicles on the road?

I'm not saying that means we should just put them on the road, consequences be damned, but I'm curious if there are alternative ways to prove them without risking bystanders' lives.

> Keeping autonomous vehicles on the road will result in thousands of edge cases, and without a clear safety benefit (as opposed to Level 1 systems) its better to stop everything and reconsider.

True, but I bet either a Cruise or a Waymo would have stopped rather than hit that woman crossing the street, and they definitely would not have fled the scene like the human driver did.

I mean it did hit her (again), ran her over, then stayed on top of her. That's not a better scenario at all. A Level 1 system can still avoid hitting people.

Did you read the whole thing? She was knocked onto its path. It might be have been physically impossible to avoid hitting her at that point.

Circling back to add some info. From this[0] page:

"But the DMV also said it was not made aware that the Cruise vehicle then tried to pull over while the pedestrian was underneath it."

Kind of proves my point. Stopping completely would have been more ok. The issue the DMV took is that it then pulled over, and that cruise withheld that information.

[0]https://spectrum.ieee.org/lta-airship-faa-clearance

> I hesitate to believe that even a human driver would do something like that.

They do! https://slate.com/news-and-politics/2015/09/why-drivers-in-c...

While appalling, this seems to be because the law has been set up to incentivize that behavior, not because the drivers fail to realize they've hit someone and/or panic and freeze up after doing so. Quite the opposite.

And people getting away with it -- even with video evidence -- seems to be an indication of police corruption. Which creates further incentive to do the wrong thing here.

"Video from Cruise reviewed by the Chronicle showed the woman was in the middle of the crosswalk and that both the Cruise taxi and the other car had a green light. The Cruise applied its brakes and attempted to “minimize the impact,” a spokesperson said, but it was unable to avoid rolling over the victim."

So a pedestrian was crossing a crosswalk when they didn't have the green light. It doesn't really make sense why neither the human driver nor Cruise car slowed down in that case and kept driving normally. Both should have stopped despite having the green light.

Your quote states the Cruise vehicle did slow down.

The DMV press release specifically cites safety-related misrepresentation as part of the basis for the suspension.

It may not be the performance of the vehicle in the incident that led to the suspensions so much as Cruise's performance in the investigation of the incident (or information about Cruise's prior behavior on safety info that came to light through the investigation.)

Yeah I get the sense (and I am absolutely not surprised) that they have been heavily downplaying the safety aspects. Like publishing the accidents per miles driven but only reporting under ideal conditions. I vaguely remember coming across an HN post about this very thing, a paper saying driverless car accidents under similar conditions as humans are much worse that was previously reported or something like that, and that the companies have been massaging the data.

It's definitely unclear to me what to do if a person is trapped under a tire. Driving more may cause more injury or allow more bleeding from the existing injury. I guess a human driver would call 911 and the dispatcher may provide advice on what to do or suggest to wait for emergency services.

I think the bigger issue is the decision to move the vehicle after a collision involving a person not in a car. I've not had the misfortune to be in such a collision, but in the collisions I've been in, I would get out, and inspect the situation before taking any further action. Moving out of traffic is a good thing to do, but especially if there's a person, it's important to consider the situation before further movement. Even if it were just a vehicle to vehicle collision, I'd want to check if someone needed urgent medical attention, and acknowledge the collision with the other driver, make a plan for where to stop to discuss, and check for and clear any obviously sharp debris that may pierce tires of my vehicle or others' vehicle. A lot of that is outside the capability of an automated vehicle... but maybe human review before pulling to a safer place is a good choice especially if a person outside of a car is involved in the collision.

To be fair, it’s not like we hold human drivers this accountable when they cause crashes.

We should be suspending all bad driver’s licenses, AI and human alike. Today we give people passes for crashing while driving badly, and we should not.

While clearly the initial accident was caused by the hit-and-run driver, when the Cruise went to pull over because it detected an accident, it drop on top of the pedestrian AND STOPPED right on her - "hurled her into the path of a driverless taxi that then ran her over, stopping on top of her as she screamed in pain".

This poor woman was involved in two accidents really, and it is fair to shut down Cruise until they address this failure scenario of driving on top of a victim in road.

The first picture in the SF Chronicle article is harrowing: https://s.hdnux.com/photos/01/34/54/62/24300697/3/1100x0.jpg

If you read the follow-up[0], it's not clear that the Cruise car did the wrong thing. It may have been unavoidable for the Cruise car to avoid hitting her after she was thrown in its path, and if it had then tried to continue driving to unpin her, she could have been injured much worse, or killed (say if the axle or another tire had then crushed her chest or head).

These autonomous cars have done some truly dumb things, but I don't think this incident is so obviously bad.

Meanwhile, the human driver who actually hit her first, and then fled the scene, still hasn't been found...

[0] https://www.sfchronicle.com/bayarea/article/woman-run-autono...

But it did continue driving. It tried to pull over and ended up dragging the poor woman for 20 ft before stopping on her leg and pinning her.

And that is what is the unacceptable failure scenario. It'll be cool in the future, I guess, to have autonomous taxis. But I'm sure glad they don't test these in public where I live.

It may be related to the AI driving the vehicle continuing to drive to pull over after hitting a person:

"When the AV tried to pull over, it continued before coming to a final stop, pulling the pedestrian forward," the company said.

From the article, it sounds like the Cruise vehicle didn't just strike the pedestrian but stopped on top of her and pinned her in place. A bystander needed to recognize the incident and reassure her until emergency services arrived and mechanically removed her from below the vehicle using the jaws of life.

While not impossible, it's hard to imagine a human-driver scenario that would look quite like that, regardless of how legal accountability might resolve. Would a human driver stop their vehicle on top of the person they just struck? Would they then ignore the person they'd trapped and wait for a bystander to intervene?

I'm not suggesting that this scenario necessarily warrants the suspension, or that most of the same problems might not occur in some human-driver scenario, but it points to some of the subtle differences of human vs automated drivers that come up in exceptional situations. There are a lot of exceptional situations when driving and it very much matters how automated processes navigate them.

You're right, a human driver might not have remained in place. And as a result they would have risked further injury or death.

"“When it comes to someone pinned beneath a vehicle, the most effective way to unpin them is to lift the vehicle,” Sgt. Kathryn Winters, a spokesperson for the department, said in an interview. Were a driver to move a vehicle with a person lying there, “you run the risk of causing more injury.”"[0]

[0] https://www.sfchronicle.com/sf/article/cruise-driverless-tax...

Somehow I doubt the Cruise vehicle was taught about Uncle Rhabdo.

Yes, but the Cruise dragged her for 20 ft first before stopping and pinning her. So it didn’t even remain in place initially, just after dragging her for 20 ft.

It seems related to the car not being aware there was an injured person under it.

It's not hard to imagine that it's simply unequipped to know something like that, and it does seem fair to require that it become so equipped.

Huge supporter of autonomous cars, still feel like this is completely reasonable to require a solution for.

I was just trying to think that through a bit myself! Say this situation happens, the car is on someones legs, it realizes it, drives forward and forward is over the head or chest? It's hard to think how you would want to program it to react even if it was aware something was stuck under it. Interesting situation with AV's I'd not considered before.

Updating my comment to include this:

“When it comes to someone pinned beneath a vehicle, the most effective way to unpin them is to lift the vehicle,” Sgt. Kathryn Winters, a spokesperson for the department, said in an interview. Were a driver to move a vehicle with a person lying there, “you run the risk of causing more injury.”

https://www.sfchronicle.com/sf/article/cruise-driverless-tax...

Reply on news.ycombinator.com