Mobileye says its software would have seen pedestrian in Uber fatality | Hacker News Reader

Mobileye says its software would have seen pedestrian in Uber fatality

reuters.com

49 pointsKKKKkkkk187 comments

Look. It's generating bad press right now for your industry. I get that. I also think that in the long run your industry will be far safer than human drivers.

But now is not the time to post speculative bullshit like this. Would your car also have run into a school shooting unarmed?

You weren't in that situation, you don't know all the edge cases, the best you can say is that you had a similar situation under controlled circumstances and reacted appropriately.

This is publicly patting yourself on the back for a competitor killing someone. To say it's in bad taste is an understatement.

Missing from the article is the nature of the link to Mobileeye -- the Volvo cars Uber is using come with collision avoidance built-in, Uber disabled it. The Volvo collision avoidance uses technology from Aptiv, which in turn uses technology from MobilEye.

So this isn't just a random company saying "we could do better", this is the company with equipment already on the car (though disconnected) defending their reputation.

Except their technology was ON the car that was involved... they are part of the story -

Uber turned their gear off in favor of their own.

Not only that, but all the data could be recorded and replayed through other software, so they could conceivably say what it would have seen and done.

Given that, maybe they're trying to head off some potential bad press if a journalist gets some facts incorrect.

Yeah this looks more like a defensive move than an opportunitistic marketing opportunity as OP framed it to be.

¿Porque no los dos?

When you say this is why we've been calling for everyone to take it slow that's a defensive move.

When you say your setup would have detected her a second before the impact, that's an opportunistic marketing opportunity.

Incidents like these could very well kill the technology just as its leaving the crib, if mishandled.

It's all defensive. In this case, the interests of the companies and the interests of the users are aligned.

The whole point is that uber is developing their own gear. Dunno about you, but for me giving two self-driving systems control over the car at same time seems like a recipe for disaster. Of course they turned off the other systems.

Absolutely not. Having two sets of sensors on the car is a good thing because it provides redundancy. If one set of sensors fails to detect say, a person in the middle of the road, the other one will step in and say "hey, brain, there's a person in the middle of the road".

If Uber intentionally disabled the second set of sensors, they were acting recklessly, and they're a danger to the development of the self-driving car industry. This one incident may have set us back years.

Mobileye isn't a "self-driving" system, it's a collision prevention/mitigation system, like airbags or ABS. Leaving it on while a human controls the car hasn't been seen as a disaster.

first of all Mobileye does have a self driving system (their EyeQ4 chips is for Level 3-4 self driving). However Amon was referring to their EyeQ3 chip from 2014 which is installed in 24 million cars and have billions of miles

Tesla’s autopilot runs on MobilEye only AP 2 runs on NVIDIA.

All kinds of technology that’s less half-baked than Uber’s self-driving cars has redundant, decoupled safety systems.

Might be better for Uber’s data collection to turn this off, but you really can’t argue it was safer. I don’t think we should be killing people because that works better for their business needs.

This is a system designed to respond when a human driver has failed to identify the threat, so we've already established that having two systems (one dumb system and one intelligent system) doesn't compromise safety.

Which one, human or automated, is the "dumb" system?

They certainly averted disaster, didn't they.

> Dunno about you, but for me giving two self-driving systems control over the car at same time seems like a recipe for disaster.

When I drive a car, I am a 'self-driving system'. It is intended that I am used in conjunction with the car's existing driving-assist functionality.

Aircraft use multiple flight systems developed by separate teams in separate languages/technologies -- and then compare their answers to get redundant consensus.

One fear is that Uber's mistakes will endanger the whole self-driving industry, by making people afraid. I'd say these companies are trying to separate themselves to avoid that.

I imagine we'll see self-driving car horror movies soon.

We've had them in the past (Christine, Trucks, Herbie the Love Bug), but the only thing Hollywood loves more than latching on to a trend (Emoji Movie) is blowing something out of proportion.

Just last year that was an arc in Ash vs Evil Dead.

Granted it was self-driving thanks to a book inked in blood and bound with human flesh, but that's beside the point...

soon? havent you been watching new Xfiles?

Damn, I’m already imagining them like the climate change movies where a 800-foot tsunami hits Manhattan. 200 million self driving cars have become hostile and are trying to kill the humans. The problem is that they have commandeered a group of friendly-looking Japanese humanoid robots to pump gas for them. One rogue humanoid robot is the one who will save humanity along with help from a 6-year-old girl with pigtails and a stuffed rabbit named Hopsy.

Who leaked our script? Ugh...

That doesn't make it the right time to publicly announce "we wouldn't have killed someone here, look how great we are".

Near my hometown a Tesla crashed and the crash killed the driver recently. Guess who gave a statement within days blaming the driver for speeding? For a European that was quite the novelty (and probably illegal, although AFAIK deceased lack privacy protection since not a natural person).

Musk definitely goes too far with his response to any criticism of Tesla. It makes him sound a little like Trump — “I’m so amazing”

i know it will sound insensitive, but one person dying is not that big of a deal... heck 1000 people dying is only 10% of the amount of people that die each year for drunk driving[1] and you don't see liquor being pulled off the shelves.

however i do see how people (including me) are sheep and can be manipulated into believing anything. taxis and trucker unions can use this an opportunity to condemn self driving vehicles.

[1] https://www.nhtsa.gov/risky-driving/drunk-driving

That doesn't just sound insensitive.

Watching the video, I'm convinced that person died for no good reason beyond Uber rushing tech out the door before it was ready. An alert human would have braked and swerved in that circumstance.

People do die from drunk driving, and those drivers go to jail because they fucked up in a huge way. That's no reason to give Uber a dead-person mulligan. Indeed, it's a reason that some Uber exec should perhaps be wearing an orange jumpsuit over this.

I'm all for self-driving cars, as done right deaths will go down. But that means we shouldn't introduce them until they are well-made enough that deaths actually go down.

Watching the video, I'm convinced that person died for no good reason beyond Uber rushing tech out the door before it was ready. An alert human would have braked and swerved in that circumstance.

It's likely that the accident still would have been fatal, or at least the pedestrian would have been seriously injured if the safety driver was driving. Based on the video there was only a second or two to react from the time the pedestrian stepped out of the shadows, and human reaction time varies from 0.7 to 3 seconds [1]. One study found the average driver's reaction time is 2.3 seconds.

[1] http://copradar.com/redlight/factors/index.html

>It's likely that the accident still would have been fatal, or at least the pedestrian would have been seriously injured if the safety driver was driving. Based on the video there was only a second or two to react from the time the pedestrian stepped out of the shadows, and human reaction time varies from 0.7 to 3 seconds [1]. One study found the average driver's reaction time is 2.3 seconds.

Yes, but it is difficult to state that it would have been "likely" fatal, and even sevral fractures are IMHO a big improvement over death, the whole point is not about the "average" braking time (BTW that same study gives 1.64 seconds average for "steering away" which is probably what most human drivers would have done in a similar situation, instead of braking or instead of only braking), it is the peaks that count.

I mean, it is about the (rightful) expectation that an automated system is safer (faster, more reactive) than the best you can find (not the average, the best) "in nature" among a random set of drivers, just like the state codes (correctly) take the worse case in the sample or however incereases the recommendation to three seconds.

If you play the game suggested on the same site you gave a link to and deliberately count up to three before clicking, you get:

Reaction time: 2916.00ms

Reaction speed age: 89 years old

Playing fairly, you are likely to get values in the range 300-500 ms.

Here is another paper:

https://www.researchgate.net/publication/233039156_Brake_Rea...

that puts it back to around 1-1.5 seconds or however maximum at 1.8 seconds in the very worst cases.

The video camera doesn't have the same vision as a human. A human would have seen something sooner and so would have had more reaction time.

You also mis-cite your numbers. Reaction time to braking in that study was 2.2 seconds, but steering is 1.6 seconds, and beginning to slow is 1.0-1.3 s. Those numbers are also from a study environment; it's plausible to me that people would be more relaxed in a test than if they see a live human appear in front them.

Should Uber Test Driver go to jail? She was on camera looking down at her phone at the moment of accident.

I think expecting a backup driver to pay that kind of attention on a continuous basis is unrealistic on Uber's part.

Are you familiar with Threat Image Projection? It's a system where baggage scanning machines put in images of fake threats into real baggage. Why is this necessary? It's because regular baggage doesn't have enough in the way of actual threats to keep operators alert.

I think it's the same deal with being a backup driver for a robot car. Sure, they should be ready to take over when the car asks for help, and it's great if they notice a problem and stop the car. But I don't think expecting them to have this kind of reaction time is realistic.

(That may not stop Uber from throwing the backup driver under the bus, though.)

She absolutely should, in my opinion.

then it negates that whole idea of having a "self-driving" vehicle. the whole point is to let the vehicle drive itself while you can do whatever-it-is you want without paying attention, like sleeping or playing on your phone... you essentially are a passenger, not a driver.

She wasn't in a self-driving car. She was testing a prototype self-driving car. She's absolutely culpable for her negligence. She had one job.

Well, they were still jaywalking in the middle of the night hoping oncoming cars would stop for them.

It's not like the car hopped the curb.

In many (all?) states, pedestrians have the right-of-way, even if they're jaywalking.

Jaywalking means they were walking when they did not have the right of way.

However, whether or not a driver is civilly liable for an accident usually doesn't even care about the right of way. For example, if a pedestrian is standing in the middle of the road attempting to block traffic. They have no right to be there, however, it is not ok to run them over.

They were crossing a low speed, brightly lit street.

It's a gamble that jaywalkers lose every year.

When you're going to jaywalk, ensure that the next car is far enough away to where it doesn't matter if they stop. That's the correct way that would've saved this person's life without any further analysis.

I'm taking issue with this:

>that person died for no good reason beyond Uber rushing tech out the door before it was ready

Well, no. They died because they lost their gamble on an unguided crossing where the oncoming car did not stop.

We are now analyzing why the car didn't stop, but notice how no charges were pressed.

No charges were filed yet, but that's unlikely to happen until after the NTSB report.

It's true that jaywalking is a gamble, but so is crossing a street with the light. Leaving the house is a gamble. That doesn't mean we can blame the victim of a collision for leaving the house. It can be simultaneously true that A) jaywalking is not always a great idea, and B) the Uber car should have avoided this collision.

i agree with you 10,000%. i think everyone is so concerned about why the car didn't stop, they forget that jaywalking or running out into the middle of the road is a gamble always.

i wonder... if the person crossing had been behind an obstruction and stepped out right in front of the car, would everyone still blame the car or the driver for hitting the person crossing?

Did you watch the video? I think any decent self-driving tech would've seen her, but I highly contend with OP's assertion that an "alert human" would have stopped in time. The road was very dark, and she came out of nowhere.

If the car was speeding, you might be able to make that argument, but I don't think anyone has made that claim yet.

Perhaps you would be so kind as to contend with what I actually said. I did not actually say a human would have stopped in time. I said a human would have braked and swerved.

The video is misrepresentative of the actual road conditions.

https://arstechnica.com/cars/2018/03/police-chief-said-uber-...

It also seems to me that this experienced company is trying to setup road blocks to stifle competition and using this fatality as cannon fodder to support his arguments.

How about we let whoever wants to research autonomous vehicles have at it, and we just throw the "Supervising driver" who failed in jail like he should be?

I mean the guy had one fucking job: hit the brake pedal if you're going to crash into something. He failed. The car is an experiment. The driver is the only safety mechanism in that car that matters, and as usual with humans it screwed up.

Why is it a 'similar situation'? They fed the same footage that the Uber system had access to.

In the lab, yes. Presumably in real time. Presumably without configuring things differently than normal, except that they used a pre-recorded feed. Presumably without multiple attempts. Presumably without...

I believe that Mobileye has said the opposite in cases where their current technology would not have been able to respond as well. And their technology was on the vehicle being involved in the crash, but not being used.

In the past, Mobileye has made it a point to be very specific about what their technology is and is not currently capable of. I don't see a problem with that.

I agree. But suppose the difference mentioned (1 sec) is true. That's such a large difference that what they call for (formal outside verification) sounds pretty reasonable. This debate will always be hard since it will involve deaths, with AI and video involved.

You're right. Developing video games has taught me that a second is a long time. But we still don't even know that Uber didn't detect them that early. The CV system or other sensors could have picked out the pedestrian but something else went wrong instead.

I tend to think that if Uber did detect the pedestrian, they would have said something by now. But perhaps they prefer the optics of not having seen her to the optics of seeing her but not braking at all. We simply don't know.

Imagine that press conference.

> Uber: "After examining the black box on the car we have determined that the car detected the pedestrian but failed to brake."

> Reporter: "Was this because the pedestrian was a woman?"

> Reporter: "Did the car do this to protect the life of the Uber employee inside?"

> Reporter: "Was her credit score too low?"

This reminds me of a survey I had taken. I can't remember the survey publisher but it was related to self-driving vehicles.

You were presented two pictures (e.g. a picture of school kids crossing a crosswalk and the other a picture of an adult male and female walking their dog across a crosswalk). Presumably you had to choose; would you swerve to hit the adults and dog or would you stay course and hit the kids.

I know there is no moral compass for a computer system to determine what it should do in the scenario but I laughed at the quiz because I couldn't believe this is the kind of data that will feed machine-learning algorithms.

It just feels bad.

Getting data on an ethical dilemma. [1]

[1] https://en.wikipedia.org/wiki/Trolley_problem

Thanks for mentioning what that dilemma was called.

For those interested here is an interesting article about the Trolley Problem.

https://www.currentaffairs.org/2017/11/the-trolley-problem-w...

I really doubt that the survey was about actually training the cars. It probably has more to do with positioning the marketing messaging around them.

AFAIK, the trolley problem isn't something that the engineers consider to actually be a major problem.

Thanks for the insight jsight. I sure hope engineers wouldn't use it.

To be fair, it has been a few years since I took the survey so I can't recall if it mentioned what the data was used for.

I don't know enough about machine-learning and self-driving technology but my one question would be, where do engineers get that kind of data to feed into a system? At some point a scenario like the Trolley Problem is going to happen (near or distant future) and what kind of data is going to be making that decision to swerve or remain on course?

Mobile-eye has a history of throwing partners and competitors under the bus (see Tesla). It's not just about bad events, it's part of their corporate DNA.

i get the feeling of some people here calling for bad taste, but mobileye ceo has been a long time proponent of playing it safe, underpromise, warning against the theoretical limits of modern technics, and has claimed for a very long time that fully autonomous driving wasn't doable in a near future ( proof of that in the numerous videos you can find on youtube).

i guess it's only fair for him to speak up one more time now that his fears has materialized.

Speaking up is understandable: Mobileye supplies its technology to Aptiv who supplies the system to Volvo for the XC90. The reports are that their system was disabled by Uber at the time of the accident. It is understandable that Mobileye does not want to share the blame in this case.

Ref: https://www.bloomberg.com/news/articles/2018-03-26/uber-disa...

“We don’t want people to be confused or think it was a failure of the technology that we supply for Volvo, because that’s not the case,” Zach Peterson, a spokesman for Aptiv Plc, said by phone. The Volvo XC90’s standard advanced driver-assistance system “has nothing to do” with the Uber test vehicle’s autonomous driving system, he said.

[...]

Intel Corp.’s Mobileye, which makes chips and sensors used in collision-avoidance systems and is a supplier to Aptiv.

We really won’t know what happened until the NTSB report comes out, but I’m not convinced this was an object detection failure. It seems really possible to me that the sensors detected the bike-walking person as a bicyclist, and a bicyclist in a different lane is not a reason to slam on the brakes. They might have just forgotten to add a rule that a bicyclist in a parallel lane is a reason to brake if it’s perpendicular to you.

Any sane self-driving system needs to be a lot smarter than this. It needs to do motion tracking and estimation of future position. Otherwise it's going to run into lots of cyclists and pedestrians, and continue to fail for objects it can't identify (imagine a cyclist carrying something big and bulky). The software needs to be smart enough to not hit anything, and in order to do so it needs to extrapolate future positions of objects based on their current velocity.

Obviously it needs to be smarter. As for motion tracking, it might have gotten confused as the victim stopped moving. But I’m just not sure that the issue was the sensors or object detection.

The victim stopped moving ... right in the path of the car, which never braked. It needs to be a lot smarter if it doesn't even understand that it should break when there's a large unidentified object, the size of a person plus something else, right in the lane it's traveling in.

We won't know yet exactly what happened, though hopefully the NTSB is getting full access to all sensor data from the car. It could be a huge combination failure of multiple different things. It sounds like it's nowhere safe enough to be driving autonomously on public roads though.

You don't slow down and proceed with caution when overtaking a bicyclist in an adjacent lane?

When they have a whole lane? No. As a cyclist I rarely even see drivers slow down when they’re going 55 and I’m just on the shoulder.

I'm struggling with the question: Who's fault is this?

The pedestrian for walking across the street, wearing dark colors, being oblivious to traffic?

The human in the drivers seat for not paying attention?

Uber kind of gets the laundry list: For reducing the number of "safety drivers" from 2 to 1, for not using eye tracking to ensure the safety driver was paying attention, for faulty software or hardware that didn't detect the pedestrian, or for software that after getting the sensor input made a decision to not adjust speed or change lanes, for having some of the worst "intervention" numbers in the industry yet plowing ahead with removing a safety driver from the car...

Who gets the blame?

I'm mostly a "get rid of idiotic safety tags and let natural selection sort it out" sort of person, but on the other hand I feel that piloting a multi-ton killing machine needs to be treated as a huge responsibility.

I wondered the same thing on the following thread, posted about a month ago:

https://news.ycombinator.com/item?id=16477295

I believe some manufacturers will take entire responsibility while in some cases it's the person that "starts" the self-driving vehicle. Legislation seems to be on a state-by-state basis at the moment.

https://en.wikipedia.org/wiki/Autonomous_car_liability

Good for these people. If Uber really had lidar+radar [1] and disabled Volvo safety features [2] (such as radar) there is no excuse for this. Uber should have clarified the video released may not be representative of the information they have.

[1] https://techcrunch.com/2018/03/19/heres-how-ubers-self-drivi... [2] https://www.bloomberg.com/news/articles/2018-03-26/uber-disa...

Good for these people. Uber has lidar+radar that should have seen this and did not make an effort to publicly respond to the police video that was released. Everyone who works on these problems knows camera exposure is very tricky... And in this case deceptive.

Ok, Mobileye would have seen the pedestrian 1 second before impact. That's not important though...

The important thing is would it have saved her life? Would it have taken action and if so when and how?

It's not clear that Uber's platform didn't see the pedestrian. In fact, I find it unlikely that it didn't see her, between radar and the bike, lidar, and optical (and sonar).

I think the situation here is that there were many failures and decisions that led to this death. Some of it is Uber being too cowboy, but there are a lot of other factors too.

At 45mph the car is traveling at 66 feet per second, so 1 second before impact means it was spotted at approx. distance of 66 feet. The volvo XC90 breaking distance at 62mph is 112 feet. Dividing by the square of the speeds gives us a 45mph stopping distance ~62 feet. So not only would it have saved her life, it could possibly have prevented the car even touching her. (The XC90 has excellent breaking, btw, better than a range rover sport or an audi Q9).

> The important thing is would it have saved her life?

It probably would have. Slamming on the brake for 1 second would have reduced the car's speed enough to change things from fatal to seriously injured.

According to: http://www.batesville.k12.in.us/Physics/PhyNet/Mechanics/Kin...

In one second you can shed 20 miles/hour of speed. That would take the car from 38mph to 18mph. (Those numbers are overly precise, but gives you an idea of the reduction that can be accomplished.)

Not only that, she was crossing from left to right and was hit by the edge of the right side of the car. She could have been missed entirely.

The car could have also swerved and missed her entirely. IF it decided to take action. That is the point I'm trying to make.

Saying your product would have seen the pedestrian is very different from saying it would have seen her and known to immediately take action and the action it would have taken would have changed the outcome.

Using Tensorflow : https://twitter.com/pramodxyle/status/978498938243227648?s=2...

Was Mobileye actually part of the implementation of Uber's self-driving technology and was somehow disabled or not used properly? Or is this just a piece of not-so-submarine PR? If the latter, I don't understand the point of this article.

Uber disabled Mobileye's detection system in the car, from what I can understand.

With all respect to Sha'ashua, this is empty PR. mobileye is a decision support system for human drivers, they can't make any claim regarding this accident.

All these people and companies trying to profit from this accident is really poor taste.

This is not profiting, this is self preservation.

If the purpose of a company is profit, is there really a distinction?

Definitely. One PR approach would be to capitalize off of this tragedy for sales. That doesn’t appear to be the approach they’re taking though (“Please don’t regulate us into oblivion” or “Self driving can be done safely, don’t judge the industry by the failures of one participant”).

This reminds me, does anybody know what's up with Geohotz's comma.ai?

The project was canceled and geohotz posted the code on GitHub:

https://www.theregister.co.uk/AMP/2016/10/28/commaai_car_kit...

https://github.com/commaai

Reply on news.ycombinator.com