Hacker News Reader

>>I can't believe that radar and LIDAR were completely unable to see this woman until the visual cameras picked her up.

The worst part is, that the car didn't react at all - if it started breaking when the cameras saw her, it would still be better than what happened. But there was no reaction. Nothing from LIDAR, nothing from Radar, nothing from visual recognition - which to me, suggests only one thing: she was picked up by all three sensors, and then Uber's algorithm decided she wasn't an actual obstacle and could be safely ignored. Like a leaflet in the wind, or an immobile traffic sign next to the road. That's far worse than a straight up hardware malfunction.

That seems like a correct read of this. They didn't react at all. If the data shows that all three systems picked her up just fine and no reaction was made, that's really damning about the quality of the software that Uber has.

This reminds me about the Tesla incident with the stoped firetruck, the software was programmed to ignore static o objects as an optimization, so we could have the software ignoring the hardware detection, I hope we can find soon what the sensors detected.

They didn't expect a problem, ignoring static objects in the road?

Something like that, https://www.wired.com/story/tesla-autopilot-why-crash-radar/

From what I remember reading it there are many static objects so they are having a hard time deciding what is safe to ignore,AFAIK Tesla is extra problematic since they want to use only cameras and not other sensors

Not just that, it seems to imply that Uber either aren't maintaining the vehicles correctly, or that their modifications are rendering them unsafe on the road.

The Volvo XC90 is an exceedingly safe car already. With the LED lighting, it shouldn't be possible for the headlamps to illuminate that poorly - it's as if the auto-levelling circuitry thinks the car is riding nose-up.

The XC90 also comes with a factory auto-brake that is supposed to prevent the vehicle from having this type of accident - their stated goal is zero fatalities. Did Uber not buy models with this included, or did they de-activate this feature? You'd think an independent system like that would actually be seen as a positive feature...

Isn't the auto brake only active at lower speeds? (<50 km/h is a figure that rings a bell). The car was driving 61 km/h which is higher than normal "city speeds".

I don't know about that specific model of Volvo, but the emergency brake works up to much higher speeds in other cars/brands - it's just that above 50km/h it's not guaranteed to slow down to zero or prevent impact, just reduce the speed. I suspect that system was disabled as it would interfere with whatever driving tech tech have though.

In Volvos case the system is called City Safety and is only active up to 50 km/h (https://en.m.wikipedia.org/wiki/City_safety)

However there might be other systems which break automatically for cars, in the case of a sudden traffic jam for instance.

Or did they put a couple hundred kilos of computing and sensing equipment and batteries into the car and change the weight distribution such that the headlights now point up?

Or did they darken the video?

Three people in the back seat is a couple hundred kilos.

Modern cars, including the XC90 have a tilt sensor to adjust the headlights automatically for this.

three people in the back seat are in front of the rear wheels/basically over the rear suspension. Its a different moment arm to have it in the trunk or on the roof.

Source: I teach statics

If you taught Dynamics you would know that the compression of the rear springs caused by most of the weight being over the back wheels would make the lights point higher.

> Or did they darken the video?

Does anyone trust Uber to provide all the evidence they have, and not just selectively release bits that support them?

Does anyone trust Uber not to alter the video, say by increasing the contrast to make darker objects invisible?

Laser is the sauce, wasn’t it?

Tesla is showing a video, clearly speed up, on their site and it looks like autopilot slowed down as they drove by a pedestrians on the side of the road. Around 1:30 https://www.tesla.com/videos/autopilot-self-driving-hardware... it looks like it is very cautious at stops and when there are things roadside, almost too cautious

I tend to agree with you, I’ll even give their culture a tiny bit of slack if they are simply a ride share/taxi company (very tiny, you shouldn’t abuse or harass people ever) and I think it would make a ton of sense for them to partner with self-driving partners but it seems flawed that they made it into an existential issue that they do their own. It’s ironic, ride share has provided a very reasonable option to driving after a bar visit, with measurable results in some places, safety is something they could sell on.

I agree with much of what you said, but GM understands the stakes? Are you forgetting the GM ignition switch controversy?

I recall one of my favorite scenes from Fight Club wherein Norton talks about the cost of lawsuit*the likelihood of crash vs the cost of a recall.

I'm all about Tech companies getting into self-driving cars, less excited about car companies with shoddy track records on safety and no history of disruption. There's a reason Cruise Automation (the self driving start up that GM bought) insisted on remaining separate from GM and not being absorbed into GM's culture.

edit: spelling

>cost of lawsuit*the likelihood of crash vs the cost of a recall.

To anyone interested in this issue, I recommend the following paper.

https://law.vanderbilt.edu/files/archive/212_Corporate-Risk-...

Also, this story about the engineer who investigated the infamous Pinto 'fire after rear end collision' issue, and the decision not to recall. https://www.newyorker.com/magazine/2015/05/04/the-engineers-...

I think once they get to 'as good as people' we should let them on the road, because they are 'as good as people' but never get tired, angry, and/or drunk :)

I agree, I don't know how we could objectively measure this but I do not see anyone trying to measure this, just selling hopes that soon the AI will be better then humans.

I think the only way to test this is empirically. Just see how many people they kill until statistical significance is found.

However, that's horrific and implies that the engineers are using something of an evolutionary algorithm. Obviously, this does not cut the cheese.

We would need to create some tests and simulations, passing this tests will not imply that the systems are safe but failing them would prevent any broken attempt to be approved for testing.

IF the sensors would be standard maybe we could create fake inputs and test different situation in a simulation, have some basic unit tests, now I am afraid that this companies are just tweaking things, something that today works tomorrow after an update may not work.

Isn't this quite the issue with machine learning algos. There is no 'standard' as the machine is always learning and getting more data in to match against. Testing such a system would mean that you have to 'freeze' the learning portion, something many will not like to do.

I do not know how this systems work, but I assume is not a complex NN but instead is made from layers, with an expert system and the NN would be used only in some sections like identify the objects in an image.

Even if the NN evolve if it recognizes a bike in image X it should still recognize it after it learns more or is updated with new hardware that supports more neurons.

It's fairly easy to measure, fatalities per million km driven. My understanding is that the industry aims to achieve 10x lower fatality rate before releasing self driving cars.

That way of measuring is terrible, any student then can put his own system on streets, kill 100 people, then after 1 million km some agency decide he needs to try again next year.

We need a better measurement that would not involve killing people, at least as a lower bar before accepting this cars for testing.

This is exactly what I was thinking as well, even if it's only as good as a human, humans comes with a whole host of other problems which impair our driving abilities.

A problem I foresee is that even if we could determine them to be as good as a human, they will still be different from a human. People will live that would otherwise have died; people will die that would otherwise have lived. Members of the second set will tend to get far more emotionally involved.

As good as humans implies that over one million people would be killed by robots every year. Some of the families, loved ones and attorneys of these people would be able to make very convincing cases that they would not have died if a human were driving. This is a hard problem to overcome.

Yes we shouldn't accept robots killing people as a fact of life, otherwise we're going to be really unprepared when they decide to do so intentionally...

So my initial read on this whole thing was a bit more sinister. What if uber altered the video to playback as darker in an effort to help their cause?

I was thinking more they intentionally installed cheap dashcams so that they can release poor quality videos that would favor their narrative of events and get ahead of the story/blame.

I don’t think Google is a good example of a company that would understand the dangers involved, given how flippantly they’ve responded to government action in the past. GM I would trust more solely because at least the engineering of cars goes through strict and government-mandated testing.

GM ignition switch scandal is far more egregious than anything Google has done IMO.

The ignition switch scandal is horrible and disgusting, but it helps that Google hasn’t before had to build any products that could have life or death impacts on their users. I suppose the closest you can get is when the revelation came out that Google had been uploading locations of Android phones even when the settings were flipped off, people criticized the danger this posed to domestic-abuse victims.

Maybe Apple, Carnegie Mellon, etc.

What I do know is that Uber, a company with virtually no scruples, is very far down the list of company's that I would trust to develop their own self-driving technology. It makes total sense for them to use self-driving cars to replace human drivers, but I'd feel a lot better about this if they leased it from another company.

Uber basically does own the Carnegie Mellon CS department, they hired up most of the faculty a few years back.

https://www.wsj.com/articles/is-uber-a-friend-or-foe-of-carn...

>> I can't believe that radar and LIDAR were completely unable to see this woman until the visual cameras picked her up.

Well, the LIDAR doesn't control the car. It's just a sensor. The car is controlled by an AI system that takes information from the sensors and makes decisions based on that information (and according to its training).

So the Lidar might very well have "seen" the woman, but if the car's AI didn't recognise her as a human in the middle of the road, or didn't know that it had to stop before hitting her, then the car wouldn't stop.

> If Uber's self-driving cars are no better than the visual acuity of humans, they shouldn't be on the road.

This is part of a test... how can you determine that these cars are better than humans if you can't try them? Put more regulation on the tests, sure, but don't stop them all, otherwise, we wouldn't be able to do any of it. Considering that woman didn't have all her focus on the road, already we see that they didn't put the right person there.

UBER is a terrible company period.

>There is no point to this at all if we can't make robots better than humans.

How about saving human time that would otherwise be spent on driving?

as long as you're ok with the tradeoff; convenience at the expense of safety

GP didn't say self-driving cars should be less safe, only that they don't need to be safer in order to provide a benefit.

If self driving cars are equally good at driving as humans then there is no trade off. It's the same level of safety with more convenience. That doesn't mean we shouldn't strive for greater safety, but I think the original poster might have been a bit over zealous with his words.

You original statement is that they are "no better". If the automated solution is just as good as humans with regard to safety, then there is no trade-off being made.

Given how many things we do that make us unsafe drivers (being distracted by music/passengers/anger/other, driving while tired or otherwise inattentive, ...) I don't see it being long before we can claim automated cars are as safe as those driven by humans, especially in some conditions. I can't say we are there yet, but things are getting pretty good.

I don't understand why people call it unsafe because there are more than zero accidents. By the same measure humans should not be allowed to drive either. Even when people have slightly more reasonable expectations they compare the automation to the best drivers, not any average or bad driver.

Perhaps there are conditions where the automated systems are still going to be worse than the average human, perhaps night conditions like this are included in that. The solution there is to make the automation refuse to continue (refuse to start if not yet en-route, ask the user to take over if so, find a safe place to park if the user can't or doesn't take over when requested) in conditions where that might be an issue. This still allows automation in the majority of driving conditions.

I think we should try finding solutions for the bad drivers that text, drive drunk or speed on the roads, is not impossible to fix that and it will not cost lives.

Do we have any proof that with our current hardware and software we can make a self driving car that is better then the average driver (also this score should be adjusted for the location where the car is used, I do not want the claim to be that this car drives better then average drivers from X where X is a location with tons of bad drivers)

I think we are too optimists that we can train this cars on the roads and by the magic of neural networks and computer vision we get better then average driver. The human brain has evolved to detect moving objects very well, add on top of that intuition where you can anticipate some situation and the capability to adapt to new things.

So we would need some numbers to measure this AIs, but I do not see anyone trying to do this measurements, creating tests courses, checking the sensors quality for this cars, checking the reaction times of this cars, politicians just approved them to test before making any minimal checks.

> I think we should try finding solutions for the bad drivers that text, drive drunk or speed on the roads, is not impossible to fix that and it will not cost lives.

In the time it took you to write that, some drunk person statistically has already killed an innocent person while driving. "It will not cost lives" is incorrect because even the time it takes to discuss this has cost us lives in the drunk driving debate.

It is impossible to fix, in my opinion. People will drive drunk no matter what. They love to do it and there is absolutely nothing that we can do to make them stop, except from eventually eliminating all human drivers licenses.

The punishments/consequences for drunk driving are already very high and they do not stop people from driving drunk and killing someone.

Do you know what "Is impossible means? It is possible to eliminate most of the drunk drivers, but here are some ideas: 1 put an AI with sensors and cameras pointed at the driver, do not allow the car to start if driver is drunk, if he starts texting or sleeping turn of the car, you can add a bypass for emergency,

If you put self driving cars on the road you still have many years until everyone is forced to use this cars so you still have drunk drivers.

So are not companies working on a similar idea like I suggested, because there are not money to be made, cars would be expensive, drivers would not like it, but similar with the airbags, helmets if it will be mandatory to have such a system in the car the safety will increase.

Btw there is a very improbable solution but not an impossible one that would eliminate all drunk drivers, don't allow alcohol production, qed is not impossible

This is more an argument for better land-use policy (allowing more density), more public transportation (proven technology) and more remote work. All of these much better solve giving humans more time than self-driving cars.

Self-driving cars don't promise to save time. It's still driving. Their promise -- their only real promise -- is to save lives. And I think saving tens of thousands of American lives every year is a worthwhile goal. Worldwide, more than 1 million people die in car crashes every year. Humans have proven that they can't drive cars well, but we have put up with it because it is convenient.

I know some people will say that they could work on their way to work, but I can't do work in a car. Staring at a laptop in a car gives me motion sickness, along with a lot of other people. And, of course, this would only apply to people who do computer work in the first place.

As someone who has been in serious car crashes and lost friends to them, I for one am all for self-driving cars. It's going to be a revolution, but we shouldn't be testing them on public roads if they are this bad.

> Self-driving cars don't promise to save time. It's still driving.

Not having to drive the car from and to the parking lot saves a lot of time. Also many people can get work or other things done while being driven to their destination, for example doing phone calls.

That is a negative. Sometimes walking from my car to the door is all the exercise I get. I make it a point to park far from the door.

>I know some people will say that they could work on their way to work, but I can't do work in a car.

But some can, and they would benefit from a self-driving car that can drive _at least as safe_ as themselves. Granted, it'd be great if they could improve on safety, but I think GP's point is that this is not their only possible use.

But why we should allow such a car that is not safe, it could be 10 times faster but 10 times less safe, for years we are trying to make transport safer, this included making cars more expensive so why should we go back in our investment in safer cars because company X wants to be the first on the market.

Way more time is saved not by letting commuters do more stuff, but by letting people who drive for a living work on something other than that.

They'll have so much time free to work on waiting in the bread lines.

Would that console you if you were run over by a self-driving car?

By that logic we should remove all speed limits on roads. Does that seem like a good idea?

I mean, Germany does this on the autobahn. I can't really find any mortality data for the autobahn though. Also, German driving laws are fairly more strict than in the US.

Germany: About 1 fatality per 10,000 cars per year, 4.1 fatalities per billion km driven per year, 38 fatalities per million people per year (2017). Relatively few accidents on the Autobahn, but the proportion of fatal accidents is higher. All in all, the Autobahn is fairly safe, in particular in relation to distance traveled.

USA: About 106 fatalities per million people per year (2013).

I suspect the difference is due to more public transport, less distance driven per person, and possibly more stringent driver education in Germany.

http://apps.who.int/gho/data/node.main.A997

https://www.destatis.de/DE/PresseService/Presse/Pressemittei...

That would increase accident rates. Parent was positing a scenario where self-driving cars don't cause fewer accidents than humans. This could mean a more or less equal amount.

The rider inside the Uber car didn't die...

>How about saving human time that would otherwise be spent on driving?

There may be solutions for your wasted time driving that not costs more human lives, like public transport. An AI is will never have the intuition of a human brain, so you can try compensating with better sensors and faster reaction time, but if you have worse sensors and reaction time then a human then you do not have a self driving car yet, such toy projects were made by students years ago.

Wow, what the hell happened here? Parent suggested that the only benefit of self-driving cars would be if it prevented accidents. This is just wrong, and I don't think you all are helping the discussion by denying this and downvoting me for pointing out that other benefits exist too, even if self-driving cars will be just as unreliable as human drivers.

We can already do that. It's called public transport. The idea of self-driving cars is they can go places public transport doesn't go but the incident happened on a major highway. When I sit on a train I can work without even thinking about where I'm going. It really doesn't look like we're anywhere near that.