Hacker News Reader

Comment by alistproducer2 | Hacker News Reader

The fact that this flaw is known and in production gives credence to some of the AI experts who are saying level 5 is waaaaay further away than we are being led to believe.

View on HN

I feel the true danger is in the "uncanny valley" of automation - it's good enough that you don't need to pay attention 98% of the time. People underestimate tail risk, ignore the manual etc.

My guess is that you're right: level 5 will take about as long as it takes to get to photorealistic rendering - never quite there.

If you apply that reliability percentage directly to time driven (inaccurate, I know, but for the sake of argument), a system which takes care of 99% of driving problems will still abandon you to chaos for 18 seconds out of every 30 minute drive. This is an eternity in a moving vehicle where milliseconds can draw the line between life & death.

The only systems safe enough to allow fully automated vehicles in the near future will have to be on dedicated lanes or roads, probably with additional safety mechanisms on the roadway itself.

It's not a problem with lidar according to the article. Everyone except for Tesla works with lidar. So the question is just when the systems are cheap and small enough for everyday use. AI doesn't appear to be the issue here, just the data input.

The article explicitly states that at least Volvo has a similar warning in their manual.

Is Volvo already building lidar sensors into their public available cars?

Volvo also relies on just radar and a camera.

https://en.wikipedia.org/wiki/Autonomous_cruise_control_syst...

I don't expect scanning lidar sensors to become common here until solid state scanning lidars reach mass production.

Available cars are not fully self-driving. I think this year only Audi will release one where the car takes full control on motorways. All others require you to remain vigilant. If you sell a self-driving car you can't put warnings like this in the manual since the manufacturer is on the hook if anything happens during self-driving (at least if it was the car's fault).

The article states that every except Tesla plans to work with lidar, not that they are currently. The technology still needs to be toughened up and made cheaper for it to be viable.

Well, level 5 without lidar at least is far away. Being able to tell both the distance and direction of obstacles makes avoiding them much easier.

The very instant the “just around the corner” hype dies on lvl 5 automation, Uber is going to drop dead as a ride sharing service. Tesla will take a hit, as will all of the companies scrambling to be first up the automation ladder. The torrent of VC and corporate money will become a trickle where it doesn’t just dry up.

All of which is just to say that those voices pointing out the technical challenges are up against seriously desperate interests. This could get messy.

Tesla I'm not so sure. I expect the hype/valuation is based more on EV leadership. But Uber? Probably? Especially with new leadership, I don't really understand the logic of continuing to subsidize rides below cost if you're not trying to maintain the fantasy that some miracle is going to happen that radically alters the cost structure.

No, it just requires the sea change of having every vehicle on the road talk with each other. That would fix this particular problem quite nicely.

It's not a panacea of course.

What about cars breaking down that stop communicating?

Well, obviously the system would have to still use radar, it can't just assume that everything on the road is communicating.

But it would fix the firetruck problem pretty neatly, as professional vehicles will be required to stay maintained.

You mean for the system to work every vehicle which might have to stop in the road will have to be upgraded? Doesn't seem feasible.

The cars around that car are still communicating. So the car next to it sees a drastic speed drop and tells everyone else.

Also say the cars regularly ping each other for updates, then one car that was in sight and is still in sight disappears doesn't respond, then it gets flagged as a possible hazard.

What if 2 cars fail to communicate and can't flag each other?

You'll have an accident. But we have accidents already. The question is, what failure rate will we tolerate from autonomous vehicles?

It should be Zero. Computers don't make mistakes is the entire reason anyone would even consider letting an automated car pickup their kids.

I disagree and I have a kid. I'd be happy to let an automated car pick up my kid if it was demonstrated to be safer than I'd likely be.

Car related deaths are fairly common, I can't see any rational reason to not switch to autonomous driving as soon as they are statistically likely to save lives.

Do you really want a large (35,000 a year or so in the USA) people to die every year until autonomous cars are perfect? Not to mention nothing is ever perfect.

It's a trust issue.

Doing your best and working towards safety is one thing. Putting 100% trust to autonomous thing is a different thing. If you know it's 100% safe, than it's rather easy to do. But if it's 98% safe.. It's pretty much gambling. You know it may fail and you have no chance to influence it. Sit back and hope you're not the (un)lucky one.

Agreed. We won't ever get autonomous systems. I don't think that should even be a goal for production cars. We'll get roads/areas retrofit with signals to help cars and I think some type of coordination signals/beacon locally between cars.

We really want to be able to have cars avoid obstacles on the road whether or not those obstacles are cars or not. And given how infrequently firetrucks are replaced we shouldn't expect that even if every new vehicle had this technology it wouldn't have made a difference anytime soon. And given the inherent security vulnerabilities of the technology, I'd expect that it would be a net negative to road safety by the time it had any substantial impact compared to the general progress of self driving cars getting better.

Reply on news.ycombinator.com