Comment by bsaul | Hacker News Reader

Comment by bsaul | Hacker News Reader

Parent Full thread

It's a very interesting video, yet am i the only one that think the whole presentation look extremely naÏve and light for something that's playing with our lives ?

I mean, there are definitely huge shortcomings in the "let's have the computer build the model from examples" approaches, and some of them are talked in the video, others are not :

- rare events are hard to train (that's talked about). The problem is that it's a long tail of unusual events.

- models generated can't be statically analyzed. You can't predict what's going to work and what's not. You can only hope. One very striking recent example is in this video : https://youtu.be/w2BWmSBog_0?t=220 . Here you can see that an AI trained on the model of AlphaZero managed to reach 3223 elo rating (so, far beyond human), yet it blundered its queen. And that's just chess, where every rules are written in advance.

- Models don't build human knowledge. That's more of a philosophical point, but imagine a perfect AI built on neural networks after having read all human knowledge. What can it teaches us ? AlphaZero chess isn't able to provide any clue or explanation on why it favors one move instead of another. You can only learn buy playing against it, but that's all. Not even the developpers can tell you what advances in chess theory has AlphaZero made.

I didn't watch the video, only the slides. But got the impression with this approach it would be pretty much impossible to create safe and reliable self-driving technology. At some point chasing a long tail of unusual events will simply become too costly to continue, but the technology still won't be safe enough to use in the wild.

Self-driving will always be held back by municipalities doing weird things with their traffic lanes and lines (and cones and hazards and signal trucks and road construction) as long as the AI puts as much faith in the lines as it does. And you'll never track down everything because construction is constant in the world, with the possibility of introducing a novel road-weirdness.

It's a road system designed for humans, and with current tech, you'll always have issues chasing the long tail down. Though I'm fairly sure self-driving is still statistically better than humans behind the wheel, especially distracted humans as the information age has made us.

The more immediate solution is white-listing safe roads that have sane paint lines and are relatively straight-forward roads. That way, the chance of the AI getting it wrong is drastically reduced. This covers most highways where you get the most out of self-driving systems anyway. Bonus points if you can automate this decision-making with recent satellite imagery.

Maybe the responsibility (and liability) for accidents due to weirdly designed or badly marked lanes would have to fall upon the municipality. You could make an analogy to trains: if a local government commissioned a rail line that included a junction that can't be navigated without derailing 1% of the time, you would not blame the trains for not being able to deal with it.

Another analogy could me made to data entry: you could allow free-form text input and try to process all of the long tail of unexpected inputs, or else set out a format that the data has to follow and enforce validation at the input.

> as long as the AI puts as much faith in the lines as it does.

The need for lines is only a temporary crux. AI systems will continually improve until the white lines are guidelines only. There are lots of other factors in an image that can be used to deduce where the lines should be.

But there will have to be handling of uncertainty which humans have to do too, simply just slowing down (not sharp braking) will help most cases.

I share your worries, as I too think that the abstractions used in the trained models in the talk are inherently leaky, meaning that they do not represent reality to a suffiently high degree (see corn flakes vs ketchup screen wiping action). As we do not (yet?) understand the underlying nature of those "2.0-ish" decision algorithms, we will not be able to make a judgement call when they will work, and when they will fail.

The main point you offer is that models can't be statically analysed and will do things unexpected that result in loss of human life. Here's my response:

- Humans also do unexpected things, like stepping on the gas instead of the brake. If Autopilot does unexpected things at 0.01% the rate that humans do unexpected things, then it is a huge safety bonus to use autopilot.

- How can we solve image recognition any other way? We must move the needle forward on our technology. If we do not struggle against the adverse side-effects of our software and make it better, we will never advance it and we will be stuck requiring human drivers for all driving tasks.

> If Autopilot does unexpected things at 0.01% the rate that humans do unexpected things, then it is a huge safety bonus to use autopilot.

You’re forgetting a variable - the frequency or likelihood of a situation to occur. Go far enough down the long tail and neural AI can get far more deadly than human drivers.

It's unclear what your argument is

I’ll try to clarify. In my opinion, the weakness of neural systems is their inability to deal with input for which they have comparatively little or no training. There’s no way around that, except by introducing structures or systems outside the neural nets themselves that provide logical frameworks for dealing with rare events. (And I don’t just mean a bunch of logic in code. Expert systems are one potential approach, for example.)

Your point was that mosern autonomous driving systems can drastically reduce fatalities over human drivers—-and I agree, but only for circumstances for which the car’s neural systems have been well-trained. But the systems ought to be able to handle ~99.99% of the types of circumstances gracefully before most of us will trust them to safely drive us around.

>Humans also do unexpected things, like stepping on the gas instead of the brake. If Autopilot does unexpected things at 0.01

But the difference is, one human doing the wrong thing does not mean every human Will do the same thing, given the same scenario. But whereas a software running on all cars will exactly do the same wrong thing given the same input. Sure, three is also an advantage (arguably), as fixing it once fixes all cars but that has other problems in taking on the update.

You've got a point, the only problem is that we know when humans are more likely to do errors : stress , night time, long hours, high traffic, etc.

With AI we could enter a world where you'll have 0.0001% of having an accident, but this could happen anywhere anytime in any random situation (such as a specially shaped cloud in the sky).

This is what makes it unacceptable, IMHO.

Reply on news.ycombinator.com