Hacker News Reader

Comment by peteretep | Hacker News Reader

Random thought: the more deep learning is used in training, the less humans will be able to retroactively explain decisions; this surely has liability implications

View on HN

Perhaps NNs need functionality to come up with fake rationalizations for their decisions just like the human brain does.

I'm thinking that this idea about "fake rationalization" is a bit backwards. We say that many actions happen before we are aware of it or have a rationalization for them but becoming aware of why we did something after the fact doesn't mean that "you" weren't part of the decision. Aren't your reflexes a part of "you"?

Of course, but nobody is going to say "my reflexes did it". We come up with stories, lines of argumentation, etc. Even though these are "ex post" we often manage to convince ourselves...

In Homer's time the Greeks viewed the unconscious as an external (divine) force, the most famous example being Telemachus's sneeze. In a weird way this feels more intellectually honest than the thing we do.

    > we often manage to convince ourselves

In this case, presumably the trick is to convince a jury.

NuTonomy has an autonomous driving technology based on formal logic [1]. It seems like formal logic is a better approach for retroactively explaining software decisions.

[1] http://spectrum.ieee.org/transportation/self-driving/after-m...

Google too uses plenty of formal logic in their autonomous OS, it's a frankenstein of various machine learning techniques. Narrow pattern recognition alone, while powerful, is unlikely to ever be suitable for things like negotiating busy four way stops.

There is the presumption that all you gotta do is gather enough training data and shazam!, you got a self driving car, but that's bollocks. There is a tremendous amount of elbow grease that foes into developing and validating an AV to six sigma reliability. Six sigma might not even be good enough for something as safety-critical as a driverless car.

It doesn't need six-sigma, just to be significantly better than humans, and for the manufacturer to take on the liability.

In strictly rational terms It has to be better than 1 fatality for every 100,000,000 miles driven, but public opinion isn't exactly rational so it probably needs to be even better than that.

It'll be that way at first, yeah, but once it becomes obvious how much better the AI drivers are, it will start to become irresponsible to not use it

I don't see how this is true. With a machine system you would at least have the ability to log the mathematical operations and results, which would be complex, sure.

But more complex than figuring out if another person is lying about a very nuanced and opaque decision they made?

The scope of your question is too large. Do not compare the complexity of AI to the human brain. Instead compare AI to traditional robotics. In expert systems rules are laid out and logic chains can be followed. AI is a black box of a billion random weighted nodes. The system does not care about 'why' when reducing loss.

Yes. This is why https://news.ycombinator.com/item?id=13111516 was on the front page last month.

I feel like this is very actively being addressed, for example by visualization of intermediate features.

Reply on news.ycombinator.com