Hacker News Reader

Comment by jml7c5 | Hacker News Reader

I wonder what went wrong here. Perhaps it wasn't trained for these lighting conditions? I can't tell if they use multiple cameras around the stadium or not.

View on HN

The thing about AI is that you never truly know what it actually looks looks for. This reminds me of the tanks neural net story: https://www.gwern.net/Tanks

It used to be that we had to chose between models that were 'powerful' (complex neural networks) and those that were 'interpretable' (linear régression, trees). In the last years a lot of research has come out.

We have since then developed ways of analysing the inner workings of neural networks.

You can understand how each feature drives the predicted result by using SHAP https://github.com/slundberg/shap

You can analyse the individual layers and gain an understanding of how each layer encodes the input

https://ai.googleblog.com/2017/11/interpreting-deep-neural-n...

https://www.kdnuggets.com/2019/07/google-technique-understan...

Do we 'know' in an absolute way why a network thinks something? Not yet. But neither do we know that for humans. We just have a huge amount of experience with their failure modes and we work around them (see aviation).

That's true, I know that you can view what "excites" specific neurons. I still think the is a problem: this only shows whenenever you give an image what it uses from that image to classify the object, but when a new, unseen image is given it's really hard to tell beforehand what the network will actually look like and which pixels would be considered. Yes, you could look at the features and see which patterns appear there, but for large networks those are thousands or millions, so it's still hard to tell what the result will be, unless you actually use the new image as an input for the network.

Lighting conditions was my first thought, the camera aimed almost at the sun presents some challenging contrast that may not have been trained properly.

Reply on news.ycombinator.com