Hacker News Reader

Comment by etothepii | Hacker News Reader

There's a famous story (probably apocryphal) about the military of a powerful nation training an early AI to find pictures of submarines beneath the sea.

There was great excitement as it was near 100%.

It later transpired the pictures with submarines in had a white border.

View on HN

Ha, I've heard a similar story about diagnosing skin cancer from pictures of moles. They were really excited about the performance of the model but it turned out if the dermatologist was concerned about the size of a mole they would include a ruler in the picture to document the size. The NN wasn't trained to "diagnose skin cancer" it was trained to recognize rulers in pictures.

haven't modern model architectures gotten better at avoiding this kind of overfitting? like obviously data quality is still very important, but my understanding is that dropout mitigates this by randomly cutting out these unwanted feature channels. the models learn to distinguish all differences, rather than just one, or fixed combinations of several.

> haven't modern model architectures gotten better at avoiding this kind of overfitting?

Overfitting is, AIUI, a training method and data issue, not a model issue alone. I doubt any model is resistant to overfitting if you give it data where the answer is reliably encoded some aspect it can use but outside of what you want it to look at.

Now, you can notice suspicious results and investigate (or you can just publish a 100% success rate and call it a day.)

It really doesn't have to do with most ML architectures. It has to do with experiment design. If some data used in testing is part of the training process there will be over fitting. That's why a final test set is required for unbiased evaluation.

I've heard this one before but with soviet tanks back in the 2000s!

May be an urban legend, according to this site: https://gwern.net/tank

I think there was something about an ANN detecting skin melanoma, assigning a high likelihood if there is a clinic/doctors office in the background.

Reply on news.ycombinator.com