In real world datasets a single class may only be in 1% of the data to begin with. The vast majority of images on the internet do not contain cats. So making a cat recognizer that is even 10% accurate is a significant challenge, simply because of the low base rate of cats. Imagenet in particular has 1000 equally balanced classes, so each class makes up 0.1% of the data. And 10% accuracy is way better than chance.

Additionally very few machine vision systems are completely hand designed. They all use some form of machine learning and optimization. Just not necessarily NNs.