I know this is a joke, but the theory of generalization in NNs is rapidly advancing and it's not quite that simplistic: https://arxiv.org/abs/2003.02139