That's not entirely true. Neural networks are fairly robust to noisy training data (a.k.a. garbage).[0] Well,
stochastic gradient descent has the noise in its name. More training data can compensate for noisy data to some extent.[1] I'm not sure know if model size can also compensate for noisy data though, but would not be surprised if it did.
[0] https://arxiv.org/abs/1705.10694
[1] https://arxiv.org/abs/2202.01994