This exactly. Or at least it seems this way to me, as a non expert with some knowledge. Add another particle, or term, or whatever to make it match. Feels very inelegant. I like your description of overfit, which hadn’t occurred to me before, ironically though I spend a lot of time with ML.

When teaching ML I talk about adding loops to the Ptolemaic planets-orbit-the-Earth-model - and how it actually worked pretty well because they spent a lot of time adding loops to the orbits to make it fit. It was overfit.