Don't confuse the terrain with the map. The formal systems which underly mathematics describing the universe are incomplete (containing true but unprovable statements), but that is not a statement about the universe.
I'm not sure I see the point. Or rather, I see The tautology. We discover something math can't explain. By definition, you can't expand math to explain it. You can try, and if you succeed you were wrong all along, in that your discovery could have been explained by math you just couldn't prove it at the time.
That's not exactly what Gödelian incompleteness means; it means that there are gaps within each formal system. Arithmetic is still there, beyond reality, beyond each formal system's ability to describe in full.