We know about lots of things that math doesn't explain. AFAICT there are reasonable information-theoretic grounds for suspecting that many if not most things we might want to ask about won't yield the the kind of explanations we find satisfying. Irreducible complexity is a thing, at least apparently, and among documented phenomena there is a spectrum of amenability to formal modelling. Statements expositing an 'unreasonable effectiveness of mathematics' or TFA's aesthetic rendition don't seem to amount to more than 'some phenomena admit simple explanations'. Much more often the best effective mathematical explanation is a lukewarm admixture of awkward cludges, or is simply non-existent.
How does that square with something like godel's incompleteness therom?
Don't confuse the terrain with the map. The formal systems which underly mathematics describing the universe are incomplete (containing true but unprovable statements), but that is not a statement about the universe.
I'm not sure I see the point. Or rather, I see The tautology. We discover something math can't explain. By definition, you can't expand math to explain it. You can try, and if you succeed you were wrong all along, in that your discovery could have been explained by math you just couldn't prove it at the time.
That's not exactly what Gödelian incompleteness means; it means that there are gaps within each formal system. Arithmetic is still there, beyond reality, beyond each formal system's ability to describe in full.