Or does it...? Case in point, the fifth axiom of geometry, attributed to Euclid, that through a point there is exactly one line parallel to a given line has been shown to be unprovable, and Lobachevski and Rieman have built alternative geometric theories assuming that axiom wrong.

I find it hard to believe that Math is that different from physics or other sciences in its robustness.

Some advantages that math has had over other sciences is low resource consumption (pen and paper), easier reproducibility(thinking), and it has gotten a long head start of 2-3 thousand years.