I'm basically with Tegmark. For one thing, what has physics ever come up with but equations, and improved equations to replace those, and so on? Everything so far has been math.
So, going forward, either everything in physics will always be rooted in math, or else some "non-math stuff" has to be discovered. (The very idea of "non math stuff" being at the root of reality seems absurd.) What kind of thing would that be? I think if we discover the ultimate equations then that's all there will be: those equations, and mysterious constants embedded in them that just are. The only explanation for those constants will be the anthropic principle: if those constants were different, we wouldn't be here to contemplate why they are, and in some other universe there are beings completely different from us also wondering why their constants and equations are they way they are.)