> It doesn't rely on any kind of structure from the universe to hold it up

I am not sure that this is the case, for math to be it needs a set of axioms.

Axioms have to mirror some sort of universe. I am really curious what can be done if we remove all the axioms.

I am sorry if I sound dumb, I don't know much, but it is really interesting.