If you can establish the axioms of zero, 1, adding, multiplication, etc., you can then continue into the rest of mathematics and also computation.
As a side note I don't believe in infinity as anything other than a mathematical concept nor do I believe in parallel universes. I do believe there is a finite amount of universes (if more than one universe exists) and therefore a finite amount of matter.
I think if we found intelligent life, they would have discovered precisely the same mathematical facts.
To circle back to the philosophy of mathematics - pure math is pure logic. I think discovering God is more about physics and the Big Bang rather than via mathematics. However mathematical facts can be so pithy and perfect, that sometimes you wonder if God made them, and you are discovering mysteries of the universe as you find more mathematical discoveries.
Discussing God on HN is not common, but given the linked post and its subject matter, it's hard to avoid that topic.