Nonetheless, I think some proof theorist have a pretty neat way to think about all this. The idea goes as follows: our brain is a computer programmed by millions of years of natural selection and doing math is the activity of trying to decompile some program running in it.
Taking that at face value would give element of answer to the "unreasonable effectiveness of mathematics". For exemple, maybe geometry is trying to make sense of some driver our brain use to interface with our eyes, and this driver better be smart about 3d space for us to have chances of survival.
So the answer would be something like it's not necessarily that the nature of the world is mathematical, but mathematics is the tiny bit of our understanding of what we have best to make sense of the world : our brain. Which we herited from evolution, hence it can be surprinsingly amazing. With this view, saying that the world is mathematical even seem a bit too self centered.
What's backing the idea on the math side of things is that thinking of math proofs as computer programs really makes sense ! There are whole theories on this see Curry-Howard correspondence and/or realizability theory. Provocatively, we can argue computer science generalizes math :-)
For a non technical covering of those ideas by someone who did world class contributions in realizability theory, have a look at Jean-Louis Krivine's last book : "Les décompilateurs". I think it is only available in french though.