In case you haven't read it; see "Science and Method by Henri Poincare" - https://archive.org/details/sciencemethod00poinuoft

Here is a summary - https://thetelos.org/science-and-method-science-et-methode-h...

Here is a video discussion - https://www.youtube.com/watch?v=sQ-t8-igDZo

The thesis is that Science is made for efficiency and hence does not include everything (which is infinite). It only looks for a hierarchy of interesting facts and focuses on simple foundational recurring phenomena from nature. It then uses the language of Mathematics to economize and impose order on the complexity to make it tractable.

Finally, our subconscious prefers aesthetic attributes and hence we often find/create harmony/symmetry/beauty in our mathematical products.