Is math invented or discovered?
Is math invented or discovered?
So I think it is a continuum, and really fantastic mathematics will feature ideas from all the way along the spectrum: “discoveries” for the beauty, “inventions” for the problem solving, and the “in between” for the subtlety and art.
Even for physics, there are often many mathematical theories that can be used to model the same physical observations (talking about equivalent structures, not about competing theories). For example, many problems can be described equivalently using vectors, complex numbers, or linear algebra. There is a good chance that there are many (perhaps infinitely many) other systems that we haven't thought about that could be used equivalently.
So, while I agree that ultimately the structures in mathematics exist independent of our use of them, so we are only discovering pre-existing structures, I would also say that new mathematical theories are developed using a process that is more similar to invention than to discovery (i.e. you can't explore the space of mathematic theories to discover new ones, as it is infinite in every direction - you can only explore the properties of a structure you essentially invent for yourself).
Our mathematical system is an invented human language. We know that all symbolic systems with sufficient complexity are equivalent (see Turing machines.) Finding an arbitrary one to be useful and flexible is not evidence of magic.
(Yours is a "turtles all the way down" argument, I think.)