Sure, there have been plenty of attempts to distill music to a mathematical essence. Certainly the ancient Greeks tried this, and traditional counterpoint resembles math in a number of ways. But at the end of the day, mathematical descriptions of math and music theory more generally are more useful as descriptive tools to help give language to what people are doing musically and to understand why we perceive some things as sounding better than others.
Starting with numbers can be good in some respects, like understanding the circle of fifths or how scales are built out of intervals, how chord progressions and harmony work and how to reharmonize, all of which can be augmented with a solid conceptual understanding. But at the end of the day, your ear and creative spirit are your primary asset when it comes to creating good music. This is why computer-generated music has been so bad up until AI took over. Great for building arpeggiators or backing tracks, but good luck creating a beautiful melody in a purely numerical rule-based system.