Yet they are languages according to formal language theory, a definition which I’d say is a bit more widespread than yours. Indeed what you propose roughly maps to (a subset of) context-sensitive languages in particular.
Defining new primitived isn’t really essential. What matters is what you can express with the existing primitives and grammar rules. And sheet music has been quite successful at expressing incredibly complex musical concepts just fine.
(Also there’s the fact that sheet music can contain arbitrarily complex instructions to the performer in natural language, and define new notations too. But composers usually refrain from doing that for the same reason that people don’t go around inventing new idiosynchratic words in English unless they’re SF authors.)