We're all literate here. But very few of us can sight-sing. Attempts to teach musical literacy like reading literacy on a mass scale have failed.
Or have they? Some efforts have done much better than others. I looked into a guy called Lars Roverud, crediting with reforming church singing in Norway in the 19th century. His system was a simple one based on numbers, very similar to the Chinese one briefly touched on in Tantacrul's video. It worked really well.
Then there's the American shape note tradition. I'd say that's an impressive success: regular churchgoers, not just an especially talented choir, actually learn to sight-sing three and four-part harmony. They don't just learn the melodies by heart, they can actually flip to a new song and sing it. They rely on a partially symbolic system, and on harmonization conventions.
There are actually a lot of didactic music success stories, and what virtually all of them have in common is a symbolic system for pitch/harmony.
But what those systems also have in common, is that most of the benefit to literacy is gone a generation after someone says, "how great! now you should advance to real music notation!"