That's not true at all. One can have a deep understanding of all the elements of music you mentioned without knowing any traditional notation. Maybe it's understood in terms of modular arithmetic and fractions and visualized as such, or visualized in terms of a piano or an abstract spiral structure or not even visualized at all. Traditional notation is only one possible representation (an a fairly arcane one at that); others are certainly possible and don't preclude deep understanding.