Ever since I started learning something about music theory (just in the past couple years... I'm far from an expert), I've realized that both sheet music and the piano layout are both very clever in unexpected ways that, as you point out, make the music notation both expressive and more compact than a straight timeline of linear note values, because they lean on the fact that sections of music tend to skip very predictable parts of the range of notes. They tend to use them in particular patterns that make it useful to reduce your focus to a subset of the available range at any given moment.