You're not missing anything - Tor is not designed to defend against a global passive network observer.

> A global passive adversary is the most commonly assumed threat when analyzing theoretical anonymity designs. But like all practical low-latency systems, Tor does not protect against such a strong adversary. Instead, we assume an adversary who can observe some fraction of network traffic; who can generate, modify, delete, or delay traffic […]

https://svn.torproject.org/svn/projects/design-paper/tor-des...