You're thinking from the perspective of a government operative (or another human) snooping through your data, in which case messages are more important. But most of our privacy is destroyed by computers sifting through the mountain of data that companies collect on us.

Your cellphone constantly broadcasts your location to your carrier and to Facebook and other apps, if you have them installed. If you analyse this data, it is possible to make out where you work, where your home is, where you like to hang out (and with who), and where you take vacations. Further, it is possible to identify a person uniquely from just 4 points of location data if you have a database of everyone's location data. As former NSA general counsel Stewart Baker said, “Metadata absolutely tells you everything about somebody’s life. If you have enough meta data you don’t really need content.”

For more uncomfortable and scary revelations like this, I recommend checking out Data and Goliath by Bruce Schneier.