Consider:
1. A phone call in which Mrs. Smith talks to a receptionist to set an appointment with a doctor for 9:30 next Wednesday.
Vs.
2. Knowing that Mrs. Smith called an abortion clinic.
#2 seems like a bigger violation of privacy. Metadata is the real data.