In the US this is covered under the Pen Register Act: https://en.wikipedia.org/wiki/Pen_register. Essentially the fact that two people have communicated is not protected under our Constitution; only the content of that communication is protected. In that context, Congress passed the Pen Register Act, which requires law enforcement to get a warrant to monitor who calls who; the bar for these particular types of warrant is particularly low.
The Patriot Act extends the concept of pen register and their required warrants to internet communication. The government specifically is required to get a warrant before it can ask an ISP, for example, to reveal who someone emails or what web sites they visit. For now, they need a warrant to know that you visited Amazon or your library's site, but they need a harder to get different warrant to know what books you've bought or checked out, or what you thought about those books when you emailed your friend.
The fundamental problem with pen registers and the internet equivalents is that it brings people under government scrutiny that otherwise would have escaped notice. If person A is a person of interest, and the government is pen registering his communication, then that makes everyone on person A's communication list a person of interest. Now the government gets a warrant to pen register person B, someone who person A communicates with. That means that everyone that person B communicates with is now known to and watched by the government, even though they are not specifically the target of any investigation or warrant.
Person A may be a drug dealer, person B may buy from person A, and I, person XYZ, may be a friend of person B, don't buy or use drugs, and don't know a thing about the relationship between person A and B. But now the Eye of the government has swung its gaze over to me. I could become collateral damage in, for example, a plea bargain negotiation. My house might be violently raided by law enforcement, merely because Person B visits me a lot; my child might have a gun pointed at his head, and my dog might be routinely killed merely for getting in the face of one of the law enforcement home invaders.
This (the pen register) is a violation of my desire to not be scrutinized by the government, whether I've done nothing wrong or not, whether I have anything to hide or not. I in fact have done nothing wrong and have nothing to hide, and I still do not want the scrutiny of the government to fall on me. The government is in theory my servant, not my master. That relationship naturally gives me the right to expect the government to leave me alone.