That is already the case. If you whisper a secret into my ear, the government can subpoena me and force me to tell a court what you said. They can force you to tell a court what you said so long as it's not incriminating to you, and even then they can do it if they give you immunity.
We can debate about what the law should be, but for the last several hundred years, the law has not really contemplated people keeping secrets from the Government. "Privacy" as it has been understood to date means protection from the government fishing for evidence without probable cause, not an absolute right to keep secrets.
Pervasive, unbreakable encryption is a game-changer that requires rethinking the existing framework. We're not just talking about not being able to get data from a terrorist's phone. We're talking about the bread-and-butter of many sorts of criminal prosecutions being opaque to the government. Good luck convicting someone on insider trading when all the relevant communications are opaque to the government.
I happen to think that the benefits of encryption outweigh the challenges to law enforcement. But its disingenuous to pretend that the government is trying to "take away" a right to "keep secrets" that you already had. Our whole legal system is built on being able to get whatever evidence is relevant wherever it may be found, with extremely narrow exceptions.