Whether traditional search warrants or breaking end-to-end encryption violates privacy more really depends on specific cases and the details of how the end-to-end encryption gets broken. Search warrants definitely aim to seize communications and intrude into private notes, collections, offline-only meetings, affairs, and all kinds of potentially embarrassing bedroom items that were purchased anonymously.

The only real defence against government abuse is strengthening democratic institutions, respect for human rights, strong oversight, transparency etc.. End-to-end encryption is quite a shallow/weak protection since a truly tyrannical government can easily make it illegal at any time. A lot of government abuse is also possible without breaking encryption.

So I think the benefit of end-to-end encryption in protecting against abusive governments is often over-stated. But what about the costs? For example technological progress is making the world more dangerous. Should it be easy for a bunch of people to use unbreakable encryption to coordinate the development extremely dangerous genetically engineered bio-weapons from their homes? If one such person is caught should it be really impossible for 5 judges to allow the police to find all those they've been talking to?