If there exists a technical method by which to decrypt arbitrary messages, it will be abused. Claiming you can keep this method secret whilst the world searches in earnest for that secret is unrealistic. Saying that only certain types of communication should be subject to vulnerable encryption infers that the encryption algorithm can tell the difference between those types of communication: is it OK to encrypt banking transactional communications with unbreakable encryption, but internet chat communications must be encrypted with breakable encryption? How can you tell the difference? How can you ensure that someone isn't sending communications that are evidence of criminal activities over the banking communications channels? The Algorithm doesn't know or care what kind of communication it is, and once encrypted, an outside observer can't tell anyway.
You could say "well, chat program providers are only allowed to use breakable encryption, but bankers are allowed to use the unbreakable kind". So, now you have created a protected class of people, who are allowed privacy, and another who are not. Bankers commit crimes too, but the evidence of their crimes won't be available to be decrypted. So maybe you could make it illegal for people to communicate plans of criminal activity over unbreakable encryption, but that won't stop criminals from doing it. You could ban encryption entirely, but then whole, legitimate industries, such as online banking; international banking; stock exchanges; online commerce; "cloud computing"; and managed service providers would be untenable, and wither and die.