If we decide we don't have the right to conceal our digital correspondence from the government, what else should we not be allowed to conceal? The conversations you have in your car? In your home? Besides the right to privacy, what other rights make fighting terrorism harder? Freedom of association and movement? Freedom of speech, which is regularly used for recruitment?
Finally, if they really want someone's data, in a covert way, they're already able to plant hardware keyloggers, or spy on someone as they enter a password, etc. But it all requires manpower. What backdoors would let them do is monitor encrypted communications covertly and in bulk. To build up a database of all your chats and forum posts, and mine it for any anti-current-politics sentiment.