Is it right to track the all online dealings of people that may be connected to terrorist organizations? Pretty reasonable people seem to think yes (Patriot Act).
Is it right for the government to track all the online dealings of an average citizen? Most people think no.
You may see a discrepancy here. That is why there is the saying "First they came for the fascists...". It is why when we talk about breaking encryption and backdoors discussion is centered around terrorists and pedos. Because those are universally hated groups. The problem comes with the fact that there is spill over. (See cases like San Bernardino shooting or the cop that is sitting in jail without sentencing because they know he has child porn on an encrypted drive. Btw, they use this same tactic to frame less controversial legal cases too)
I think if backdoors/searches/whatever could only be applied to universally hated groups and not include a single random citizen than there would be almost unanimous agreement that we should do that (you are not violating a "non-criminal's" rights). The problem is that technology doesn't allow us to exclusively target those kinds of groups. As tech oriented people we understand that this spill over HAS to happen, but the average citizen (outside SV) does not understand the how it works, the scale, nor ramifications. I'd like to remind everyone that 1984 wasn't about the government watching everyone at all times, but having the ability to. I'm not sure why we call that a dystopian society but many people readily accept the use of this technology by the government. Sure, the police have the technological ability do facial recognition without Amazon, but technological capability is not the question. It is SHOULD THEY?