I guess what people don't have a problem with there is that at least someone is physically listening in on a phone line - it requires manpower to do and thus is very deliberate.
It's when we are being data mined by the government there's a problem. I think we all don't mind being data mined for advertising (within reason) but when we are being watched to be 'kept in line' that's where there's a problem. At that point it's more like 'Minority Report' where you are being scrutinized before you even do anything.