>What's wrong with the letting the government get access to the phones of a few potential criminals/terrorists?!

Not much wrong with the FBI ruining guilty peoples' lives; it's all the innocent people they'll inevitably fuck over.

edit: Not to mention there's no such thing as a "back door for law enforcement only".