Assume for sake of argument that it were possible to develop an encryption regime which protected bona fide users, but did not protect criminals. That would be a good thing.
As a matter of public policy, criminals shouldn't be given the right to communicate for the sake of furthering criminal activity without the possibility of lawful interception. It's here that some technologists lose sight of the real world. In the real world, society expects that authorities be able to fight crime using proportionate means.
The real issue is that you can't separate the two from each other - weakening encryption for criminals means weakening it for everyone.