By peer to peer exchange, you mean an exchange built on blockchain?
One exists that I know of (bitshares). I'm sure others are being developed.
Turning fiat into crypto is a separate problem of its own. The problems there are legal and political, not so much technical.