S/MIME also neatly solves the problem.
The problem is you don't have public keys for people you send to.
And there's not a reason for many to get the keys.
I wish I could say "I'll read your unencrypted email tomorrow" (and delay it from getting to my inbox).
Then we get to argue whether NSA can get bogus valid certificates from the commercial CA's... Of course you could roll your own CA but then both parties need to trust it.