The main casualty would be mailing lists, which would likely have to move to forums or subscriptions. It might be technically possible to also say "I accept mail from X without demanding payment" but that's not necessary to make a pay-for email market work, and have much less spam.
A large list may have hundreds, thousands, and potentially millions of recipients (though I'm not aware of any such at present). The Debian Project provides email subscription stats, with its biggest list being debian-security-announce, at 34,234 subscribers. That's a cost of $342.34 per message transmitted.
https://lists.debian.org/stats/
It would also be a large overhead to any system with a large userbase that relied on email based notifications. Facebook's financials put its per-user revenues at around $8-$10/year, which would be completely eaten up at the rate of only 3 emails per user per day.
The thing about email is that it's really, really cheap. That's the benefit and the detriment to the system. What you want to do is make the costs asymmetric. Cheap for legitimate and white-hat users, really expensive for the black hats.
And your last point is really just a specialized form of a web of trust anyway.