It would also kill several small businesses that rely on junk mail as their only affordable advertising medium (e.g. think Pennysavers).
We could jack the price of a stamp to $5, it would still be a reasonable price, and nobody but the stingiest of cheap people could possibly complain. If mailing something isn't worth the price of a hamburger to you, then you shouldn't be mailing that something.
People who are running businesses that involve sending legitimate mail cheaply will just have to start collecting larger shipping fees. It needn't affect anyone shipping anything but small things.