Much of the mail is bulk (aka junk) mail sent by companies and organizations that have lobbyists. These lobbyists make it very difficult to raise postal rates as you suggest.

Add this to the Mephistophelean bargains that they have made with their unions over the years (because they too are politically powerful) and you can start to see the predicament. They're being squeezed dry.