The USPS does a lot more than ship packages and letters. They are also the government's mass printer. That VA letter and social security check probably wasn't printed in a Virginia suburb, but at one of the regional USPS printing offices. They also handle very high security items, not just to you, but between government agencies under conditions that would make the warden of a maximum security prison blush. The term "going postal" isn't about some guy at your local post office flipping out over the number of people waiting to get their packages weighed, but about some guy who has spent decades wondering if the guy standing next to him is a postal inspector just itching to see him screw up.
To put it another way, they offset a lot of other agencies costs (at least in overhead). I'm certainly not saying it would be better or worse if all of this were handled by the private sector, but I will say it would require a lot of restructuring and make someone a huge pile of cash. Still, it is going to happen sooner or later. Politically it is just too sweet.