The downside is that if you commit a crime and there is a nexus with the mail, you’re facing a serious felony. Politicians submit ethics paperwork via hand delivery for this reason.
Letters cannot be opened without a warrant, but it sounds like they could be subject to scans.
In general though, almost all laws (especially with respect to privacy) were written with physical privacy in mind, not virtual. The statutes around opening a letter is much more well defined those around recording unencrypted emails.
I once worked for the USPS in a Middle Eastern country, and it was our job to assist local customs officials as they scanned / inspected every piece of inbound mail.
They ran each piece through an x-ray, and whenever they spotted a banned item (liquor bottles, sex toys--hell, they even inspected optical discs to make sure they weren't porn), they opened the package and confiscated it.
We cleared about 1 mail truck of mail every night, and it took 5-10 hours. No way that's sustainable on the scale of domestic mail throughput.