Your credit cards can be turned off and bank accounts frozen. If all your money and payment ability is digital and controlled by a central authority, your ability to transact can be turned off with the flip of a switch. And in the complex world we live in, that can literally mean that your ability to keep eating can be digitally switched off at almost no cost to your attacker. Is that something that can be continued? Maybe not, but the attacker could drag out the issue in a bureaucratic and legal mess for long enough for you to fall into homelessness and despair if they're motivated enough to destroy you.
Cash has the nice property that it enables permissionless economic transaction. We can exchange cash peer to peer without anyone's explicit or implicit consent. And being private is also nice because it gives the people you transact with some protection from potentially aiding some kind of blacklisted individual. I wonder if people like Harriet Tubman and the people that aided her would have been able to do what they did if their money could have been turned off due to their illegal actions helping slaves escape.
Permissionless and peer to peer cash is a critical feature of a free society that we should be very fearful of losing.