Ultimately, you just have to let go. Every possible action you take in your life could be taken advantage by criminals. Do you want to let that restrict you ability to lead your life? I don’t.
So perhaps the ideal is a sliding scale. The more powerful you are then the less anonymity/privacy you should be allowed, at least in financial or political matters.