If we want to preserve these rights we can't just appeal to their utility for the individual.
Free speech affects my ability to improve society. Right to privacy affects my ability to cover my own ass.
Without protection of privacy - very few people will exercise their free speech when things get hard.
That said I understand people who are afraid that loss of privacy would be asymmetric but I don't think that would happen. Celebrities and public figures will always have less privacy than everyone else.
If Trump wants to know your tax returns - he will. You don't know his, no matter how much you want to.
And anyway, Trump insist he could murder someone and give himself a pardon. You can't. So, even if the information flow is symetric - the consequences aren't.
Actually this isn't the most important reason for privacy.
Privacy and other basic rights do very little for me as an random law-abiding dude. I don't break laws, so don't have any reason to cover my ass. I don't have much to say, so not being allowed to express my opinion doesn't seem like that big of a deal. Even in the doomsday scenario where evil totalitarian dictator takes over the country and mass-surveils everyone and otherwise stomps on our rights, it probably won't be that big a deal to lowly me.
The most important reason for protecting individual privacy is to protect the future of society and civilization itself. The average dude in Nazi Germany and Pol Pot era Cambodia probably barely noticed his loss of individual rights as far as daily life was concerned. Mass surveillance enables mass control, and the worst atrocities in history have all happened in environments where individual rights - including privacy - were disregarded. It's bigger than just me. It's about protecting society itself and giving my children a decent world to live in.