Simon does a great job highlighting why this doesn't make sense from the technical and business standpoints: https://simonwillison.net/2023/Dec/14/ai-trust-crisis/#faceb...
It's played out over and over again, especially with Facebook. They don't deserve the benefit of the doubt, even when "logically" the ROI for the company doesn't make sense. Facebook specifically can't be trusted at the executive level either; that culture trickles down.
You can’t use net profit as a yardstick of rationality in the corporate world, but especially Corporate America. People will seek revenue that results in negative returns on investment. There’s nothing rational about it, which is the problem.
They can increase revenue by making ethically terrible decisions. Full stop.
"We're not actually transcribing every word you hear and say, we're just using capabilities that the Stasi could only dream of at rates of speed they couldn't even conceive of, and it's just that these coincidences happen so often that it looks like we're transcribing your speech. Come on, be less paranoid."
If people think "they listen to me through the microphone" it distracts them from understanding what's actually happening - the whole sequence of things you listed there.
Which means they can't take measures to protect themselves, or campaign for better practices - because they're working off the wrong mental model of how this stuff works.