> Storage and processing happens on your device and it is encrypted

Microsoft controls this entire process. They could easily send copies back to "the mothership" like they probably already do with the processed results.

> It seems like the actual issue people have is that they don't trust Microsoft or they don't trust the government/people/everyone.

Whether or not this invasion of privacy is cause for concern depends on your level of trust. But it is still a privacy issue.