I'd rather live in a world that preserves the fundamental right to privacy than one that can automatically organize my housewarming party.
An employee can blackmail another person, but the model simply has no reason to, or am I misinterpreting the "whys" of needing privacy here?
The concern isn't judgement from the AI, but that products from the model trained on your data could expose sensitive information.
Since it's never quite clear exactly how the data could be used in situations like this, there's a chance that very sensitive data could be parroted back to people who were not the intended audience.