The absolute answer is yes. You have a right to say it. Nobody is owed comfort.

However this clearly won't lead to a productive workplace, which is why it's not a good idea to allow these discussions. There are plenty of other restrictions within a private company to create the best conditions for the work they're paying you for. It's perfectly reasonable.