And yet when it happens (not a rare event), it can be quite unproductive for both parties. I would hope that you deal with other worst-practices in your domain with greater grace than a total refusal to deal with and prevent pathological outcomes.
Additionally, if you truly believe what you stated, then so long as I avoid sacrificing significant efficiency/productivity to keep my words from bothering people, then I do not see why it should bother you either. If I had to self-judge, I really don't think that the added conversational padding pans out to more than 15 minutes per day. That's worth spending to make sure that the people you work with are engaged with you and the task at hand in an accommodating way, rather than fearing being rebuked at each turn.