>But it's also totally fine to try to start a conversation with someone.

I would agree (and really wish I could) if most people would recognize disinterest and stop there, but anecdotally I'd say half the time it's met with anger or some form of pointed dissatisfaction. People have outright touched me to get my attention about some off-hand comment I ignored initially. It's frequent enough where I'm conditioned to start looking for an exit when I'm approached by someone in public.

It's incredibly unpleasant to feel obligated to continue small talk because you fear what happens if you don't. I can not support any kind of small talk under the reality of what comes along with it. It's one of those cases where some people ruin it for everyone else.