It’s not clear to me what you mean when you say it’s the same concept.

Your position is based on your own subjective preference rather than any objective fact.

We’ve gone from “your actions are having a direct negative impact on my health”, to “I’m not particularly fond of what you’re doing.”

Far from being the same, your position is actually very different conceptually.

The personal preferences of some aren’t valid reasons to ban things in public places for all. In most cases that would be a gross violation of human rights.

In my city the vast majority have a strong preference against pan handling. But it’s been made clear time and time again the pans handlers have every right to sit in public and ask for money, whether people like it or not. Moreover if they like they can also smoke a cigarette or bang a drum.