Smoking is kind of a bad example here. Smoking has obvious secondary effects, i.e. secondhand smoke. I have no problem with tobacco smoking being banned from shared office spaces and other common public spaces. Sure, maybe make it possible to get somewhat easy permits to allow for things like smoking lounges, certain kinds of bars, etc. but overall I still consider it a positive that nobody is smoking at my office, movie theaters, practically all restaurants, on the train, etc.
You should have the right to inhale whatever the hell you want. But you don't have the right to force me to breathe whatever you're wanting to inhale.