Not necessarily. If it's the smoke and smell that bothers me rather than intoxicated antics, I might be more annoyed by someone lighting up at the coffee table next to mine than if they were just sitting there stoned out of their mind.
The stink of people, be it perfume, deodorant, hairspray, or the utter lack of washing and cleaning hair and clothes, sometimes even their breath from afar.
Well at least pre-corona. Now I don't notice it so often, since I don't share that much space & time with them anymore.
Anyway, that can even cause tears in my eyes and a strange sting in the nose, making it run. Smoke of any kind doesn't.