I don't see gambling as having the same kind of direct harms to others as drunk driving or second hand smoke.
If I gamble and I lose all my money and go bankrupt, that still doesn't necessarily harm others or even massively increase their risk of harm. Can't say the same for the other two actions.