Anecdotally:
I've been trying to lose some weight, and one of the first things I moved to do was drop drinking (as most resources on weight loss will tell you to do). It was very tough to do because I love my beer. I found the process was making me miserable.
Sure if I leave the beer out of the equation my calorie deficit would grow, but including some beer still keeps me at a deficit regardless so that I still am losing 2-3 lbs a week, and very much happier all the while then if I was losing more and more quickly.
edit: except for the smoking. That's a slippery slope.
Every now and then, I indulge in some ice cream or some sushi. Slow progress is better than no progress. I know I am better off without alcohol, weed, or tobacco though.