This is very true in my personal experience, too. I can quite easily binge-eat an extra 500 calories for dinner because I'm feeling a bit too hungry, but to work that back off with exercise might take 30 - 45 minutes of cardio. With hectic work schedules, it's also very easy to skip exercise sessions, but good eating habits are "working" even at these times. Of course, sometimes we don't have time to eat well, but I find that it takes a lot less time (per calorie) to eat well, than it is to work it off.
Another point I'd make is that small differences add up scarily quickly over time, as someone else pointed out. When I was just out of university, I started putting on 1 - 1.5 kgs of weight per year, nice and slow that it was hardly noticeable. Then, just a handful of years later, I was 10 kgs overweight - it creeps up on you. The opposite is also true, a nice steady decline is easy to arrange, and gives your body all the more time to adjust to your new eating habits, without it breaking your willpower.