I gave up sugar a few years ago and there was pretty much no effect. I didn't have much sugar before, and there are plenty of other sources of calories.
Calories in calories out.
Every time someone says that, the people with a hobby-horse to ride come out and beat on the person for a bit. Especially if they imply that honey (no, sorry, the correct term now is HFCS) isn't the Devil's own temptation sent to corrupt and impurify our precious bodily fluids.