Of course sugar has its own issues as well.
These days I use very little additional sweetening in any of my food. It's hard to avoid though, it's added to a lot of foods (check ingredients) and even fruit such as apples and oranges has been bred to be sweeter than it naturally was 100 years ago.
Biggest difference-maker for most people is probably sweetened drinks. Avoid soft drinks (diet or regular), sweetened tea or coffee, sports drinks, and most packaged juices.