You can’t outwork a bad diet.
Exercise all you want, but for most people, if you eat garbage food in large quantities, you will be overweight.
I am exactly the same, btw. Most of my family was overweight when I was growing up. I was a fat kid, all the way through high school. Since then, I have been exercising consistently for 40+ years. Lifting weights, bicycling, walking every day, etc. But I still need to not just eat everything I want or I will gain weight. I try to avoid junk food, fast food, eating out, MOST days. Personally, I do one “cheat day” per week (see Tim Ferris’ Slow Carb diet for roughly the idea, although I’m not militant about the foods he says are ok, etc.).
I’m around 20% bodyfat at 5’10” in my early 60s, so I could use to drop 5-10 pounds of fat. What boggles my mind is that everyone says I’m crazy to think I need to lose ANY weight. I’ve got clearly visible fat around my middle and other areas, even if I’m not “technically obese”. I don’t look great in most clothes. But compared to the typical person (my age or not), people think I’m in great shape.
I wouldn’t say what I do is incredibly hard. But it’s also not just “do whatever you want all the time”.