What you say is perfectly logical, but subtly flawed when it comes to the USA.
If you're poor, odds are that you're eating fast food. The primary cost of fast food is not the cost of the raw ingredients, it is the time and effort to make the fast food. Furthermore many of the customers there see large portions as a value. Therefore fast food places have found it worth their while to offer large portions at a low price, and to inflate those portions with the cheapest foods that they can - french fries and soft drinks.
As long as fast food places do that, there is no real cost difference between 2500 calories and 3500 calories. Given how addictive most people find food that is high in sugar, fat and salt (which describes soft drink + fries), people tend to finish the portions that they are given. That will make you obese fairly fast, even if you don't "supersize" your portions.
Food banks often offer similar foods to fast food places because it is cheap and what their clientele is used to. This causes similar consequences. So in the USA even if you have no money at all, it is easy to get fat.
If there were cheap, convenient places to eat that made you pay according to calories and were willing to give realistic but smaller portions for significantly lower prices, then your reasoning would be absolutely correct in the USA.
Now you can argue that the underlying cause that makes this possible is that the country is rich enough that we can afford to be wasteful even on poor people. This is absolutely correct. But for practical purposes that detail doesn't matter. If you personally have little money and live in the USA, it is really easy to become fat.
An incidental note on exercise. Exercise burns calories. Obviously. However it takes very little extra food to offset the calorie expenditure from moderate amounts of additional exercise. Luckily exercise also affects appetite for many people, and muscle mass increases your basal metabolism. Therefore exercise is a win, but for moderate amounts of exercise not mainly because it directly burns off calories.