I feel that's a common misconception. Sure, you can't buy fresh, organic foods but I'm able to build muscle on about ~$10/day and stay healthy as a horse.
Frozen peas (most frozen veggies) are $1 for 4 servings; canned black beans and chickpeas are $0.89 for 4 servings; whole-wheat bread is $3-4 for 18 pieces; 1 gallon of water is $0.89 (if you faucet water isn't drinkable); etc.
You certainly can eat healthy on the cheap. It's just that most people don't want to make the sacrifice.