>It's much more efficient to grow food where it grows best

Food grows best (when measured in both biomass-per-acre and biomass-per-man-hour) in forests, not monoculture fields. This becomes pretty obvious when you compare the percentage of sunlight that hits bare soil.

The only trick is to figure out how to design a forest (a non-trivial task by itself) using species that produces edible biomass, and arrange everything such that it's easy to harvest.

One example of forest gardening: https://www.youtube.com/watch?v=GFbcn06h8w4