Humans are not ruminants. We can not survive on hay. We can survive on the high-nutrition parts of plants, but creating these parts is resource-intensive. In some cases the whole plant is simply difficult to grow (pests, fertilizer, etc.) and in other cases we don't get very much food from each plant.

There is a whole lot of tree attached to a cashew.

Goats and sheep are happy to eat the weeds on a rocky hill, and cattle do almost as well.

Other food animals are happy to eat disgusting waste. Pigs, chickens, and catfish are especially willing.