I mean, if you're happy with your Neo-Atkins Diet, that's cool, but enough already with this looking down on people who don't eat as trendily as you do.
Also, keep in mind that modern grain agriculture is 100% dependent on fossil fuels.
Back in the day, animals were raised on the farm along with vegetables. The animals ate the cellulose (inedible to us) and scraps and created protein. They also created fertilizer (today it is synthesized from fossil fuels and wreaks environmental havoc - the Dead Zone at the mouth of the Mississippi is mostly caused by nitrogen runoff from farms) and ate bugs, so there was no need for pesticides.
Of course if people wanted to eat much less meat, and if there were a lot fewer people, things might work better sure.
Ideally, animals should be raised either on land that's not suitable for agriculture, or on the farm, eating the inedible plant parts and providing more of a closed-loop ecosystem.