(This may be a good place for me to clear some misconceptions I have about farming; treat this comment as an application of Cunningham's Law.)
> The thing is that soil used for grazing cattle can’t be used for planting vegetables efficiently.
Not planting vegetables efficiently != not planting them at all, right? Could the lower yield on such fields still come ahead of lower energy efficiency of meat (in terms of energy expended per calorie of final human food produced)?
> So the effect would be to reduce the area of where we can produce food.
Surely we can grow something human-edible there. Or could perhaps this land become earmarked for construction, while equivalent amount of very fertile land could be switched from construction to farming? AFAIK there's plenty of settlements built on land that would be better used for farming.
> And if you would remove cattle you no longer have natural fertilizer.
Is industrialized agriculture even using natural fertilizer? I was under impression modern fertilizers are made in chemical plants and their production doesn't involve animal excrement.