Livestock are part of an integrated agriculture system. Livestock and agriculture depend on one another. Cattle and ruminants, for example, convert low quality wild scrub, fescue and leftover agricultural byprods into high quality human-edible meat and nutrients.
About 50% of the corn in the US is grown not for humans or animal feed, but to produce ethanol fuel. The residual of that ethanol process is then processed into feed for livestock.
Quite a bit of the Amazon region is used to grow soybeans. The soybeans are shipped to Asian countries where they make soybean oil out of it for cooking. The residual soy byproducts from that process is then turned into feed which goes to pigs and chickens in that region.
Also mentioned in an earlier reply that about 50% of all fertilizer needed for crop cultivation comes from animal manure and byproducts. The remainder (and alternative approach) is to synthesize the fertilizer from fossil fuels.
It's important to realize that livestock aren't simply a standalone component in the food system. They are a key part of the total agriculture picture.
Food also isn't simply calories, it's about nutrition-- getting absorbable animal-format vitamins, minerals, amino acids and other key nutrients. There are quite a few nutrients which are essential to the human body and yet are very difficult (some virtually impossible) to obtain from plant-only sources. So reducing meat does come with costs/risks in terms of nutrition, I think that shouldn't be overlooked.