Most crops are consumed by livestock, not humans. So by eating crops directly you save more animals than just the ones you don't consume.