That's too simplistic, since it is in fact very common to be concerned about animal welfare even among people who do eat meat.
The argument goes that livestock have a capacity for suffering, but killing them for meat without causing them suffering is ethical.
You can disagree with the position, or with its implementation in practice, but it's a consistent position in principle.