So it’s only immoral if the number of the animals is really low? Seems like an arbitrary trait to determine whether animals deserve to live or not
This is a rather obtuse outlook on your part. It should not be difficult to see that these positions on killing animals are not logically equivalent, or logically exclusive of each other.
Even forgetting about any concept of morality or ethics for a moment, from a purely utilitarian viewpoint raising animals for food isn't mutually exclusive with wanting to preserve the balance in an ecosystem in which whales play a part, and whose disruption could have far-reaching implications.