We're talking about a moral issue though. If you want to argue against advocates on moral grounds, that's one thing - but how is "it's not important to me" a relevant argument?
If you don't donate to battered women's shelters, can you still refrain from domestic abuse? If you don't advocate for the protection of children, can you still treat them with respect personally?
You don't have to take an interest in animals or veganism or farming or any of that to recognize a moral argument about them. If you think that argument is flawed, engage it at that level. Otherwise, I just don't see how your sentiment matters in this discussion. It wouldn't be mentioned in any other discussion about morals.