I'm not sure I understand your position. Do you believe there any possible opinion is legitimate, and that there is nothing that someone could believe in that should get them labeled a hater and socially ostracized? Should we feel a need to become friends with klansmen to prove that we are open minded?

And if not, then how do you draw the line? If there are people who find your own opinion abhorrent or othering or whatever and wish not to associate or discuss with you, by what authority are they wrong? Do you personally feel the need to debate every opinion you hear? How many times should you defend your own opinion before it becoming acceptable not to want to discuss it again?

And just to engage a little bit with your particular opinion, I for one am immediately suspicious of any argument that says "non-sexual non-violent behavior X is acceptable, but not around children". Of course, I am open to the idea that in principle you may have some compelling arguments that I haven't heard before. However, I don't think it's very likely, so my prior would be that you are indeed not very trans friendly. If I were trans, I would be quite inclined to avoid you because of that, and very disinclined to debate this particular point with you in any setting where a more hostile discussion might reflect poorly on me (say, in the workplace). I don't think this reasoning is overly emotional or thought-ending. It's a rational way to respond to speech that may become confrontational.