I don't see anyone arguing that transsexual women should be allowed to compete with women, so I am not sure who are you directing your comment at.
Next, I would say it's undisputed that even if someone has gone through body development as male, after prolonged period of lacking testosterone, their advantages diminish and they are not really competitive with males (it seems like you might be disputing this, but not in so many words).
If we agree on the above, the question is simple: do we want to include them in sporting competitions, and if so, how would we do that?
I think we all agree on including them, and for the same reasons we've got a gender split in sports at all (to allow women to win and thus encourage them to play sports), an obvious way is to introduce another (or few) categories. It's easy to say what shouldn't happen (they should not compete against women), but the question is what should happen without outright banning some athletes or putting them in a losing proposition against men.
Btw, your premise about Special Olympics is also untrue. If you get brain impaired at a later date (or become a paraplegic), you will be admitted to competition. I am sure there are strict rules to reduce cheating.
Again, the goal should at all times be to support anyone and everyone to compete in sports, while maintaining fairness to all participants. That's sometimes not clear cut or simple, or maybe not economical, but those should be the goals we strive for.