Agreed, and I think that's entirely deliberate. Using the existence of DSDs to try to discredit the sex binary and then leverage that into dismantling sex-based protections.
My mental scenario is one of these guys trying to go into the women's restroom they are now legally obligated to use, and a whole bunch of bystanders bringing violence to the table to prevent it.
That's the desired outcome for them. They want us to either die or comply.
https://en.wikipedia.org/wiki/For_Women_Scotland_Ltd_v_The_S...
Though in practice I wouldn't be surprised if we eventually converge on "women only" and "unisex" as some sports have done. The risks posed by self-identification are very much one-way; this isn't a symmetrical situation.