This has puzzled me for decades. There are some sports in which men and women compete together at the highest levels (e.g. horse racing), and there are some efforts at other mixed competition at lower levels. However, these seem to be minimized by everyone, and not even referenced as a benefit of which we'd like to see more. I can't think of any arena besides sports of which that could be said, although perhaps it might almost be said of the military. Even the military is criticized to an extent for the opportunities it denies women. Yet, sports get a complete pass.
It is true that many sports would have to change in fundamental ways to allow women and men to compete together at the highest level. That isn't really an excuse, though, is it?