Not having gender separated competition would probably decimate female sport participation in quite a lot of domains.
The short armed argument that you actually raise is more interesting. There are weight classes in boxing, and there could probably be arm length classes in swimming as well, but other physical differentiator that would take the place of arm length. I'm not really able to conclude on anything, but have feeling that the physical and cultural gender difference is large and important enough that it is prioritized, while arm length for instance is not.