If you believe in 'meritocracy' and rational behavior you can't suddenly 'choose' to ignore the real risk to women and their careers from men who hold such views. Even with a small proportion of such men and there is no way to tell the numbers, there is a huge risk women will be discouraged right at the start and face a possibly hostile environment.
Research and life is difficult enough and women will drop off or be discouraged. And this will then further feed sexists more data points to justify their positions making a neat cycle of discrimination and 'scientific data'. The large number of men who continue to defend these regressive and unscientific positions proves beyond doubt the huge risk women in professions continue to face with them in it.