If you don't want science to be attacked politically, don't politicize science.
We don't need to make sure that our scientists represent every color of the skin spectrum with a representative range of disabilities. We need objective, competent workers, with a diversity of scientific, not cultural views.
In terms of getting science done, you are almost certainly right.
However, if we want the results of science to be applied, rather than just remaining of academic interest, diversity among scientists might be important because the members of the general public and the politicians that represent them are probably more likely to listen to scientists who are like them.
Your cultural views can affect your scientific work. The imagination of children in America vs. China is different, people have different dreams and experiences, etc. I am constantly baffled by people like yourself who don't understand that cultural diversity IS diversity of ideas. Whether that culture comes from your location, race, upbringing, wealth, religion, or something else, any sort of diversity is good in business or other organization as it helps you think of new ways to innovate within your area of expertise.