I appreciate where you are coming from. And I agree we should, "strive towards a world in which we stop making exactly those kinds of distinctions and instead judge people on the content of their character and the value they can create." And I honestly believe that diversity and outreach programs are part of that process.
The economic and social structure of the United States is the result of historical processes that were driven by racism and sexism; disparities in wealth and power between these groups continue to exist today due to that legacy.
To take one example, "If average black family wealth continues to grow at the same pace it has over the past three decades, it would take black families 228 years to amass the same amount of wealth white families have today." [0]
While it is a great sentiment that everyone should be treated purely as individuals and it is something to strive for, it completely ignores the economic reality of our country and is an ahistorical approach to public policy or the construction of a hiring pipeline.
We can argue about to what role public companies can or should play to reduce these historical inequities. I think that outreach and education at all levels to increase the number of candidates from underrepresented groups in the tech hiring pipeline is a great step and that reducing these inequities, beyond being ethical, will create a stronger workforce. Others might disagree and have good reasons to disagree. That is a conversation worth having, but it needs to happen in the historical context in which we find ourselves.
Writing a manifesto that claims that the disparity in tech is the result of biological differences that lead women to be inferior at performing in tech and leadership capacities is not the way to start this conversation. In fact this is the exact kind of sentiment that you call out by saying, "I personally find it inflammatory to equate the concept of diversity with differences in sex and race in the first place."
[0] http://www.ips-dc.org/report-ever-growing-gap/