Hacker News Reader

Comment by bryanlarsen | Hacker News Reader

A large part of the problem is the failure to distinguish between the aggregate and the individual. For the sake of argument let's say that the population of individuals who would make good engineers is 60% male and 40% female. From an aggregate level that's a huge difference. Once you gather enough of people that statistical inferences can be made, the difference becomes obvious. In a classroom, for instance. But at an individual level that difference becomes insignificant. If you grab one boy and one girl at random, there's a very significant chance the girl is better at engineering than the boy. At the individual level, there are much better predictors of whether a person is good at engineering than their sex. Five minutes of conversation wouldn't be authoritative but it would be much more predictive than the individual's sex.

View on HN

That really is the crux of the matter. Everyone should be given an equal chance with no prejudice, and then if population differences still emerge, that's OK.

Unfortunately there isn't much room for this middle ground. People who want to acknowledge sex difference want to use sex as a filter, e.g. "no women in combat roles." People who want to give everybody a chance also want to even things out, e.g. "Allow women in combat roles, and lower the standards for women so they have just as much of a chance as men have."

I hesitate to bring up extreme cases, but the relationship between social darwinism and the rise of the nazi party make many people, including myself, very hesitant to speak about differences in sex etc. Not because we deny their existence, but because we've seen how far large groups of uninformed electorate can be persuaded into supporting some scary stuff because of it. It's just a very tricky subject when it comes to the masses.

EDIT: Clarified to social darwinism from darwinism.

It's not the masses you need to be worried about, hiou. It wasn't the masses gassing people by the millions... it was the elites, who got a few people to go along with them. "The masses" organize poorly... it's the people who are smart enough to successfully do something with their bad idea, but not smart enough to see through the bad idea, that must be feared.

The deep-rooted fear of academia isn't that the masses may do something stupid with the idea of gender differences... it's the fear that they, the academics, will do something stupid. (Again. Although, not the same literal people, of course.)

Did you really manage to blame academics for the Holocaust in a mere four sentences? That's really impressive!

It's not just "academics". It's anyone who can take an idea and successfully run with it, without stopping to notice all the signs that are yelling at them "Stop! This isn't working as you expected!" Academia merely happens to have a large concentration of such people, for structural reasons. (The "ivory tower" is a real effect, and it is both a good thing because it permits concentration and focus, and a bad thing, because it begets people too stuck in their theories and smart enough to "explain away" anything, until it is much too late.)

This is not something "the masses" do, because "the masses" do not have the ability to successfully run with an idea and gather enough power to impose it.

It's not just the holocaust, either. There's a huge history of 20th-century "social engineering" that rather a lot of people would rather forget, because it pretty uniformly went badly. See also "eugenics" for another reason that academics are afraid to think too hard about how people may be different or how genes may determine things about people... and not entirely without reason. "The masses" did not impose eugenics. In fact a lot of eugenics had to be hidden from the masses.

With power comes the power to screw up. Academics have a lot of power. It would be strange if they'd never screwed up.

Academics have approximately zero power. This is the craziest conspiracy theory I've seen in a while, because it's just so obviously wrong. Oh yes, college professors need to be kept in check because otherwise they'll go out and kill all the Jews. WTF?

Go research where Lenin came from. Go research who did eugenics, and where they got their ideas from. Go research the origin of a lot of the Nazi figures and where their ideas came from. (Not just Hitler.)

Express all the humorous strawmen you like. How do you think these things happen? In the 20th century, the only bad social engineering event I can think of that could even remotely be considered the outgrowth of a broad movement of the masses is sort of Nazi-ism. Everything else was led by somebody with an idea from academia. "The masses" don't originate many ideas, and in the era of the all-powerful State (which the 20th century is firmly in), "the masses" don't run around committing genocide, or doing any of the things they may have done in previous social systems, because all that power is now reserved to the State.

And the State and academia have always been attached at the hip, as they are today. How else would it be? Do you think it's some sort of bizarre coincidence that the dominant ideology of academia today and the dominant ideology of the State are exactly the same? Of course they are... they're causally connected. (Where do the "technocrats" in technocracy come from? Certainly not the farm!)

As for this being "conspiracy" theory... no, it's just an understanding of how the world actually works. It may happen to explain this opinion of mine, but that's only because as a ground fact of the current world it explains lots of things. Academics wield power through the State. It's clear as daylight... it would be some sort of bizarre theory that they don't. What would that even look like?

Oh yes, the dominant ideology of academia is the same as that of the state, which is why academics universally agree that anthropogenic global warming is real while the government can't even agree on whether climate is changing at all, let alone the cause. This must be why universities are chronically underfunded and professors are constantly scrambling for their next research grant. This must be why roughly nobody in the top levels of government has so much as a PhD. The tight link between the two explains why governments persist in outlawing drugs even though research shows it's more effective to legalize them. I could go on, but I think you get the idea.

Looks to me like Lenin graduated from university but didn't go anywhere near an advanced degree. I'm not sure which Nazis you're referring to but I'm not aware of a particularly large number of university professors among the ranks of the top Nazis.

I agree with mikeash on this. It's not the academics that are committing crimes. And anyway, I'm tired of people telling me whether a certain idea is right or wrong. I just want to hear the evidence, and I think that's how most "academics" are.

You don't understand, recognizing sex differences leads to industrial-scale mass murder, just like it has in every human society that has ever recognized sex differences. Wake up sheeple!

No No No! I thought it was NOT recognizing sex differences that leads to industrial scale mass murder!

Now I'm confused. Not really, because this thread is nonsense -- confirmation bias mixed with absurdity.

"Academics have approximately zero power."

Sure, academic don't rove about with black helicopters and guns, but academia has the ear of industry, politics and government for good reason. Applied academics has the ability to predict outcomes, grow economies, win wars and generate massive revenues.

But it also has the ability to go horribly wrong. It's undeniable that Nazi eugenicists modeled their ideology after the American eugenics movement, encouraged by American academics and NGOs.

https://en.wikipedia.org/wiki/Nazi_eugenics#Origins_in_the_w...

Nothing is as powerful as an idea whose time has come. Even a bad idea.

Some academics have the ear of powerful people. Some don't. Like in any other field.

Academia tends to be where far-out ideas come from, just because it's an environment that encourages it. Some of those ideas will be discovered by people in power and used to shape policy. Other will not. To blame academia for this or point to this process as evidence that academics are powerful seems an awful lot like being afraid of language because language can be used to convince people to do bad things.

I agree. Language is just a form of communication, it can be used or misused.

It would be a fiction to downplay the power of academics in modern society. Academics has the power to mold and frame the minds of the next generation of leaders, alter government policy and to influence an entire populace from childhood.

I see your point that academics has been demonized at times, however I think it's more appropriate to judge it on outcomes.

What is the relationship between darwinism (what a ridiculous term, by the way) and the rise of the Nazis?

I think they're talking about social darwinism. The idea that "survival of the fittest" can be naively applied to politics is a pretty easy way to convince people to commit genocide.

https://en.wikipedia.org/wiki/Social_Darwinism

It appears you're right. However, I find it puzzling that it's mentioned here, since the gulf between "there are some genetic differences between the sexes" and "let's attempt to improve the human gene pool by murdering any group we find displeasing" is so great that they barely have any relationship at all.

It's not about murdering people. The problem is, if you support policies that disadvantage people based on their genetics, you can gradually dehumanize those people. The (real) disadvantages of that outweigh the (supposed) advantages.

OK, but the gulf between that and "there is some genetic differences between the sexes" is still so large that you'd need a powerful telescope to see from one to the other.

And people with sexist and racist tendencies have those telescopes glued to their face.

As much as I disagree with your overall viewpoints, I'll agree on this one you have here. For a lot of individuals, the gap that you mention is huge such as to not have an immediate, explicit effect, and for one particular reason:

Something as simple as validating and completely "accepting" the notion that "there is[sic] some genetic differences between the sexes" requires individuals to throw out entire swathes of subsequent ideas/notions/etc, if we were to be logical about things. I.e. You now have "Men and women are equal but ..." or "Men and women are equal except when...", instead of the pure and logically consistent "Men and women are equal".

If you ask me, the entire thing (sex-differences topic) is starting to smell full of tiny errors and corner-cases. Perhaps we're on the verge of a paradigm shift happening once people clarify their ideas, without exceptions and acceptable-errors:

https://en.wikipedia.org/wiki/Paradigm_shift

Equality isn't about people being literally identical. Nobody actually believes that. There are troubling facts like how men as a rule have a really hard time becoming pregnant that nobody denies.

Equality is about equal treatment. It's about judging the individual on their own merits, not some group they happen to belong to. "Men and women are equal" doesn't mean women are equally muscular and men can have babies. It means that when writing a law, selecting an employee, or counting a vote, you don't change your approach based on whether the person is a man or a woman.

There is nothing wrong with saying, for example, most of our warehouse workers are men because more men are able to do the heavy lifting required. There's nothing unequal about that. What equality demands is that you never say, I will not consider you for a warehouse job, regardless of your actual strength, because you are a woman and all women are too weak for it.

It's not actually that complicated IMO.

But if you are disadvantaging people that have objectively inferior genetics? Do you actually lose anything? Otherwise, are those people taking advantage of medical advances that prevent their deficiencies from being fatal?

What does "objectively inferior" mean?

Celiacs, for instance, seem to be a group that in previous generations succumbed to their condition, if not directly than indirectly because of general unfitness. To be honest, as a survivor of childhood lymphoma, I ought to be dead, had I been born even 25 years earlier. I question whether my genetics have enough offsetting benefits to counter that predisposition to life-threatening cancer for any of my offspring.

Advanced medical care is part of the environment in a first-world country now. I'm sure a fish would think us genetically inferior for our dependence on air to live, but we don't see it that way. A dependence on advanced medical care is ultimately no different. Assuming civilization doesn't collapse, I'd expect various dependencies on medical treatment to gradually arise in the population, but there's nothing objectively wrong with that.

There's also the eugenicist argument that the undesirables shouldn't be permitted to procreate.

For example, Margaret Sanger didn't want to round up people and exterminate them but she did want to keep them from reproducing.

Given that procreation requires one man and one woman to participate, it's going to be really hard to practice eugenics based on genetic differences between the sexes.

Have you ever heard of sex-selective abortions?

Did you miss China during the "one-child" policy era[1]?

[1]https://en.wikipedia.org/wiki/One-child_policy#Effect_on_inf...

Sex-selective abortion is not eugenics. Nobody with even the most cursory understanding of human reproduction thinks that you'll permanently alter the gene pool by selecting one sex over another. This is totally unrelated, and I'm rather sad that I managed to get not one but two replies bringing it up, as if it were somehow relevant. The quality of this discussion is just astonishingly bad.

Beside selective abortion and embryo selection, it's already possible to derive gametes of both kinds from stem cells of arbitrary sex in mice, and it will most likely be possible to do the same with human cells soon.

Mice that descend from two males or two females have already been created in the lab.

You still need wombs, but they can be grafted and will probably be farmed from stem cells in the future too.

https://en.wikipedia.org/wiki/Uterus_transplantation#First_s...

And (at approximately same time) rejection of darwinism has stopped soviet regime from inducing famines, using slave labour and purges. Except it did not.

I think clear-headed thinking along with a moral foundation of compassion for everyone is going to work out better in the long-run than attempting to skirt around possible truths deemed 'dangerous'.

4th comment from the top of the thread and we've already reached Godwin's Law.

Have you ever wondered how the electorate becomes uninformed? Certain information is declared off-limits because of what it might make people think or do.

Don't read below the parent comment, the thread has devolved into an argument about Nazis.

https://en.wikipedia.org/wiki/Godwin%27s_law

> Everyone should be given an equal chance with no prejudice,

Agreed. But the way it tends to work is that if (say) 60% of all engineers are men and the rest are women, then a man more closely fits the idea of an engineer that those people who are in charge of giving the chances have in mind, and so they're more likely to prefer a man to a woman.

You sort of run into this chicken/egg problem though: are women underrepresented in engineering because of what I described above (bad), or do they actually have equal opportunities and simply aren't choosing to become engineers (okay), OR do they actually have equal opportunities and are simply not choosing to be engineers just because the profession currently comprises mostly men and they feel discouraged when they see that, even though they're otherwise interested (bad), or etc. etc.

> simply aren't choosing to become engineers (okay)

It is not certain that this choice is OK, though. While the causes of gender disparity are undoubtedly complicated, there seems to be some persistent factor in engineering & technology that results greater disparity than in other fields like medicine and law. It is unlikely that current participation levels are at their natural equilibrium; the gender distribution was more balanced in the past than it is today. The problem is not merely that women may "choose" not to enter (or that they choose to leave and not return) engineering, but that the status quo of the community might be actively driving them away. Engineering & tech have a reputation for allowing behaviors and personalities that would not be tolerated in other fields that are more gender balanced.

Finding the natural gender distribution in tech will require actively changing the culture to make it more inclusive and inviting, not merely removing explicit barriers to entry for women.

Yeah, I agree. Whatever the causes are, the best thing we can do is make the tech culture such that as many people as possible feel included as long they choose to be a part of it, so the decision to pursue it or not comes down to personal interest and nothing more.

Engineering and tech have a reputation for tolerating people (with "non-normal" characteristics), so we need active change to be more inclusive...

> Unfortunately there isn't much room for this middle ground. People who want to acknowledge sex difference want to use sex as a filter, e.g. "no women in combat roles." People who want to give everybody a chance also want to even things out, e.g. "Allow women in combat roles, and lower the standards for women so they have just as much of a chance as men have."

Actually, I would say the vast majority of people not currently writing a post on an internet forum fall in some middle ground there. Like, the vast, vast majority.

That really is the crux of the matter. Everyone should be given an equal chance with no prejudice, and then if population differences still emerge, that's OK.

Yup. One of the things that annoys me most about prejudice and such is that it is inefficient. For a well-run and rich (economically, socially) civilization, we ideally want everyone to excel to the best of their best ability. To find the best use for everyone's talents. We want everyone to find their own niche, irrespective of their gender, race, orientation, etc.

Yes, but more and more, courts and laws try to enforce "equal outcomes" and not "equal opportunity."

Where do they do that? Even the notorious punching bag of Affirmative Action is intended to make up for prior bias, not force equal outcomes regardless of all merit. (Whether it actually does that is another question, but "try to enforce" is all about intent.)

Let's start with this NYTimes report about allegations of bias on teacher's qualification tests, because of the disparity in results:

http://www.nytimes.com/2015/04/08/nyregion/questions-of-bias...

You probably assume that the test is unbiased unless proven otherwise, and that any discrepancy is therefore likely to be actual differences in the underlying populations. The court is merely assuming the opposite: that discrepancies in the test results are a result of bias in the test itself unless proven otherwise.

They aren't trying to force equal outcomes, they're trying to make sure that if a test produces worse results for minorities then it had better be because of actual differences in the population and not subtle bias in the test.

I've literally never heard the latter argument made by any proponent of women in combat roles. The closest I've seen is calls for the requirements for certain combat roles to be reviewed for appropriateness to the real requirements of the specific role; most proponents of relaxing blanket restrictions on women in combat roles I've heard will explicitly accept that, for most roles, any reasonable set of standards will still make it easier for men.

Where have all the different standards for men and women in the armed forces come from, then? The Army and Navy both have different physical fitness standards for men and women. I didn't look up other branches but I don't expect them to be any different. I wouldn't expect there to be a big push for different standards because that's already where things are now.

> Where have all the different standards for men and women in the armed forces come from, then?

They came when women were first admitted to the armed forces at all, and are contemporaneous with the rules barring them from combat roles. They are not the product of pressure to include them in combat roles.

Roles (including, but not, AFAIK, limited to, combat roles) within the military can have physical standards above the minimum standard for the branch of service, and no one that I know of that has argued for removing the restriction on women serving in combat roles has advocated against gender-blind role-specific standards.

There are two separate reasons for fitness standards.

The first is the one you're probably thinking of - you need to be so strong or so fast in order to do your job.

The second is more subtle, along the lines of "we want soldiers to have the discipline and willingness to follow rules that physical fitness signals." This is why they want a drone pilot, for example, to meet physical fitness standards.

Really, the military should have three fitness standards: one for all male soldiers, one for all female soldiers, and another for the soldier's specific speciality.

There's a slightly stronger argument to be made in this space. Signs of technical competence "screens off" the effect of gender. That is, once you've had that five-minute conversation with someone, learning their gender doesn't give you any additional information about their engineering competence.

Actually, no. When you pick out an individual there's still a 60% chance than one is better than the other given no other information than their sex. That's how statistics works.

But I agree that it's better not to make the prejudgement at all. The problem is that there are many things that are true in the aggregate but we can't acknowledge them because we are scared of demotivating people. Is that the best we can do?

Yup, "insignificant" is the wrong word, since it has a specific meaning in statistics, and for that meaning, my paragraph is definitely wrong. How about if I change that word: "But at an individual level, that difference becomes trivial."

I get what you mean, regardless of wording. People should be seen as individuals. I don't know how we can be blind to aggregate differences enough to not be perturbed when we have unequal outcomes but not so cognizant of them that we demotivate people.

But as I write this, I'm arriving at an answer: we need to recognize that people are making choices based upon who they are and what they like.

As I have read this thread, I keep coming back to how we measure competence and ability.

For example, at one point in history our IQ tests included questions based on baseball rules. So someone taking the test that didn't know the rules of baseball couldn't do well – regardless of their mathematical or logical ability, i.e. what the question was supposedly measuring.

Along those lines, I would strongly:

- recommend listening to the Hanselminutes episode on Women in Technology in the Muslim World (http://hanselminutes.com/203/women-in-technology-in-the-musl...)

- remind people that the first programmer was Ada Lovelace, and computers used to be actual people and mostly female (https://en.wikipedia.org/wiki/Human_computer) & suggest reading some of the reasons this changed (http://www.smithsonianmag.com/smart-news/computer-programmin...)

In other words, culture can strongly impact both perceptions of ability and measurements of ability, as well as impacting the equitable training and opportunity that people are afforded. I suspect that most differences in most fields are not actual sex differences (i.e. biological), but in fact are social differences based on gender (i.e. cultural based on a person's gender identity).

I recently took an artificial intelligence class that required writing an agent to answer non-verbal IQ test questions. One of the greatest challenges was that some of the questions had ambiguous answers. For example, an axial symmetry in the example frame could be reproduced with either a rotation or a mirror operation in the output frame, and both options were available as candidate solutions.

Naturally, the question came up: how do we define the "right" answer when there are multiple valid choices? As it turns out, the folks who write the test determine the correct answer to these types of question based on the most common response from test takers who get the highest scores on non-ambiguous questions. The thinking goes that if folks who get most of the other questions right largely agree about a preference for one answer over the other, that must be the most correct answer.

The most interesting part of that discussion to me was when a classmate observed that we use human intelligence as a baseline for measuring human intelligence, and then marvel at the extent of our own intelligence; if a machine used more sophisticated logic than a human was capable of to choose a different answer, we would look down on the machine for being less intelligent than man. It reminded me very much of Plato and the shadows in the cave.

You’re trying to redefine words out of context.

Significant is often used in science to mean large enough to be concerned with. So, a 1% chance of death tomorrow is a significant chance of death tomorrow.

That is not how statistics works. This is how you reverse a conditional probability (Bayes's Rule):

  P(B|A) = P(A|B) * P(B) / P(A)

If A is "female" and ~A is "male", and B is "good engineer", then what we know , including ancestor post's assumption, is this:

  P(A|B) = 0.4 (the population of known-good engineers is 40% female)
  P(A) = 0.5 (the population as a whole is 50% female)

We can further assume, arbitrarily, that P(B) = 0.02 (one in every fifty people make good engineers). So the likelihood that a person is a good engineer given that they are female is P(B|A).

  P(B|A) = 0.4 * 0.02 / 0.5 = 0.016
  P(B|~A) = 0.6 * 0.02 / 0.5 = 0.024

What we learn from this is that sex is not a good screening criterion for finding good engineers. You realize an advantage of only 0.8%, not 60%.

Thanks for presenting this so clearly. However, note that, as you show in this scenario, males are, on a relative basis, 50% more likely than females to be good engineers, knowing nothing else about them other than gender (P(B|~A) = 1.5 * P(B|A)). While the absolute probabilities (and difference in probabilities) are small, the relative improvement in classification error rate attributable to knowing the person's gender could very well mean that gender is still a useful feature to include (among many others! it's not so useful on its own, as you note) in a classifier for good engineers.

If you were writing an automated Bayesian classifier to find good engineers, males would get one measly additional point towards the refer-to-a-human threshold.

In contrast, there are many other hypothetical rules that would be better predictors. Liking Star Wars, 4 points. Star Trek, 9 points. Knowing how much mana a Lightning Bolt costs, 6 points. Owning a 3-wolf shirt, 2 points. Being left-handed, 15 points. Reading HN, 40 points.

Anything you can glean statistically from a population of known-good engineers can be transformed into Bayesian classifier rules, in exactly the same way you can predict an e-mail is spam if it has certain strings in it.

But even including the one point for knowing the sex could be considered sex discrimination, even though it would be totally supported by the math. But since you likely want your threshold value to be high enough to weed out false positives, while still low enough to avoid false negatives, that one point rule is practically a waste of time. The cases where that one point makes a difference will be just those people who barely meet the threshold. If you have two people who are only just barely good enough to be considered good engineers, and otherwise exactly the same, the statistical argument says to prefer the male.

I have never met anyone that is content to hire engineers who are likely to be only barely adequate in preference to those who are likely much better. That hypothetical person is the only one who might care about male or female. Everyone else will be looking for the highest point totals, gained from criteria that are better predictors.

This assumes it's independent of other data.

If your classifier also knows that an applicant has a CS degree and hypothetically a higher percentage of females with CS degrees than males with CS degrees are talented. Then the relationship might reverse.

The point is stats about populations often fail to carry over to sub populations.

PS: This flip is actually fairly common in the face of discrimination. Basically, you would expect the first hundred female fighter pilots to be way above average.

There are three hypotheses:

1. The selection criteria are the same for different populations, and the there are significant statistical differences in the populations that may lead to the observed result.

2. The selection criteria are the same for different populations, and random noise produced the observed result.

3. The selection criteria are different for different populations.

You cannot assert one without ruling out the others.

In the fighter pilot case, I imagine that it would be easy to disprove #1 and #2. Not all cases of discrimination are so obvious that there are no non-imaginary solutions to the equations that conform to the assumptions of the hypothesis.

It makes some people uncomfortable to face the idea that in order to prove discrimination is occurring, they must first employ accurate statistics for partitioned populations, which may be unflattering to one or more of the partitioned groups. You can't really say "because race" or "because sex" or "because disability" with certainty over "because selection-relevant attribute" using a statistical argument without showing the statistics. But those who attempt to gather accurate statistics are often accused of something-ism and vilified just because some people don't like the results.

Some people just want to work without constantly watching their backs and covering their asses. And that's why we assume independence without having ironclad proof that is also easily understood by the general public. If you're going to suggest that there is something intrinsic to femaleness that causally links to less engineering skill, you had better have a flawless data set with mathematically perfect analysis, and then also invent a psychological "out" which absolves anyone of guilt for relying on the results.

This is why all HR departments suck. Most of their job is covering someone else's ass, and if they choose to use statistics, they must be both factually correct and politically correct.

This also isn't how statistics works.

> We can further assume, arbitrarily, that P(B) = 0.02 (one in every fifty people make good engineers). So the likelihood that a person is a good engineer given that they are female is P(B|A).

  > P(B|A) = 0.4 * 0.02 / 0.5 = 0.016
  > P(B|~A) = 0.6 * 0.02 / 0.5 = 0.024

> What we learn from this is that sex is not a good screening criterion for finding good engineers. You realize an advantage of only 0.8%, not 60%.

if you multiply two numbers (here 0.4 and 0.6) by the same constant (here 0.02 / 0.5) you can't then say "You realize an advantage of only 0.8%, not 60%"... 0.016/0.024 == 40/60.

Re: OP's point ("When you pick out an individual there's still a 60% chance than one is better than the other given no other information than their sex"), I agree that's wrong, but not for the reason given. He's not reversing a conditional probability, but rather asking a question about the unconditional probability (the base rate), so the 60/40 conditional information is not relevant.

If you pick two random individuals, one male and one female, and the IQ average of the two groups is equal (roughly true, from the article), the chance that one has a higher IQ than the other is 50%.

  julia> sum([randn() > (randn() * 10) for i=1:1_000_000])
  499884

IE assuming that men have 10x the IQ variance, the variance makes literally no difference when comparing two random people.

Your post looks like a non-sequitur to me. But for the sake of math...

For an arbitrary variance multiplier of 10 (std. dev. sqrt(10)), there are no non-imaginary solutions for a high-pass threshold where you can achieve a 60:40 split between two populations that have the same median.

You can get about a 58:42 split, putting your threshold a bit below the common median. But to achieve a 60:40 split, you need one population to have 25 times the variance, or for the populations to have different medians. Above that, you have two possible real solutions for the threshold value.

You can try it yourself, and hopefully my math is correct for this one:

  60/40 * erfc( x / sqrt(2))/2 = erfc( sqrt( variance_ratio ) * x / sqrt(2))/2

Since the threshold value equates to a different standard deviation for each population, you can say with some certainty that given the knowledge of which population a person is in, there is a different probability that person is above the threshold value, which works out to be 60% and 40%, respectively. But that's just begging the question, since those are the same numbers you used to work out the threshold value, and you still had to assume values for the medians and variances of both populations.

I'm super confused by this reply. The article says that the stddev of the distribution of male IQ is larger than the stddev of the distribution for female IQ, and also seems to state the mean and median are equal. GGP threw out 60/40 as their believed split of men to women being good engineers. I took this to mean that GGP was saying he believed engineering talent to be independent of gender when IQ was present, and that engineering required a higher than average IQ. More formally:

  P(good engineer | IQ) * P(IQ | gender) == P(good engineer | gender)

In that model the 60/40 split comes from a conditional distribution of gender given an IQ above some threshold. So something like:

  julia> women = sum([randn() > 1 for i=1:10_000_000])
  julia> men = sum([(randn() * 2) > 1 for i=1:10_000_000])
  julia> women / (women + men)
  0.3395449535000696

Which isn't exactly 60/40, but is fairly close. It says that [iff the 60/40 split is true], the stddev difference needed is less than 2x.

If you prefer exact math, the ratio above is

  erfc(1/sqrt(2)) / (erfc(1 / (2 * sqrt(2))) + erfc(1 / sqrt(2))) =~ 0.339593

and the exact solution for a 60/40 split is

  1 / (sqrt(2) * inverseerfc((3/2) * erfc(1/sqrt(2)))) =~ 1.4

Though FWIW I find the simulation version more intuitive (likely because of a misspent youth programming rather than a misspent youth mathing :p).

Can you explain the model you were using to motivate the mean vs median calculation? I feel like I'm missing something interesting.

Easy to explain. I made a mistake in my math. The equation I needed to solve was

  40/60 * erfc( x / sqrt(2))/2 = erfc( sqrt( variance_ratio ) * x / sqrt(2))/2

I don't understand why you are taking the advantage between the two options to be the difference between their probabilities, rather than the ratio. If I am 2% successful at X and you are only 1% successful, I am 0.02/0.01 = twice as successful as you, not 0.02-0.01 = 1% more successful.

As dlss has explained, it doesn't matter what constant you multiply by, the ratio is the same.

You are taking the ratio of P(B|~A) over P(B|A):

  P(A) = P(~A) = 0.5
  P(B|~A) / P(B|A) = ( P(~A|B) * P(B) / P(~A)) / ( P(A|B) * P(B) / P(A))
  = P(~A|B) / P(A|B)
  = ( 1 - P(A|B)) / P(A|B)

I don't understand why you consider this to be a meaningful number. It isn't even a probability any more; it's just a dimensionless ratio.

In your example, your result is 2. That alone tells me it cannot be a probability, as those numbers are restricted to the domain from zero to one.

If you partition the population by sex, the probability of randomly selecting a good engineer from the all-male group is only 0.8% higher than picking from the all-female group. That is the advantage you realize by selecting that criterion for your partition. If you are taking a ratio of conditional probabilities, you are no longer partitioning the probability space by that condition.

Height is a good example to explain this with.

Everyone knows men are taller than women on average, and that a lot of women are taller than a lot of men.

I'm a female computer scientist / developer, and after my experience both online, at school, and at work, I would really just rather ignore it. I went to a college with 4:1 ratio of male to female.

I've had periods in my life where I felt like I was the beautiful princess at the school, and periods where I feel like a somewhat disfigured monster with an average intellect. It really sucks when culture places so much emphasis on appearance, and I don't have that when I engage in dialogue and activity online.

The fact that I've been able to think about this since I was maybe 14 years old, on a hypothetical and applicative level, shows that there is more that varies to predicting individual differences than what these tests measure, and what causes are inferred, as the act of measuring may change the outcome. Males may take more risks because culture rewards them if they do, supposedly, and I as a female have sought stability, predictability and consistency in my education. When people are trying very hard to either sleep with you, hit on you, or even screw with you (because for some reason, your existence bothers them), it's very hard to know whether you are learning the truth, or whether the truth has been distorted, made easier to swallow, or prepackaged otherwise, because of your teachers' and classmates' perceptive biases.

I take a lot of risks in learning. I try to avoid taking those risks in real life, because while at one point, I did, I found myself on the precipice of losing almost everything.

I agree with you wholeheartedly, that the individual is different from the aggregate. But even myself, I am a hypocrite, because these are cynical inferences I made along the way, about my education, while I was receiving it. And I made those inferences mostly from having a behind the curtain peek into boys clubs, online. I was judging my classmates and peers by assholes on the internet, and that really isn't right, but neither is it right to assume that a person fits on the distributive curve of which they can be categorized into, based on a very small subselection of data.

Thinking about this has led me to the conclusion, that I really don't know a lot of things, and it seems neither does anyone else. Every point in time can be as unpredictable as infinity. People make so many assumptions about what is stuffed up in other people's heads, what guides how they think, what they do with that knowledge, how it directs them, and the resulting thoughts, the determination of envisioning, assisting, and planning the life paths of others.

My greatest comfort level is being an anonymous blob with no face, on the internet, where I learn a little bit of haskell here and there, and maybe find some elegant nuance and subtlety in code, computational theory, mathematics, philosophy. It's a bit hypocritical for me to share my gender willingly, over and over, but at this rate, I go through probably hundreds of aliases in a year.

Thank you though, for clarifying an important point concisely. The last time sex differences came up on a forum, I don't remember feeling quite so calm.

A cousin of mine with a high IQ (TNS level) and who would be categorized as a 'cute blond' expressed similar laments. Moderately sociable as a kid, reasonably well-heeled, one who loved spending most of her time devouring science and math tomes, conducted experiments on her own; an individual. Then adolescence hit coupled with menarche and amplified social pressure to 'fit in' which caused her sense of strong individualization to wane. In late teens she talked to me about majoring in law or sociology and I questioned: 'is that YOU?' What followed was a long diatribe, de rerum natura, which had the earmarks of a 'something' being repressed. Using salient observations like those opined by Marilyn vos Savant in her old 'ask Marilyn' column it was conceded that individualization and mutualization are sustainable while open-end charity with no sense of culturally enforced mutualization is fatally maladaptive, given received human nature. And the old social dictates evolved for survival in times prior to the advent of modern medicine were just so THEN but this is NOW. And the great seminal inventors and scientists of the late 19th century, almost all of whom were male, were individualists by temperament who while developing their ideas would partition themselves off from the vicissitudes of life for the duration. That exchange was capped-off with a paraphrase of an extract from the 'Bulfinch's Mythology' section 'Glaucus and Scylla'. Glaucus was a river god who developed the hots for a maiden called Scylla, who scorned his every approach. Retreating in to sulk mode he was given a pep talk by a mentor goddess Circe (where the word cereal comes from). To paraphrase: Recognize and objectify your worth. Then realize you are someone to be sought after rather than one who seeks in vain. Be willing to meet people half way. But if they spurn you, spurn them! Why should they disturb your Wa? (Wa = inner harmony (jp); we were both into Japanese and anime). In any event, my cousin segued back into math and IT, is herself, and suffers no fools lightly (IE, abides no BS).

Right. You have to make your laws and policies blind to difference, even though they're necessarily operating at the aggregate level where there are differences, so that individuals don't suffer. An interesting example where willfully disregarding statistics is actually beneficial.

Reply on news.ycombinator.com