Hacker News Reader

Comment by bjourne | Hacker News Reader

The dataset is biased. Otherwise woman~nurse wouldn't have been the second hit for the query. Had the dataset not been biased it would have been able to produce the analogy "woman is to doctor as man is to nurse". See table 4b where they perform that experiment.

I think it is also important to note that this is an argument over which of two methodologies is best. The answer likely is that it depends on the use case. Researchers are not being accused of using underhanded methods, manipulating data or cheating.

View on HN

Bias is not a boolean quantity. The underlying data set is somewhat biased, and the "trick" being used greatly exaggerates the impact of that bias.

Bias is a boolean quality. A random distribution is not biased. It really is not a "trick".

Any finite sampling of any distribution (biased or not) can show bias. A feasible process that doesn't introduce or exagerate bias is the biggest problem when trying to get representative statistics.

And bias can certainly be of different magnitude. A weighted dice can have a bias that shows up after only 10 rolls and another one can only show it's bias after 1000.

> A random distribution is not biased. It really is not a "trick".

A random sample is not a product of a biased sampling method, but it can be (and almost always is) biased, it's just if you take enough such samples, the biases will tend to offset, which is why we say the sampling method isn't biased.

You are splitting hairs.

> The dataset is biased. Otherwise woman~nurse wouldn't have been the second hit for the query. Had the dataset not been biased it would have been able to produce the analogy "woman is to doctor as man is to nurse".

I think that this hypothetical example would be evidence of bias. "woman:doctor :: man:nurse" as an analogy only works because of gendered implications of 'doctor' versus 'nurse', so if that were a robust output of the model it would imply that the word2vec found that gender was of great significance in the manifold near 'doctor'. (The analogy would just be applying a negative weight to the gender basis, but that's mathematically fine even if not customary.)

Reply on news.ycombinator.com