Hacker News Reader

Comment by jari_mustonen | Hacker News Reader

The gender bias is not primarily about LLMs but rather a reflection of the training material, which mirrors our culture. This is evident as the bias remains fairly consistent across different models.

The bias toward the first presented candidate is interesting. The effect size for this bias is larger, and while it is generally consistent across models, there is an exception: Gemini 2.0.

If things in the beginning of the prompt are considered "better", does this affect chat like interface where LLM would "weight" first messages to be more important? For example, I have some experience with Aider, where LLM seems to prefer the first version of a file that it has seen.

View on HN

IME chats do seem to get "stuck" on elements of the first message sent to it, even if you correct yourself later.

As for gender bias being a reflection of training data, LLMs being likely to reproduce existing biases without being able to go back to a human who made the decision to correct it is a danger that was warned of years ago. Timnit Gebru was right, and now it seems that the increasing use of these systems will mean that the only way to counteract bias will be to measure and correct for disparate impact.

A bit unrelated to the topic at hand: how do you make resume based selection completely unbiased?

You can clearly cut off the name, gender, marital status.

You can eliminate their age, but older candidates will possibly have more work experience listed and how do you eliminate that without being biased in other ways?

You should eliminate any free form description of their job responsabilities because the way they phrase it can trigger biases.

You also need to cut off the work place names. Maybe they worked at a controversial place because it was the only job available in their area.

So what are you left with? Last 3 jobs, and only the keywords for them?

I think the problem is that removing factors like name, gender, or marital status does not truly make the process unbiased. These factors are only sources of bias if there is no correlation between, for example, marital status and the ability to work or some secondary characteristic that is preferable to employer such as loyalty. It can be easily hypothesized that marital status might stabilize a person or make them more likely to stay with one employer, or other traits that are preferable.

Similar examples can also be made for name and gender.

Well the point is if you remove any potential source of bias you end up with nothing and may as well throw dice.

I think the real solution is having a million small organizations instead of a few large behemoths. This way everyone will find their place in a compatible culture.

Create a low-subjectivity rubric before looking at any resumes and blindly apply the rubric. YoE, # of direct reports, titles that match the position, degree, certifications, etc are all objective metrics. If you're using any other criteria for evaluating resumes, you should stop and wonder 1) are your criteria just subjective biases? 2) are you accidentally actually just selecting the most confident liars?

> are all objective metrics

gameable too, sadly

> The gender bias is not primarily about LLMs but rather a reflection of the training material, which mirrors our culture.

It seems weird to even include identifying material like that in the input.

Reply on news.ycombinator.com