The only way to do this is to whitelist acceptable input variables. Detecting bias is a meaningless exercise.
A core lesson in ML: this is difficult or impossible. See the classic example with wolves, dogs, and snow. [1]
For a resume example, suppose you don't want to be biased again black candidates, or women. Well, good luck trying to filter out resume experience, skills, schools, and interests so that they don't list anything that is statistically more likely to be found among black folks or women. It's impossible.
So now you have a hiring process that looks at... what exactly?
[1] https://www.researchgate.net/figure/A-husky-on-the-left-is-c...
The whole point of white listing variables is to only allow things you are comfortable using todiscimrinate. And yes, that likely reduces the power of the model.
Edit: to clarify, your example had the model failing to classify something correctly because the training data did not generalize across real world examples. The problem with diversity bias is that the modes are trained on past data where the labels themselves tend to have some bias. Black candidates do worse. So recreatingthese will generally produce worse outcomes for minority candidates. Now for a lot of these cases the average minority individual is more likely to do worse. Not intrinsically because of race, but still worse.
You can’t realistically expect the model to produce equal outcomes for all classes when all classes are not equal on the actual traits. But you do want to try and make it so that the model doesn’t use race indicators as a crutch that ignores actual good indicators. The common complaint is you hear about a man and a woman applying with the exact same credentials but different outcomes. Well obviously the model had to have additional information besides those credentials if it made different decisions. If this causes problems, you probably should not permit those additional pieces of information.
The problem is that variables where you can safely discriminate don't exist in the real world because everything is intertwined.
For example, if you see that somebody went to school at {Howard, Bryn Mawr, BYU} you can make some pretty solid inferences about their {race, gender, religion} even if they don't explicitly provide them.
Howard is a historically black college, which means their student body today is majority black, not 100% black.
Bryn Mawr is a womens' school and admits only women. However, gender is not always fixed going through life.
BYU is a Christian school, but also admits several non-religious people too, and people regularly become apostates to their respective religion.
Technically true of course, but >99% of the population will identify with a single gender throughout their life. The correlation is so strong that it's not even really a correlation anymore.
That doesn't mean you aren't stereotyping people, though.
If the training data shows a history of bias against Howard graduates, the model will learn to replicate.
Job applications are kind of dumb. The common use case is credit decisions. Surely, for example, you’re ok looking at credit history and income when deciding to make a loan, right? Even though it won’t be equitable outcomes across races, it is still “fair”
Other goals can be to have equal model accuracy across subgroups (e.g., models has the same precision/auroc/etc for each subgroup, even if the predictions themselves differ across subgroups) or identical distribution of statistical biases across subgroups (e.g., the model makes the same types of mistakes within each subgroup)
Like, let’s look at that for credit decisions. Black folks have a lower income distribution. So it’s like saying the model is ok if it rejects a Black applicant making 80k if it would also reject a White candidate making 100k. That is the insane bullshit you’re effectively supporting by desiring comparable error distributions.
You have to accept that equal outcomes are not just impossible but not desirable. And then you only use fields that people agree are fair to use. If Black people get disproportionately better or worse outcomes it is either because those outcomes make sense given their stats or it’s not ok because if a White dude had the same info, that you agree is fair to use, they would have passed and the underlying reason for the difference is some bullshit other input.
That might mean that the model is a dumb: you must make $100k for this loan (illustrative) which is going to mean a lot of Black folks cannot get it. Well that model is at least “fair”. Accommodations for race should not be coming from an AI model
Even without the bias issues, if I were a hiring manager how do I verify that the weights "MagicHireAI SaaS Corp" are using aren't leaving swaths of talent on the table that I don't even know exist?
The amount of stuff they make you fill out that is then parsed with AI is insane.
Indeed ruined a lot. Before if you had an ok resume but we're able to talk to them in person you might be able to convince them.
Now with all those aptitude training you have to do well there.
Not to mention the applications that require you to turn on your webcam (!) while you do them. Disgusting.
That last one by the way was the same company where my wife had an in person interview. She drove there. The interview was on Zoom with somebody in another office.
She declined the offer. Wastefulness and already showing that they're not against monitoring by AI means 0 humanity.
I know I have to.
I know I don’t have to.
I don't understand what this means. Did she have to turn on her webcam to upload her cv? How does that even make sense.
Also, how is that legal? I thought requiring a photograph to apply for a position is illegal (except for stuff like acting).
> Now with all those aptitude training [...] that require you to turn on your webcam
The problem here is that companies have now gone from identity verification to this 'aptitude test' ML bullshit that's just another cover for increasingly weird arbitrary discrimination against applicants.
It doesn't benefit genuine candidates to blatantly lie. They'll get caught later in the process. But you're assuming identity thieves are acting rationally or fear getting caught. They don't, and they're playing a numbers game. If you get past an interviewer and get fired after two weeks, your salary at a US tech company can be $5k, which is the average annual salary in many developing countries.
In the US, it is illegal per the EEOC, which has barely any resources to actually enforce the law: "Similarly, employers should not ask for a photograph of an applicant. If needed for identification purposes, a photograph may be obtained after an offer of employment is made and accepted." [1]
[1] https://www.eeoc.gov/prohibited-employment-policiespractices
Asking for a photo opens an employer to claims of discrimination, but it would still have to be proven, so the government is advising against it. But it may well be that in practice, the government will not be spending resources on trying to or be able to prove discrimination, even if a photo is required.
Don't pay for this cruft, sponsor local meetups instead.
For example you can infer the race of a candidate from other facts about them. The model could still illegally discriminate by first inferring race, then discriminating on the basis of inferred race.
The problem you describe affects blacklists that try to remove only the things you obviously don’t want to use.