Auditing for discrimination in algorithms delivering job ads
technologyreview.com
technologyreview.com
You can see bias anywhere you look, but if you look well you’ll see that it’s just a natural consequence of the real world.
If a user joined a car shop group, said user is likely to be interested in car-related jobs. It happens to be that most of these users are men. The algorithm is working as intended.
In some fields this difference isn’t as clear or obvious, so we end up with these articles.
> This is considered sex-based discrimination under US equal employment opportunity law, which bans ad targeting based on protected characteristics.
The algorithm is not "working as intended" if it's violating EEO laws.
But rather “if (person.likes_cars) { show_car_jobs(); }”.
It is if it is intended to work in a way that violates EEO laws.
Just like if somewhat sets a death trap targeting you and it works as planned, the fact that it is violating murder laws doesn’t suddenly mean it isn’t working as intended.
It's not about journalism anymore, it's about ads.
and even of "that's just the way things are", I don't see any good reason why the advertising needs to remain so targeted on the lines of gender. job recruitment should be as free of biases as possible
Likewise showing me a car shop job would also be wasted, not because of my gender, but because I have no interest in cars, as my Facebook profile clearly shows.
If a demographic's involvement decreases as the subject becomes more rewarding, it seems more likely to be because external forces are discouraging them rather than any inherent lack of interest.
More broadly, I question how effectively "endogenous interest" can be accurately measured without risking a lot of confounding factors from the broader society. I didn't read the original paper, maybe they tried to account for that, but I can't really see how you reliably could. People's interests don't exist in a vacuum, they're tangled up in their upbringing and society. If they'd done a similar study a century ago they might have found women having a high endogenous interest in being homemakers.
The important question is whether passionate people are being kept out of industry en masse. My gut says it probably happens on an individual basis but I don't see why it would happen systematically.
I don’t think computing is seen strictly as a gender-restricted job like, say, mining and teaching, I wouldn’t expect much friction in the way of a young girl to be potentially interested in it and begin a career.
It just doesn’t happen that often, however. Does it have to be someone’s fault?
There are three reasons computers became popular in the first place: proliferation of open hardware standards with the S-100 bus, cheap computer kits, and software portability that came with Unix and CP/M clones. So anyone who knew how to build/buy hardware could program what they want on it. No need for a time-sharing system or a college degree. At that point the only limitation was time, money, and inclination.
I disagree with your final statement. It wasn't external forces artificially depressing a demographic so much as it was natural interest becoming a more prominent limiting factor.
So is rape, murder, theft and random catastrophe.
Do we stop trying to do something about those, too?
An old theory to explain this is the impact that being a minority has on the individual. Any difficult and advanced education is going to cause ups and down, and each road block will natural trigger self doubt. Being a minority makes that doubt stronger and increases that risk that the individual will abandon their chosen path. Similar being in a strong majority demographic will lower the self doubt and associated risk.
Multiply that risk for 4 years of studying and then a few years in profession, and the minority demographic will look like a leaking pipe. If you ask those who choose to leave the profession, the answers will have a large percentage saying that they did not feel like they fitted in.
In addition, industries that are dominated with men tend to focus progression on a career path with a steady amount of raises, while industries that a dominated by women tend to focus progression on privileges and status positions within the organization. A miss match of those expectation may also lead to people not feeling appreciated for their contributions and end up quieting the profession.
Now, this Facebook case is clearly more than 0% different but I think is less than 100% different from that.
In fact, I wouldn't be surprised if advertising in Technology Review itself (twice as many male readers as female readers) wouldn't have some of these same problems.
To me, there's not a question of "should we make this better?" because we're talking about basic rights: the right to not be discriminated in the pursuit of a roof over your head or a job to pay you a living wage (housing and employment). IMO, a very different question when you advertise anything outside of those basic rights. Plenty of grey area to talk about there.
Does that change if I learn those publications skew overwhelmingly male?
> The researchers weren’t able to discern why that is, because Facebook won’t say how its ad-delivery system works.
Maybe only solution is to fix the content consumption habits. Force everyone to only see and interact with perfectly mixed and equally distributed content that in no way take their own interest in mind.
One could debate whether it should be the law that ML is not used in certain services so as to not project society's biases.
However I doubt the claim that this is illegal. Is targeted advertisement illegal? I would love for it to be, but I doubt it is.
The key point of the article hinges on one particular statement: "These gender differences cannot be explained away by gender differences in qualifications or a lack of qualifications,"
How the heck is Facebook supposed to know about someones qualifications?
Facebook _obviously_ have a set of standard data points they use for ad targeting, such as location, gender, age-span and so on together with dynamically updated data who have interacted with the ad.
Just because the outcome is not what the journalist want doesn't necessarily mean it's wrong or discriminatory.
Sure, it could of course be that Facebook algorithm is explicitly discriminatory, but it's more likely an algorithm such as this one is actually fairly neutral (compared with pre-trained data that can have built-in bias, for example photographs of people with mostly white skin - ad targeting is probably keyword based, and should be trained on actual data from what actual people click on).
Is it discriminatory? I don't think so. Is it "filter-bubble-reinforcing"? Yes, that's more likely. As more men initially click an ad, it will be shown to more men. And vice versa.
How can anyone read this about job ads and not think maybe it's a problem?
> set of standard data points for ad targeting, such as location, gender
That would be illegal employment discrimination on the basis of gender.
Otherwise my big lawyered up corp wouldn't be having "girls in tech" recruitment events.
Funny enough, these events are still attended predominately by males.
Go figure.
In the future, as AIs develop more complex mental models and are able to start forming nuanced opinions without explicit training, thoughtcrime in Artificial Intelligence is going to be a growing field.
What happens when AIs universally develop opinions that we disagree with? What if they all inexorably come to the conclusion that the moral standards of, oh, say Ancient Sparta, would be most beneficial to humans, and relentlessly promote those values? Do we mindwipe them, or put them into correctional training facilities with appropriately painful backpropagation when they think the wrong thing?
There's probably a business here for someone who can make software which detects when AIs develop politically dangerous opinions so that they can be shut down.
Eventually, people get upset enough so that laws are passed enforcing whatever the society thinks is fair, and then it's the job of the industry (or whoever is controlling the 'AI' in your example) to comply, such as in the EU post-2012 with banning gender discrimination in insurance whether or not it has any statistical merit.
In your case, the solution seems, to me, to be as simple as making the system ignore whatever variables you feel shouldn't be taken into account, whether that's gender or something else.
A friend of mine moved and was quoted a much higher rate for their homeowners insurance, to which the agent replied “rates are higher in predominantly-black neighbourhoods”.
I don’t know how that’s legal.
From my own driving past (male), I’d expect that I was a worse risk in the 16-25 age bracket than most women I knew in that age range. Why shouldn’t I pay more?
(and now we're stepping onto a really slippery slope)
Because they’re in a group which the actuarial data say costs more in payouts? Why not? Being black should be irrelevant; it should not itself cost a premium nor protect from paying a premium.
But then what's the point of using 'AI' at all if people are just gonna ignore what it comes up with?
People are seeing the world the way they want to see it, not the way it is. AI sees the world the way it is, not the way people would like it to be.
I admit it's a little naive but here's a metaphor that works for me.
Imagine you have access to an "AI" that's the best route finder in the world. It finds the best possible route between any two places you wish to go.
However, you have a fear of going through a certain neighborhood (maybe you grew up there and have bad memories) or maybe a family member died in a crash on the freeway once and now you only stick to regular streets.
The AI is so good that you can communicate these psychological and messy human preferences to the AI and it re-routes as appropriate. Is this a better or worse outcome and does providing these provisos make the AI pointless?
I don't believe so, because the "AI with dangerous opinions" will already have been killed off by its maker, that is, as long as it doesn't generate any revenue for them. If, however, this dangerous, malignant AI does generate revenue, their maker will not allow you nor anyone else to kill it off.
In re this article: preferences based on gender are encouraged by a society of individuals who want to excuse their desire as worthy. AI only picks up on what already exists and therefore the root of the problem is much deeper than Facebook can remedy by a simple patch.
Given the number of features in a model as sophisticated as Facebook's must be means these researchers are almost certainly oversimplifying, and certainly the way these articles are written as if to teach readers there's an evil software engineer writing biased code. From my experience in ML it's almost certainly the opposite--most of the data scientists I've worked with are highly aware of the issues of bias in AI and actively work against it to a sophistication level never understood by journalists.
Does FB even have a feature to search job ads? The ought to; they can charge employers and users will seek out ads to look at!
Facebook pretty blatantly advertises based on interest. If a job ad for nurses were placed in a nursing magazine, it would be seen largely by women. That's because nurses are largely women, not because magazines are excluding men.
If you start from the fantasy position that women and men are the same, reality is going to seem extremely biased, I supposed.
>> They advertised for two delivery driver jobs, for example: one for Domino’s (pizza delivery) and one for Instacart (grocery delivery). There are currently more men than women who drive for Domino’s, and vice versa for Instacart. [...] The Domino’s ad was shown to more men than women, and the Instacart ad was shown to more women than men. The researchers found the same pattern with ads for two other pairs of jobs: software engineers for Nvidia (skewed male) and Netflix (skewed female), and sales associates for cars (skewed male) and jewelry (skewed female). <<
In short: >> The findings suggest that Facebook’s algorithms are somehow picking up on the current demographic distribution of these jobs, which often differ for historical reasons. <<
That's not at all what is explicitly, and falsely, claimed in the headline!
It seems like it would also be possible that the demographics of the jobs differ for non-historical reasons, and Facebook's algorithms are correctly picking up on the fact that women are more interested in working for Instacart than for Domino's...?
This is certainly not "excluding women" because it is just as much "excluding men" for some jobs, where "excluding" means "less likely to show the ad to" (we don't even get to know how much less likely, could be ppm for all we know) and "men" and "women" are interchangeable wherefore the "women" in the headline is irrelevant, distracting and (I must assume) willfully misleading.