Hacker News Reader

Comment by jacquesm | Hacker News Reader

It won't have that line in their code. What it very likely will have is some Bayesian algo or maybe if they're really high tech some ML black box that will have gender as one of its inputs. And it probably shouldn't have that input.

The whole idea behind decisions like these is that they have to be explainable and ML especially does not lend itself well for that.

View on HN

Here's a thread on Apple support[1], with many reporting similar experience and an anecdote going the opposite way:

I’ve been having the same issue since Monday 10/14. I paid my balance in full, money was taken from my Chase account right away and cleared next day, but my Apple Card available balance hasn’t updated. I’ve also gotten the same response from support. It could take several days. Funny thing I paid my wife’s card off a few weeks ago and it updated within seconds, same bank account and everything......

[1] https://discussions.apple.com/thread/250676909?page=2

Note that an ML box doesn’t even need gender as an input. Name alone probably gets you 90% of the way there.

True, you could have an internal table that maps name to gender with a very high probability of being right.

That points to the core of the issue. "Fairness" in ML algorithms can be hard to define and assess.

It's easy to say "omit gender from the model", but the real issue here has to do with the _causal_ pathways between your input variables and the output variable.

Since ML mostly works by exploiting correlations between the input and output variables, omitting gender doesn't mean gender's influence is removed. You'll have to omit all the causal pathways from gender -> the output, effectively "d-separating" [1] gender from the output. Whether that's practical or not depends on how well we understand the data generating process.

[1] http://bayes.cs.ucla.edu/BOOK-2K/d-sep.html

You can simulate it with artificial querries with the same data except gender and name and see if women get less or more.

Reply on news.ycombinator.com