Hacker News Reader

Comment by kyrieeschaton | Hacker News Reader

I guess it worked for him to throw a hissy fit, but there's absolutely no reason to actually believe they're conditioning on gender.

It could be "it's a new product, we randomly assign credit limits to see how it affects behavior".

It could be "it's a community property state and we're overexposed to this household if we give the second card the same limit as the first".

It could, realistically, be almost anything except for evil bankers deciding to use illegal criteria to underwrite that has a side effect of limiting the amount that can be charged to the account each month (you know, how they actually make money).

Oh, random CSRs don't get a pithy explanation of a multivariate nonlinear underwriting decision to poorly convey to customers? That's kind of precedented!

View on HN

I mean, don't you think it's kind of a strawman to assume anyone thinks machines are explicitly sexist? It seems much more likely that whatever method they use to do feature selection on this obviously high-dimensional data just happens to end up picking something that was a proxy for gender.

You can be implicitly biased without being aware of it. This is true for both humans and algorithms.

> It seems much more likely that whatever method they use to do feature selection on this obviously high-dimensional data just happens to end up picking something that was a proxy for gender.

Yes, this is exactly what everyone thinks happened.

Nobody thinks people at Goldman-Sachs wrote

    if (applicant.gender == "F")
        limit /= 20

somewhere in their algorithm.

But regardless of how the thing happened, if millions of people are treated significantly differently for no reason other than their plumbing, that's a major problem. People have been talking about this "accidental proxy for gender" for years now; there's absolutely no excuse for no doing a basic sanity check to make sure that this kind of thing isn't happening.

edit: typo

"for no other reason than their plumbing" is exactly what you just excluded in the prior sentence.

It is absolutely not a strawman. It is incredibly easy to find people granted authority by the state that will claim that, eg, mathematics per se is an instance of exclusionist masculine thinking. It's even easier to find instances where people granted authority by the state will claim that any discrepancy in outcomes is ipso facto intentional.

I would love a customer-company relationship in which the customer gets meaningful information by privately contacting the company with a polite message, but in my experience this is rare. When an answer comes, it tends to misdirect rather than tell the whole truth.

Here you have a bank and a company known for secrecy.

But you acknowledge this;

> random CSRs don't get a pithy explanation

What the throwers of hissy fits are pointing to is that, by building blackboxes, you can get whatever result you want (which doesn't mean all results are expected) with plausible deniability.

Perhaps because this exact approach of a candid private question can be abused to support litigation or corporate espionage.

I've heard anecdotally a number of times that actuaries peg men with higher car insurance rates. I'm only just learning that there's anything like this in the credit system to prevent gender bias. Does something like that protection exist for auto insurance?

As a side note, Idon't really understand the thread's appeal to credit scores here, considering that the TransUnion rating system is supposed to be the inferior one that Apple is looking to replace.

But most of all, I'm shocked by the number of people here outraged by Apple's behavior who will not even be considering switching off their iPhone (Twitter OP included, hell, he even posted a screenshot of his recurring TransUnion payment still being served via Apple Pay). I actually happen to agree with OP, but this blaise attitude towards real customer complaints has kept me from using Apple products for years.

For car insurance? This is incredibly well known and not at all anecdotal. There was even a popular car insurance firm in the UK that only gave women car insurance if they were women called “Sheilas Wheels” which touted lower fees than “other insurance companies that have to also cater to men”.

https://youtu.be/GzNJh1o84-E

It's also now illegal for insurers to do this in the EU since the ECJ reasoned that discriminating on gender, even if as a result of such a correlation, was prohibited.

Anecdotally I've been told that (British car) insurers don't care very much about your real actuarial risk, they're focused more on whether you'll actually pay their premiums. Specifically I was told that work to integrate with credit checking services was a priority whereas an integration with the UK Government's service which gives them access to driving offences and other records related to a driving license was back-burnered.

The reason I was given was that in practice they'd found if you require drivers to give their license details, it causes a big drop in purchases, if you make the license details _optional_ lots of people fill them in, and you can just give all those people a better price even though you don't use the details to actually check anything automatically.

Even though it wasn't a protected class, GEICO (Government Employees Insurance Company) originally started as an auto insurance company specifically for federal government employees, a pool perceived to be less risky than the driving population

> I guess it worked for him to throw a hissy fit, but there's absolutely no reason to actually believe they're conditioning on gender.

There certainly is. Ignoring the anecdotal data of people replying to him who saw the same outcome, credit scores for women skew lower than for men.

One of many sources: https://www.federalreserve.gov/econres/notes/feds-notes/gend...

> credit scores for women skew lower than for men.

Which is crazy, because for most of my guy friends (myself included), saying that their significant others/women make significantly better financial decisions is putting it far too lightly. I basically wasn't making decisions at all (besides savings) until my s/o set me right.

This discrimination is tragic and disgusting.

"Good financial decisions" are only at a very specific margin associated with credit score. If you have an underwater car loan you can't really afford comfortably that you nonetheless make the minimums on whilst eating rice and beans, you'll have great credit after a couple years. If you strategically defaulted immediately after buying your ridiculously underwater condo in 2008 and it took them 3 years to foreclose on you, you made a great financial decision and your credit is trashed for the next six years.

Credit scores specifically correlate with default risk, not some abstracted measure of financial health.

Women tend to have lower incomes and lower net worths. Here is a recent article squeeing about women finally being willing to buy homes on their own:

https://realestate.usnews.com/real-estate/articles/the-rise-...

One of the statistics in it: Single women can only afford about 39% of homes. Single men can afford more than half.

Historically, couples with traditional marriages (primary breadwinner husband, wife whose primary responsibilities were women's work) often did not bother to put her name on a real estate transaction or car purchase. This substantially impairs a woman's ability to establish a credit record and reduces her legal claim to assets.

If you get divorced, the person with a real career and resume to match will likely continue having a good income. Someone who was a full-time wife and mom will face serious barriers to establishing a real career at all and may well remain poor for years to come.

I'm quite financially savvy. Financial savvy only goes so far. Ability to pay still matters and men are more likely to have that piece covered.

> If you get divorced, the person with a real career and resume to match will likely continue having a good income

Since I believe that gender roles is about splitting benefit and responsibilities, I asked myself what the outcomes of a divorce should result in in gender stereotypical situation.

https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5992251/

A man who focus on the traditional gender role of breadwinner and a woman who focus on the traditional gender role of social management will results in decreased income for the woman and decreased social support for the man past divorce. This puts the woman at increased risk of poverty, while the man has an increased risk of loneliness.

There is also some implied finding that women who remarry has on average a decrease in house income, while men do not gain persistent increase in loneliness. I do not think there is a major mystery why that is considering differences in inter-sex competition for men and women.

Yeah. It wasn't that long ago the mortgage industry was rife with blatant gender discrimination, which contributed to this issue. I've learned this from the compliance training at the mortgage lender where I currently work.

I don't doubt it, but I think it's far more complicated than that. I think a great deal of potentially low hanging fruit is actively overlooked by the tendency the world has to go looking for ill intent.

In cases where the explanation is something like "No, seriously, she simply cannot afford it." efforts to root out presumed bias are not only unhelpful, they are actively counterproductive.

I need more income. Period. If you were to approve me for a mortgage this minute for the $550k building I desire, I simply cannot make the payments. It's not going to fix anything.

First, get me the money. That's the one thing no one on this planet seems seriously willing to help me with and there is always some BS excuse.

Totally agree. When I taught high school in a impoverished area, someone somewhere said that poverty is acute lack of money, period. Everything else is a symptom (or feedback cause). So we spend all this time creating convoluted strategies to address poverty, except actually giving people money. Obviously, doing so wouldn't fix all problems on Day One for all people, but I now suspect it would be a lot more efficient than a lot of our efforts, and it would make the remaining problems more tractable.

> Women tend to have lower incomes and lower net worths.

Very true, because they aren't as smart and they don't work as hard.

No, wait, sorry, it's actually because there are stereotypes about women that aren't even remotely true that are reinforced decades after they were found to be offensive through institutional sexism such as the gender pay gap and the glass ceiling.

So, maybe the lower credit scores simply reflect the deck stacked against women in the workplace.

I'm plenty frustrated with actual sexism of the sort you and others here keep insisting is the entire explanation.

But, no, that's not remotely the entire story. Other factors that help suppress female income:

1. They tend to spend far more time on unpaid activities ("women's work") than men.

2. When moving to a new city, a man is very likely moving to a new job. This usually entails a pay hike. A woman is much more likely to be following her man's career to a new town. This typically involves taking a pay cut and derailing her career aspirations.

3. Men are more likely to pick a college that supports their career goals. Women are much more likely to attend whatever happens to be locally available and affordable.

4. Men who start businesses are typically making a career move. Women who start businesses are more likely to be starting a "lifestyle business" to accommodate other demands placed on their time and energy, such as special needs children.

I'm sure I could go on. I've read research on the topic for literally decades to try to understand where my life went wrong, but it's ultimately pointless since I can't cite specific sources, most of the members of HN aren't familiar with such stats and, no, just taking my word for it as a SME in the sorts of things that have negatively impacted my life is simply not anything anyone here will do.

Except for

1) according to him, the bureau they were using for underwriting data indicated she had higher credit score

2) "conditioning on credit score which is correlated with gender" is not conditioning on gender. The algorithms that derive credit score from a set of credit data do not condition on gender.

> 2) "conditioning on credit score which is correlated with gender" is not conditioning on gender. The algorithms that derive credit score from a set of credit data do not condition on gender.

It does if the algorithm is based on flawed assumptions, or worse, fed data based on institutionalized sexism.

I dare you to claim Goldman is using gender as an input variable for credit underwriting.

Since no one is actually doing this, we're back to something like "maybe (altho there's no evidence for this either) people indistinguishable from Mrs. DHH actually do look more likely to default, which is society's fault and thus Goldman should just take the hit".

I mean you know the system is sexist when one gender out-lives the other by a considerable number of years yet the retirement age for both is the same.

It could, realistically, be almost anything except for evil bankers deciding to use illegal criteria

FTA:

“My belief isn’t there was some nefarious person wanting to discriminate. But that doesn’t matter. How do you know there isn’t an issue with the machine-learning algo when no one can explain how this decision was made?”

They can explain, they most likely don't want to. Also how does this guy know this is a "machine-learning algo" and not just a series of ifs and elses?

It’s much more likely a series of ifs and else’s, not a fancy ML model. Companies have been doing automatic credit decisions for decades and there’s no evidence that GS bank is doing anything different than all the others.

> but there's absolutely no reason to actually believe they're conditioning on gender.

The anecdotal evidence, that many women are claiming that they received much lower credit limits than their partners, despite similar or superior credit conditions, would seem to indicate otherwise.

I don't believe anyone is saying that they're directly doing it. Just that their algorithms have that outcome.

It could also very well be "We trained our neural net on historical allocations of credit limits by professional analysts, who had a subconscious gender bias". The point is there's no way to know.

> I guess it worked for him to throw a hissy fit, but there's absolutely no reason to actually believe they're conditioning on gender.

Of course not, but why spoil the outrage mobs victimhood narrative du jour?

Reply on news.ycombinator.com