Goldman faces probe after entrepreneur slams Apple Card algorithm in tweets | Hacker News Reader

Goldman faces probe after entrepreneur slams Apple Card algorithm in tweets

reuters.com

29 pointsshakes31 comments

> David Heinemeier Hansson railed against the Apple Card for giving him 20 times the credit limit that his wife got

Heh, railed. Good on DHH for using his platform and bringing light to an issue like this.

Original thread on Twitter: https://twitter.com/dhh/status/1192540900393705474

Went to the article to find all the other evidence, and no it pretty much boils down to your single quote. Some guy is not happy with his wife’s credit limit and went on a tweet-rant accusing sexism.

No idea how credit limits are calculated but maybe his credit limit is higher not because he’s a man but because he’s an entrepreneur?

Sure. Someone on Twitter pointed out that GS is accustomed to catering to business accounts. (The article here also says it is GS' first credit card.)

They, and presumably Apple moreso, should have considered this before using the biased algorithm for their first consumer credit card. They should have analyzed -who- would be affected by it.

> should have considered this before using the biased algorithm for their first consumer credit card

Why would such bias be bad when assigning a credit limit? Are you suggesting that this just isn't related to creditworthiness?

It's pretty much impossible to determine from the outside whether the algorithm discriminates against woman (without a reasonably large and actually random sample of customers). But that is also part of the complaint, as the algorithm is a black box that even the customer support employees can't look into. Humans just blindly following a black box algorithm without the option to manually handle cases where the algorithm produces weird results has quite a dystopian feel to me. Especially if it's used for more consequential cases than credit card limits.

Hence the probe by DFS.

Confused. They are saying that nearly all married couples (male:female) with joint everything and the same credit history are showing consistently the husbands having more credit? Or just one single person out of millions? As one would assume there are probably many similar cases but where the wife got more credit no?

> Or just one single person out of millions

This appears to be about a single loud person out of millions.

I see mixed reports while searching on twitter, but nothing seems to suggest a bias in either direction.

When I read the Twitter thread yesterday there were plenty of people posting that they had the same observation: men getting more credit than their wives. I don't recall seeing anyone posting the opposite.

there are a lot of variables that go into determining a credit score.

in this particular case, i could imagine that in his role as partner as Basecamp there is a longer "credit trail" attributed to him just from a business operational standpoint beyond his joint family tax filing. that alone could explain the discrepancy between him and his wife beyond sexism.

Explaining the sexism away does not make it less sexist or defensible. The point is the algorithm has a sexist bias, as multiple anecdotes show.

>The point is the algorithm has a sexist bias, as multiple anecdotes show.

By definition anecdotes prove nothing. Until there is statistically significant evidence of sexism this is just hearsay and your post gets close to slander.

In this case it will be absolutely trivial to check. Just feed in the same data for the same person only with the gender flipped.

The whole point of the original tweet was to point potential systemic gender bias in the algorithm. The argument here is that the black box algorithm may produce sexist results whether gender is included or not.

While there may only be anecdotes at this point, there should be some investigation and discussion into whether bias may occur toward particular classes outside of explicit gender checks. Seems like that would be done by comparing the allowed limits with gender, race, etc. and considering why the algorithm may unnecessarily limit those classes on a broad scale.

Could you share this compelling evidence that the algorithm has a sexist bias?

Later on the in the thread, he points out that his wife has a higher credit rating than he does which seems to undercut this theory?

https://twitter.com/dhh/status/1192945760909676544

Honestly it just sounds like this guy hasn't applied for very many credit cards :)

Sometimes you get a good limit, sometimes you don't.

When I was looking through the twitter replies last night, multiple people claimed the opposite happened (the female got the higher limit). It’s sad that the journalist either didn’t look into it or preferred not to stray from the implication that the algorithm is sexist rather than just buggy.

When I was looking through the thread I didn't see this. Do you have links to tweets?

Here's what I could find in 5 minutes:

https://twitter.com/Flkittie/status/1193725782788628480

https://twitter.com/HailMaryHumble/status/119385501173590835...

https://twitter.com/mindnotnull/status/1193794029189767168

https://twitter.com/Chaubin123/status/1193729943701794823

https://twitter.com/StrosniderDave/status/119369802228851098...

https://twitter.com/ksgant/status/1193666460561879042

https://twitter.com/fauxgressive/status/1193635810475683842

https://twitter.com/snitstwits/status/1193626585821319168

https://twitter.com/chrishedlund/status/1193539000176455680

https://twitter.com/Nukes4A/status/1193341783180283906

https://twitter.com/Wayner83/status/1193332092291239936

https://twitter.com/JeffBessling/status/1193331529453363202

https://twitter.com/bdruth/status/1193315200147054593

https://twitter.com/Dcorbett818/status/1193282534622158848

https://twitter.com/NatalieEvansMD/status/119326982416994713...

https://twitter.com/jetkatphoto/status/1193211571222142976

https://twitter.com/mbfuchik/status/1193143158504349697

https://twitter.com/allenjs/status/1192959576477298688

https://twitter.com/bstahlhood/status/1192816047117746178

Of course, I'm just some "mansplaining" asshole. You should just ignore me.

Thanks for these links.

What's the sample size here?

Non-existent apparently.

Enough to know a 20-to-1 difference is abhorrent, given they file joint tax returns and she has a better credit score?

If you're married and you're filing jointly, it's really no different than the same individual applying twice in a row. They've already extended one credit line to a specific entity, so it makes perfect sense that the second line would either be denied or minimal. It baffles me that this has exploded so much

Surely tax filing status is not the only factor taken into account, nor is credit score. If I were a stay-at-home dad and my wife was a doctor, I’d expect her creditworthiness to be much more than mine, regardless of how we did our taxes. If both of us took out $100k in credit and then got divorced who is most likely to be able to pay it back?

Theoretically both, because the lower earning spouse will be taking in alimony

Oh don't get me wrong, I'd love to understand the algorithm that determines who gets how much credit.

But we're assuming that it has to do with gender, and none of us actually know the difference between these two people historically. My wife has better credit than I do but I can borrow more because my income is much higher, for example.

I think the idea behind joint accounts and joint filing is that, for all intents and purposes, the couple have equal incomes.

But if I'm doing an actuarial and know, for instance, that X% of marriages end in divorce, that's probably a calculation I'm going to make.

If you're doing that calculation and conclude that she is somewhat likely to end up divorced and with less income, because she is a woman, then... congratulations, your algorithm fulfills the definition of "sexism".

And just to be clear: It is not sexist to point out that women make less money on average than men. It is sexist to treat individual women worse than men because of this general expectation, without factoring in any other information.

I've been denied cards despite an excellent credit history, is this evidence of sexism or just otherwise shitty algorithms?

Reply on news.ycombinator.com