Hacker News Reader

Comment by hourago | Hacker News Reader

> Ban social scoring, not the AI, the tool doesn't matter.

Automated human classification without recourse nor transparency is what the EU is against. "You cannot get a loan because the AI says so" is not acceptable.

> Would it be good if it was computed by hand on paper instead of through AI?

Yes. As a judge can check what information was used and what algorithm applied to get to a conclusion. And it may declare that use of data or algorithm illegal.

> They fear the amplification factor of such tech, but the amplification factor works for the good and for the bad

Maybe just go slower. Something like an Hippocratic oath can be a good principle to follow for high impact technologies. To do some good may not justify the harm.

View on HN

>Automated human classification without recourse nor transparency is what the EU is against. "You cannot get a loan because the AI says so" is not acceptable.

The credit score system already in place in the US is frequently as arbitrary in some cases already. You get punished for just looking at it. I don't think I need to preach to HN about that (crypto money and all that). Does Europe do it much differently?

Not an arguement for AI to do it, but the only reason why the current system is better (at least in the US) is its sluggish incompetence.

> The credit score system already in place in the US is frequently as arbitrary in some cases already. You get punished for just looking at it. I don't think I need to preach to HN about that (crypto money and all that). Does Europe do it much differently?

Almost every country has some kind of credit reporting system, but many of them differ from the US in how they do it. In particular, in many countries, credit bureaus only hold negative information such as failing to pay bills or repay debts – which means if you keep up to date with all your bills and debts, your credit history will be blank. At which point, lenders will generally ask you to provide evidence of your employment history, salary, bank balance, assets, etc, and base their lending decision on that. They may well internally convert that all info into a "score", but every lender is going to have a different scoring system. Here's an article – https://www.businessinsider.com/credit-score-around-the-worl... – which discusses the systems used in different countries, including several European countries. There is no uniform approach to this in Europe – the UK and Germany are closer to the US in their approach, the Netherlands, Spain and France are closer to the "credit bureaus hold negative info only" system.

Yup. Until recently, Australia only had negative event reporting. Still, you’d regularly have kids that had been brainwashed by US-centric personal finance gurus (and American media more generally) asking how they could boost their “credit score”. Fully willing to sign up for and use credit cards as an 18-year-old student to juice some system they had been led to believe existed in the first place.

How is the system in Australia now and why was it changed?

I think in Finland having too many credit lines would actually hurt you with loans. However well you have handled those. As important factor is total amount of loans or capacity to get them.

Yeah, amount of credit cards & the total available credit limit is used as a negative signal when applying for a loan. But I think you'd have to have some completely silly amount for it to _really_ affect the outcome. You can easily show that you've not used the cards, for example. If you have 10 credit cards which you use constantly, it would affect your ability to get a loan. To me this makes sense, but of course, it's what I'm used to.

10 cards, with average of 3000 limit. Would mean you could take a 30000 more loans at high interest rates very quickly. It is not about having used and paid them, but capability to get loans beyond what you can manage very quickly.

It turns into a bit of a pointless ritual though - “I need a mortgage, I’m going to cancel cards I don’t use and slash my credit limits to maximise my borrowing power” - but no sooner have I bought the property, I will apply to increase them again, and they are unlikely to say “no”-even if the credit card is issued by the same bank as the mortgage. Or at least, that’s my personal experience in Australia

> the UK and Germany are closer to the US in their approach

Germany here.

a) But we are still not required to "build score"

b) There is currently yet another legal challenge against the Schufa (our credit scoring agency), this time in front of the ECJ.

Interesting, specially given how hard it has become to rent anything without giving a Schufa report.

Well, so far they more-or-less won every time. Though, from news reports, this time it seems serious. I guess we’ll see.

> The credit score system already in place in the US [...] Does Europe do it much differently?

Yes, as far as I'm aware the US & Canada are unique in having 'a credit score' for people; elsewhere there may be agencies offering scoring services to lenders, but it's not something individuals ordinarily nor need to care about, and it's not 'their score' it's just that particular company's (Equifax's, or whoever) rating.

It's not taken as seriously, nor does it need to be.

yeah, german here. I know schufa exists but right now it's really barely relevant. I don't plan to take on large debt or something (I think that's what they are used for?) and I've never used it so far.

>The credit score system already in place in the US is frequently as arbitrary in some cases already.

The credit score system is not arbitrary, it's a specific formula that applies equally to everyone.

>You get punished for just looking at it.

No you do not. There are hard inquiries and soft inquiries, and an individual viewing their own credit score is considered a soft inquiry which has absolutely no effect on your credit score.

As for Europe, there is no one homogeneous answer. Different parts of Europe handle credit differently. The UK and Germany are very similar to the U.S., whereas in France there is no concept of a credit rating or credit score.

With that said, generally the countries that do not have a credit scoring system are much harder to get a loan from.

> it's a specific formula that applies equally to everyone.

That doesn’t mean the formula is reasonable or transparent. An AI is also a specific formula that applies equally to everyone. It’s about how the formula takes “weight” on the individual’s differences.

> There are hard inquiries and soft inquiries

Why there are “hard inquiries”? Why someone else inquiring my credit score should affect my credit score?

I certainly don’t love the system, but the difference is “checking credit score in order to take more credit (eg a loan or credit card)” (hard) vs “checking credit score for my own knowledge” (soft). This distinction feels reasonable to me.

How is it reasonable? Anyone checking your credit score already knows that you're applying for new credit, because you're applying for new credit with them.

Dinging your score over that only punishes you for soliciting credit offers from multiple lenders at once so you can choose the best terms. Which ought to be an antitrust violation.

It's a defense against the double spend problem. It minimizes potential timing issues that could allow someone to simultaneously taking out multiple credit lines without the ability to properly evaluate the risk.

But it isn't, because the reduction in credit score from the query doesn't necessarily prohibit two simultaneous loans, it just makes the terms worse (and so makes default more likely).

And there are obviously better ways to prevent "double spend," like reporting the new loan being granted, which actually should reduce the credit score due to the new debt. It's not as if we can't make computers capable of updating this information in real time.

The interesting thing I've noticed is that opening a new credit account often doesn't actually get that information on your credit report. It sometimes only shows up after you've made the first payment (either loan or credit card). I wonder if that's why the system dings you for too many hard credit inquiries. I agree with you that there's no reason why this couldn't be fixed, though.

Or rather it discourages people from attempting to apply at multiple places to get the best offer..

Typical rate shopping is not discouraged: https://www.experian.com/blogs/ask-experian/do-multiple-loan...

That's usually not the case, though. I was just looking around at Chase's credit report tracking thing, and they have this to say on credit inquiries:

> The VantageScore credit score model takes rate shopping, e.g., for a mortgage or car loan, into consideration. All inquiries for mortgages, auto loans and major credit cards that appear in your credit file within a 14-day window are interpreted as a single inquiry.

I believe that's the case for the FICO score algorithm as well. I think a 14-day window is probably too short, but it likely does cover most situations.

But if you're applying for, and repeatedly being rejected for, extension of credit by third parties that is absolutely something a pre-existing lender would reasonably consider when thinking about risk.

You have to understand that there is a no "credit denied" feedback into the credit rating system. The standard pattern for an approved credit application is a hard-pull followed by the opening of a new tradeline, both of which affect your score. If you're applying for credit, but not being approved, then that just shows up as a hard-pull with no new line.

The FICO score is also set up specifically to ignore additional hard-pulls for the same type of credit within a bounded period; so if you go shopping for a mortgage (or an auto loan, or a student loan), then multiple queries within a 30 day period are coalesced, and only count once from a scoring perspective.

I dont think its unreasonable for one’s credit score to get dinged if half a dozen lenders check their score - that means the consumer went through the initial stages of trying to get multiple loans.

But more importantly the ding is maybe 10 points which is pretty much irrelevant to the “buckets” that define credit worthiness (>800, 700-800, etc).

Plus individual lenders can look at the report themselves and make their own determination.

> An AI is also a specific formula that applies equally to everyone.

AI’s are not “a specific formula” and not only that, it’s decision-making is opaque to humans.

>That doesn’t mean the formula is reasonable or transparent. An AI is also a specific formula that applies equally to everyone.

You can learn literally everything you need to know about how the FICO credit scoring systems works in probably two or three hours of dedicated research, even though the model itself is a trade-secret. When lenders deny credit, they're legally obliged to provide the specific reasons why.

This is not-even-in-the-same-ballpark as AI, where no-one can even tell you how inputs relate to outputs, not even the creators.

https://www.cnbc.com/2019/02/27/american-consumer-credit-rat...

https://www.ftc.gov/news-events/news/press-releases/2013/02/...

It's one of the few things thats so obvious its bipartisan. I'm amazed you think its worth defending. There should not be a punishment for LOOKING at a credit score hard or soft, that's insane.

To require someone take on debt in order to have good credit is pretty arbitrary. You can have all the money in the world, but no debt? you're fucked. The fact you can juice your credit with someone elses credit cards is also really arbitrary.

You're misunderstanding the point behind credit scoring.

Banks lending you money don't care (at least not as a primary concern) how much money you have, their first concern is whether you have a history of paying back debt you owe. A credit score scores you on your ability and willingness to pay back your debts on time.

And yes, this means someone who takes on debt with a credit card only to immediately pay the balance before any interest accrues both has a stellar credit score and no debt in any practical sense.

This also means someone who is making their house or car payments properly also has a good credit score, outstanding debts negatively affecting the credit score slightly notwithstanding.

> You can have all the money in the world, but no debt? you're fucked.

No, you’d be totally fine paying cash for anything you needed.

”Does Europe do it much differently?”

Yes (Finland and afaik other nordic countries at least).

The lenders use statistics, not individul spying to set their rates.

Uhuh.

https://www.luottotietohaku.fi/?l=en

https://oikeus.fi/talousjavelkaneuvonta/en/index/unnamed3/ba...

> You get punished for just looking at it.

Not completely false, but misleading: you get penalized if a lender looks at it with the express purpose of using it to decide whether or not to offer you credit (called a "hard pull"). You looking at your own credit report does not penalize you, and certain other activities (like getting pre-approved for a loan) won't either.

The penalty for hard pulls isn't that bad, anyway, and disappears after a relatively short while. I currently have 7 hard pulls (I've been chasing credit card rewards offers lately), but my overall credit score is still over 800.

So, GET request not only aren't idempotent but they also affect the party that is read. Who came up with that lovely system ? I gave it roughly 15 seconds of thought and came come up with multiple ways this design fucks up people lives.

It’s not a GET request on the credit history; those do not affect the score. It’s a PUT* request on the “considering taking additional credit” record.

* perhaps closer to PATCH as multiple requests in short time are combined as one (as in mortgage shopping)

The key though is this operation only happens when it’s part of a requested credit evaluation for the purposes of expanding credit. The act of requesting new credit diminishes your credit worthiness. The intent is to throttle issuance of credit by providing for a race condition - if I go out and apply for 100 credit cards at the same time, they each get a credit report without the others line of credit established because they’re all in the pending state. But each request to evaluate the worthiness of credit dampens the quality of credit. At a certain point all credit requests thereafter fail.

On the flip side it also causes a hole that’s increasingly hard to dig out of because if you have bad credit or borderline credit and apply for a card to see if you’re allowed back in at the capitalist buffet, you enter into a feedback loop of diminishing credit worthiness.

It isn’t true however that any form of read changes credit worthiness. There are many routine reads that have no impact. It’s specifically when you’ve directly requested an evaluation for expanded credit. It can’t happen passively, as sometimes banks refresh your credit history and even offer more credit automatically. It also can’t happen for other purposes than expansion of credit.

A hard-pull is not a GET; it's a POST. The semantics are "this person is actively seeking credit, and we are considering whether to extend it to them". The system regards the extent of credit-seeking behavior, and changes to its pattern, as a factor that affects the credit profile.

That's not prima facie unreasonable. If you have someone who suddenly starts applying for lots of personal loans (for example), that's probably something you'd be interested in, as someone who has previously extended credit to that person.

As the prior poster indicated, when you look at your own credit file, it's not a hard pull. Nor is it a hard pull when a creditor updates their records on you as part of ongoing monitoring of your account. It's only a hard pull when the subject is specifically requesting a new, or increased, extension of credit. Those soft-pulls are GETs.

> You get punished for just looking at it.

Can you elaborate? Everyone of the banks I use have an affordance on their website/app that allows me to see my credit score, history, etc. The bank puts it pretty front and center, so if even juts viewing it has a negative impact, why do you think the bank would make it so easy?

>The credit score system already in place in the US is frequently as arbitrary in some cases already.

people often think that the problems that pertain to their country are universal, but they would be wrong.

> Yes. As a judge can check what information was used and what algorithm applied to get to a conclusion. And it may declare that use of data or algorithm illegal.

A ban on "AI social scoring" would be neither necessary nor sufficient to make your scenario a reality.

> Yes. As a judge can check what information was used and what algorithm applied to get to a conclusion. And it may declare that use of data or algorithm illegal.

Genuinely, why does process matter? Shouldn’t outcomes?

Maybe you’re not from the US, but we have a looooong history of “good unbiased processes” leading to awful and biased outcomes.

Outcomes are what the process decides. If you had some other process that could tell you what the outcome should be, you could have just used that as the original process.

To note, the discussion is on the EU side.

Yes, process matters in that they need to be reviewable and fixable. Them being good or unbiased is a function of how much scrutinity and oversight there is, but before even talking about biases, we need to know what's happening.

Even outside of fixing the system, in most countries if your get a credit application refused you can request the info that lead to the decision and act on it (update/fix your credit history for instance). Even if you can't fix specific aspects you at least know what conditions block you and deal with it accordingly (no use in reapplying twelve times if you're banned for life for instance, same way you might want to wait 6 months if it's a matter of prescription)

So yes, knowing what, why and how is as critical as the final outcome.

Sometimes there seems to be an expectation that people are entitled to a loan if they meet some criteria.

That's not the case. As long as there is no discrimination based on a legally protected characteristic (e.g. race) you can lend to whoever you want based on whatever criteria you want.

The process is what we have to judge because in inputs are bias, so even if with a perfectly fair process you will get bias outcomes...

> To do some good may not justify the harm.

Bread can be baked to feed the needy, or to fill the bellies of soldiers who invade a peaceful neighbor. Who is to blame? The people inventing industrial bakeries, or the people using them to feed their invasion army?

> "You cannot get a loan because the AI says so" is not acceptable.

To me this makes it seem like acces to loans is a right,in which case the EU should make a lender of last resort that can loan to people who would otherwise get rejected.

How do you leap from banning black box AI decisions to thinking it means everything is allowed?

Perhaps the assumption is that humans functioning as black box loan deniers is also a problem.

Haven't loan decisions always been black box? Sometimes they'll attribute decisions to credit scores, but those scoring systems have always been proprietary, and they won't even tell you what data your scoring is based on.

> Haven't loan decisions always been black box?

Not to the people making those decisions. That's brand new.

People haven’t been making loan decisions for decades in the EU as well. You put numbers into a computer and what comes out is either an approval with a rate or a decline.

No bank EU or not has people manually approving loans.

You can file a lawsuit 'this bank does not give me loan because I am not white' and check. You can inspect the excell formula in the computer.

It does not matter who personally made that decisi9n, the bank is operating this code and paid someone to write it.

With 'AI' there is no way to know.

No, what you are entitled in the EU is to have the decision reviewed by a human, I fail to see how an AI changes that.

No what?

Let's imagine my bank is sexist, the code literally says "if WOMAN, Reject'

You can sue my bank for discrimination, and in process of discovery you can request information and documents from the bank, including the code / formulas.

If you have a solid argument, let's say my bank never approved a mortgage for a woman, the judge will grant your request. You, or someone appointed on your behalf, will look at the code / excel formula, see that it's discriminatory by design, and you can request massive punitive fines, because this demonstrates contempt for the law.

Now let's say I instruct my developers to create a neural network that is as convoluted as possible but is unlikely to approves mortgage for a woman. That can be easily done. There will be no proof that we ever did this on purpose, and even proving that it is discriminatory will be challenging. At best, my bank will get a slap on the wrist.

That is why they are banning AI from some areas of life.

That has nothing to do with entitlements…

That isn’t how it would work in reality.

It would be up to the bank to prove that their practices are non discriminatory if there will be a discrepancy between the number of applications issued.

The plaintiffs will be able to sue and win based on the outcome alone.

As far as actual rights go then you have the right to appeal any automated decision and have a human review it without any bias. Which again it doesn’t matter if it’s AI or a an algorithm based on flipping a coin.

'Black box' doesn't mean 'numbers in a computer', it means 'nobody can explain to me precisely why I was rejected'. A score calculated from an obvious formula is fine (regulations permitting) since you (or a regulator or court or the institution) can establish precisely why you were rejected. A black box system is one in which this cannot be established by any person.

This is obviously bad for both legal and moral reasons. Legally you may be (unintentionally) violating anti-discrimination laws. Morally you may be setting up a system that arbitrarily denies people who are otherwise deserving with little to no recourse.

By that definition, hiring and dating appear to be black boxes.

In both of those you can receive plausible explanations from anyone honest or introspective enough. If you think that all decisions are actually black box emotions that can't be introspected or something then yes, they would be total black boxes, and I suppose the goal would be to minimise that in any system we build that has to discriminate.

As the other commentators said, humans are also black box. They can, of course, provide vaguely reasonable justifications but these often aren't the actual reasons and ChatGPT can generate reasonable sounding rationalisations on demand.

Reply on news.ycombinator.com