Shouldn't we be allowed to everything into account when determining credit worthiness? Even if it's socially uncomfortable?
We get locally more accurate pricing / risk assessment that way, but we also create feedback loops that we as a society have decided we want to not contribute to. The cost for this is the pricing inaccuracy which we collectively eat.
So, I don't think you're wrong, but maybe consider if what we're socializing the cost of is worth the price?
It’s like applying for home fire insurance and saying you burned down your previous 3 homes and will likely burn down this one.
These feedback loops you're referring to, what order are you proposing for your chicken and your egg? What's causing what in your view?
This is a strange take. If you had to "consent" to laws there would be utter chaos. The places where you can pick and choose which laws apply to you have no real laws. And all of those places are dangerous and destitute. That's why laws are decided by the majority, there will always be dissenters for every law out there. Some harmless, but others that want to pillage, burn things down, and worse.
There will always be laws that you don't like, unless there's no laws at all.
The government plows the streets. Provides police and fire service. Makes sure street lights and sidewalks exist. Handles deeds for property. Handles court cases. If you had to "consent" to all these services providing benefits to other people the entire government would cease to exist.
"I have to consent to anything that doesn't directly benefit me" means a ton of things aren't even going to exist when you actually need them.
Which one of the causes was first in history seems academic and immaterial if we're at t>0 and wanting to devise a strategy to slow it down or stop it.
[digression into risky territory, here be dragons]
...at least, that's the idea if you believe that racial differences are 100% socially constructed. It gets super tricky if it turns out there is an inherent component, which is why the very idea is such a taboo. It wouldn't be especially shocking, from a scientific standpoint, if it turned out that different "races" tended to have subtly different inherent personality traits that might be of interest to creditors or insurers. But it would cause major headaches, if you want to convince people to break those feedback loops! Even if the inherent differences are only a seed, and the feedback loops account for most of the differences in outcome, you have a massive "political" problem on your hands.
But then on the flip side, if you look at say, athletic selection for a sport like basketball or athletics, you'd find that race is inherently a criteria, because certain races seems to have an inherent advantage (or more correctly, certain races seems to have a _disadvantage). But nobody ever says anything in those cases.
I think a charitable explanation for why this is, is sports is pretty much always a voluntary activity, so it's natural to not get as worked up over either implicitly or explicitly favoring a race, compared to, say, doing the same thing in employment or scoring someone's credit, which will almost certainly impact everyone in society involuntarily.
Using race captures an obvious group of people by how they've already been discriminated against
Depending on the situation the same people behave differently. So it's not the skin color in itself but the different context that makes races behave differently. Being poor or uneducated for example is a better prediction of defaulting on payments, but race, wealth and education are correlated.
The affordances of the context influence choices - how many good options vs bad options you have in your situation. For example, people who would behave perfectly OK in normal life might try to cheat the system to get vaccinated sooner. Because they only got bad options, they might pick a bad behavior. "I got no choice!" Equality of opportunity, and not of outcomes, seems to me the best approach.
The reverse of this is also happening - in the more developed and feminist leaning cultures fewer women choose to do scientific research. Why? They got better options, so many great choices to pick from. It's insulting to question people's choices when they are made in an abundance of good options, as if they were children who can't make their own decisions.
But you do not socialize the cost, you just move it on someone else.
For example, we have two people A, B, with risk-assessing algorithm risk1 gives risk1(A) = 3.3, risk1(B) = 2.9. Risk limit for creditors is 3, so person A would not get credit.
But risk1() uses same forbidden information, so we use instead less informed risk2() that got risk2(A) = 2.9, while risk2(B) = 3.1, so now person A would get credit, but person B would not.
Like, presumably you could improve your creditworthiness prediction by cross-referencing against bankruptcy law related searches on Google and in library checkouts. We absolutely should not allow this, though - we care a lot more about having a society where people are free to read and speak about what they want than optimal consumer credit allocation.
Credit worthiness should be because of an individuals past actions and not some attribute they may have been born with. Why should a tall individual who's never missed a payment have some invisible penalization applied because some other tall people are worse at re-paying loans.
Don't you see the issue here? You're penalizing individuals not based on their own measurable behavioral signals, but simply due to some physical trait they share with others. Not to mention, we wouldn't even understand why there's a correlation in the first place. Maybe tall people are 30% likely to have some unknown gene, why should the other 70% of tall people who don't have the gene be treated the same?
Credit risk, f(X), is an unknown population function that needs to be estimated using observed data X.
If including tallness into X improves our estimate of f(X), then we've gotten a better model.
You've asserted that X should only contain an individual's past actions instead of their inherent traits such as tallness. This may satisfy certain political objectives, and that's fine if we're being upfront about the underlying motivation, but from an ML perspective your prescription doesn't make much sense unless you have some prior knowledge about the function f(X) that tells you that tallness both isn't relevant and isn't acting as an instrumental variable for some other missing feature.
Unless you have such domain knowledge, you've little business asserting what are appropriate features to use in order to improve model quality.
If you think it is, that's fine. We might as well just taking the same approach to crime, and start locking individuals up or not extending job offers, NOT because they've done a single thing wrong, but simply because they're statistically more likely to.
Which is an important thing that isn’t math.
The main difference is that people have right to trial and to be considered innocent until proven guilty. But there is no 'right to credit'. Credit is fundamentally two-party contract.
Also there is shared limit to risk, forcing creditors to take more risk with some people means they may not take that risk in other cases (not giving credit to someone who would be marked lower risk with more informed decision) or forcing them to raise credit cost to everybody.
The crime analogy is inappropriate. Insurance is a private voluntary arrangement between two consenting entities. Convictions on the other hand are an involuntary imposition on an unwilling party.
One could make the argument that allowing inherent traits in the pricing of insurance is the less authoritarian and more utilitarian option, since it is less forceful state interference in private business and leads to more accurate claims pricing and less subsidisation of insurance for person A by person B. Your prison sentencing analogy on the other hand implies more force, which is why I don't view it as a valid analogy.
The ethical argument can go either way depending on the axioms we pick a priori. If we pick Libertarian deontological axioms, then the ethical choice is to allow inherent traits into the model. If we pick racial equity deontological axioms, then we get another conclusion.
Consider a sample size of 10/1million with height value of 7m. And somehow 7/10 had poor ability to repay loans. With such a small sample size of this relevant factor be a good thing to rely on?
Would you be OK if banks started charging you 4% more interest because say you have irish heritage? Your behaviour hasn't changed and nobody can explain why it matters that your father was irish, but the models determined it's statistically significant,
What if at some point 30 years down the line it's determined it wasn't even related to irish heritage. There was just coincidentally long running extreme predatory lending by banks in cities with large irish populations.
Would you still feel there's nothing wrong with it?
What if, one bank started doing it because they used this model. But another bank doesn't, because they don't believe this model?
If you were an irish person, you would just move banks. And if it turns out, indeed, that the first bank was over-charging the 4% (ie., no irish person defaults due to their being irish), then they've lost business for no reason, and the 2nd bank got more business.
However, if it turns out that being irish _does_ indeed cause you to default more, then the bank charging an extra 4% was correct. The 2nd bank, upon finding proof of this, would also start charging an extra 4% (otherwise, they'd be losing out money to the first bank).
So in the end, a bank can only charge the most appropriate interest rate for the market to bear.
Most banks just copy each other.
The market can remain irrational longer than you can stay alive.
Many people like that.
Consider this example: lesbians are statistically more likely to divorce. The ML models decides that lesbians are a higher mortgage risk and raises their interest rates. It does this indirectly by raising interest rates on married couples with different last names and owners of Subaru vehicles.
In another, the model sees that people who follow NBA players on twitter are more likely to default. It then assigns then higher interest rates. Well, it turns out that poor urban black people are far more likely to be interested in the NBA than the average person. Probably because it's one of the only sports you can play in a parking lot. The model decides to raise interest rates and cut lending to most black people.
"Perfect" credit worthiness modelling is only going to be beneficial to rich white and asian people. Essentially favoring the stereotypically richest ethnic groups.
This is extremely problematic for any sort of equality initiatives.
Treating everyone the same regardless of means is definitely socialist. It's the same as removing the pre-existing conditions exclusions for health insurance. It favors the unfortunate at the expense of the fortunate.
I'm not going to go into whether that is good or bad for society but I ask you to consider the extreme, historic level wealth inequality in the US where 1% of the population has over half the money.
If these models lead to good data, then what's wrong with them? If people who follow the NBA do in fact default on their loans more, shouldn't they pay higher interest?
I agree that the healthy should not be compelled to subsidize the costs of healthcare for the unhealthy. Can you tell me why it's ethical to force my compliance?
It seems you primarily see things along racial divides, grouping people into black, white, and asian. There are, more nuanced ways to group people, which is exactly what these agorythms are doing.
Also, what's wrong with wealth inequality? I'd be much more concerned about absolute quality of life, which has increased dramatically for everyone in the last half century.
- May be based on bad, inaccurate, or nonrepresentative data.
- May be based on factors intrinsic to individuals which they cannot change.
- May mask underlying bias factors (as in the NBA example above.)
- May reflect existing biases in behaviours. E.g., arrest, promotion, recruitment, or admissions data used to encode an AI selector, which merely codify biases reflected in the underlying historical data.
- May reverse causality, implying A causes B, when in fact B results in A.
- Amplify existing inequalities. Starting points in life matter, and an algorithm which simply amplifies the existing inequities of wealth, race, place of birth, religion, health, etc., compound rather than address these issues.
- Fail to consider fairness or equity in decisionmaking. This is the underlying fundamental problem in credit-based resource allocation. People need access to resources regardless of their creditworthiness, though there might well be cases in which allocations are modified or managed given behavioural issues. Reinforcing standing biases does not address the underlying inequities.
Doesn't a more accurate model in fact ensure that fewer people are lumped into groups they don't belong to, and that more deserving people (ie. people who are likley to repay loans) have access to credit?
I give you Karl Marx:
As soon as land becomes private property, the landlord demands a share of almost all the produce which the labourer can either raise, or collect from it. His rent makes the first deduction from the produce of the labour which is employed upon land.
...
Civil government, so far as it is instituted for the security of property, is in reality instituted for the defense of the rich against the poor, or of those who have some property against those who have none at all.
...
The interest of [businessmen] is always in some respects different from, and even opposite to, that of the public ... The proposal of any new law or regulation of commerce which comes from this order ... ought never to be adopted, till after having been long and carefully examined ... with the most suspicious attention. It comes from an order of men ... who have generally an interest to deceive and even oppress the public.
...
The masters, being fewer in number, can combine much more easily; and the law, besides, authorizes, or at least does not prohibit their combinations, while it prohibits those of the workmen. We have no acts of parliament against combining to lower the price of work; but many against combining to raise it. In all such disputes the masters can hold out much longer.
...
Masters are always and everywhere in a sort of tacit, but constant and uniform combination, not to raise the wages of labour above their actual rate. To violate this combination is everywhere a most unpopular action, and a sort of reproach to a master among his neighbours and equals. We seldom, indeed, hear of this combination, because it is the usual, and one may say, the natural state of things, which nobody ever hears of. Masters, too, sometimes enter into particular combinations to sink the wages of labour even below this rate.
...
A man must always live by his work, and his wages must at least be sufficient to maintain him. They must even upon most occasions be somewhat more; otherwise it would be impossible for him to bring up a family, and the race of such workmen could not last beyond the first generation.
...
No society can surely be flourishing and happy, of which the far greater part of the members are poor and miserable. It is but equity, besides, that they who feed, clothe, and lodge the whole body of the people, should have such a share of the produce of their own labour as to be themselves tolerably well fed, clothed, and lodged.
...
Wealth, as Mr Hobbes says, is power.
...
POLITICAL œconomy, considered as a branch of the science of a statesman or legislator, proposes two distinct objects: first, to provide a plentiful revenue or subsistence for the people, or more properly to enable them to provide such a revenue or subsistence for themselves; and secondly, to supply the state or commonwealth with a revenue sufficient for the public services. It proposes to enrich both the people and the sovereign.
The first object of political economy is to provide subsistence for the people.
Oh, silly me, that's Adam Smith. So hard to tell them apart.
https://en.wikisource.org/wiki/The_Wealth_of_Nations/
http://oll.libertyfund.org/titles/smith-an-inquiry-into-the-...
As well as your politics and economics, your morals and statistical understanding are deeply flawed.
Liberty itself is in opposition to flourishing and collective good. Is it right to murder a man to save 1000 innocent children? What about 5? What about one? If it is right, please do explain why.
All of this treats "the people" as entities without agency, some formless mass that the enlightened politicians have a duty to care for, whether they like it or not. In absence of divine authority, there is simply no basis for this sort sense of superiority. You decide what's right for you, and leave others to do the same. Then we can all get along.
There are numerous holes in your view. You seem strongly motivated to avoid seeing or admitting any of them.
Cheers.
You've not described how my views are amoral either. What's wrong with consent exactly?
Smith makes his case. He is the father of modern economics whether or not you subscribe to his beliefs or not. The question of equity specifically is addressed in the passage:
No society can surely be flourishing and happy, of which the far greater part of the members are poor and miserable. It is but equity, besides, that they who feed, clothe, and lodge the whole body of the people, should have such a share of the produce of their own labour as to be themselves tolerably well fed, clothed, and lodged.
That makes measurement unambiguous and shows the failure of your multiple assertions otherwise.
The values you propose of "accuracy", "efficiency", "dignity", and political vs. economic concerns (Smith clearly combines the terms) are equally if not far more arbitrary and poorly supported by empirical evidence.
I'm not going to spoon-feed him to you, though the curation of passages posted above would be an excellent starting point for one actually capable of benefiting from exposure.
The main value I'm proposing is "consent". It's rather less arbitrary than "flourishing" or "happy" or "fair". The basic premise is that all human interactions, relationships and associations ought to be as consensual as we have realized romantic ones ought to be. You're no more entitled to a surgery or a loan than a partner. If you disagree, then all I'm asking is from where do you derive the entitlement?
Because it makes health insurance impossible to get when you need it most. It makes health insurance a scam because any serious illness and your company will fire you and you won't be able to get any other insurance. Insurance becomes cheap because it's useless.
Wealth inequality is a problem because money = power. Extreme wealth concentration is not compatible with democracy because eventually these people and corporations are so wealthy they can just buy votes. Spiraling wealth inequality gets you a Russia situation, an Oligarchy in everything but name.
Your argument is not a far stretch from "why don't we just let poor and sick people die". It's pretty much equivalent to "why don't we allow anyone to discriminate however they want", which got us great things like slavery
I strongly beleive in caring for the poor and the sick. That is one of the oldest definitions of the virtue of charity. I am wholly against imposing my own personal virtues on other people though. Why are you so comfortable doing so?
I'm only explaining the world as I see it, just as you did in the comments above. You shouldn't be so dismissive of dissenting opinions.
In my opinion, your viewpoint that you should be able to choose everything you want to pay for will never work. A sizable portion of the population will decide to pay nothing while still receiving benefits, leading the rest to stop paying.
It's not like you can deposit a nickel for every street light you pass, or every crime prevented by the police. Some services must be paid by everyone if they are to exist at all because their value is not directly tangible.
Education is a good example. You may not go to a school and say "well, I don't go so I shouldn't have to pay for it". Yet you still reap most of the benefits of the rest of the population being able to read and write. If most of the population couldn't read or write the affect on GDP and job availability would be terrible and definitely effect you directly. The value is provided to some extent to everyone in society. The only way to avoid any personal benefit is to live isolated from the rest of society in the middle of nowhere.
Another clear example is government provided healthcare for infectious diseases. The government does a lot of vaccinations and it's impossible to avoid benefitting from that without cutting off all interaction with other humans. You directly benefit from others being vaccinated by having a lower personal risk of getting infected.
I think your view is too narrow and focuses on transactional exchanges, like buying gas or paying a toll. But it doesn't capture the larger benefit to everyone including yourself for things like having less criminals, illiterate people, and junkies running around. Which is the societal benefit to a lot of government programs and why everyone is expected to pay for them.
Your idea of not paying for services you don't use falls apart when considering that many of these services directly benefit you even if you don't directly consume them. And the only way to avoid the benefits is to avoid society completely.
Basically, loans should only be given to those who are 99% likely to pay them back. Anyone who is at risk of default, say because they lost their job should be denied, making a permanent descent into poverty and misery even more likely.
Health insurance should only be given to those who are already healthy. Anyone who is sick enough to significantly benefit from insurance should be rejected. Even if they have an easily treatable condition that costs $50 in medicine a month, they should be rejected, guaranteeing their health gets worse.
In a world that worked so hard over centuries to create modern healthcare, we should only use it for people who barely need it. We may have literally figured out how to transplant organs and save lives, but nah its too expensive so why bother?
In your world, the slightest shock in someone's life would mean all safety nets get pulled, guaranteeing they will never recover. Then that data gets fed back to an ML model, confirming that yes indeed, people who need money or healthcare never do well anyways, so why bother with a lost cause?
Anyways, your worldview is fucked. Hope you think much more deeply about it. Or just grow old in the world you wish for, maybe one day you will need surgery, insurance will reject you, banks will not loan you the $200k it costs, and you will die for the sake of the algorithms and profit maximization.
Once you know the level of risk you can then give loans, adjusting the interest rate to make up for the risk. By hiding information from lenders, you actually make them less likely to give a loan, rather than more likely. An unknown level of risk is 100x worse than a known high level of risk.
What are the alternatives?
In fact I have. In my estimation, if we adopted a model of interaction that emphasizes consent, the incapable and unfortunate among us would rely on charity to have their needs met, as they have for most of history. This seems much more ethical than to rely on force through taxation. It can't be compassionate to impose your idea of compassion onto other unwilling people.
It is also worth noting that as long as we consume goods made by people in conditions we don't tolerate domestically, we're being wildly hypocritical. All we've done is offshore the suffering, and in the process massively harm the environment shipping things to and from more lenient jurisdictions.
> In a world that worked so hard over centuries to create modern healthcare, we should only use it for people who barely need it. We may have literally figured out how to transplant organs and save lives, but nah its too expensive so why bother?
It should be reserved for those who can afford to pay for it. If you want a modern operating theatre, filled with expensive equipent and highly educated people, someone has to pay for it regardless of whether it's for yourself or for your horse. I'm only saying that the person who pays should do so consensually.
Likewise, if all you can afford is a street dentist with rusty tools, then I don't see from where one derives entitlement to anything else. As mentioned earlier, we already buy things made by people who live in such conditions, so the point is moot.
> Anyways, your worldview is fucked. Hope you think much more deeply about it. Or just grow old in the world you wish for, maybe one day you will need surgery, insurance will reject you, banks will not loan you the $200k it costs, and you will die for the sake of the algorithms and profit maximization.
This all may be, but I will be dying with my dignity intact, not greedily reaching into the pocket of a stranger, deluded into thinking I'm entitled to what's inside.
Maybe we can only fix one evil at a time.
The approach that rejects the rusty-tools dentist is also more likely to reject the offshored oppressive offshored (or quite frequently, domestic) labour, rather than binning all empathy looking at either, other similar situations, or worse, and shrugging it off with a "meh, not my problem".
The worldview you're expressing has absolutely no dignity.
Also I don't think we agree on the definition of dignity. In my eyes it's perfectly possible to die of disease or starvation with dignity. Otherwise you're implying that nature itself is bereft of it.
TDR-TB.
There's a summary of it on the TED Radio Hour here:
https://www.npr.org/2018/01/26/580617998/cathy-oneil-do-algo...
TED Talk https://embed-ssl.ted.com/talks/cathy_o_neil_the_era_of_blin... (video)
Cathy O’Neil Is Unimpressed by Your AI Bias Removal Tool (A RedTail Q&A) https://redtailmedia.org/2018/10/29/redtail-talks-about-flip...
Weapons of Math Destruction: Cathy O'Neil adds up the damage of algorithms https://www.theguardian.com/books/2016/oct/27/cathy-oneil-we...
Human Insights missing from Big Data: https://www.ted.com/talks/tricia_wang_the_human_insights_mis...
Further references:
Weapons of Math Detruction outlines dangers of relying on data analytics: https://www.npr.org/2016/09/12/493654950/weapons-of-math-des...
Can Big Data Really Help Screen Immigrants? https://www.npr.org/2017/12/15/571199955/dhs-wants-to-build-...
If blacks are for some reason (and I am not saying they are I do not know) less likely to pay back loans, then that is the problem that needs addressing. It might be a little painful to admit the huge racial divide but it is real and that is a problem we need to solve, taking the easy way out lying to ourselves only harms society in the long run and destroys the lives of people who cannot pay back the loans and end up in a huge debt cycle very difficult to get out of.
If there's a systematic financial disadvantage experienced by black people, possibly as a result of racism, then continuing to financially disenfranchise them will hardly help, will it? Access to liquid capital is pretty critical if you want to make money, isn't it?
Judgement is notoriously self-fulfilling. The best way to keep someone poor is to treat them like a poor person. The best way to drive someone to crime is to treat them like a criminal. The best way to keep someone ignorant is to treat them like an idiot.
Sometimes I do wonder if people who present the argument you just presented realize that that is exactly how we've been doing things, and it just keep making the problem bigger.
Maybe I'm too much of a romantic, but I still harbor hope at some point they will realize it and begin to contemplate trying something different.
Nobody is arguing against fairly assessing an individual person's ability to pay off a loan - the issue at question is whether it's okay to use race as a statistical proxy for that.
They could start by not making the problem worse giving loans to people who can't pay them back.
Maybe someone is objectively less likely to pay back their loans, less likely to succeed in school and more likely to be arrested for a crime, but when those predictions end up forcing them to live in a worse neighborhood because they can't qualify for a loan in the better one, go to a worse school because they aren't expected to succeed in the better one and live with a higher police presence because they're more likely to be arrested, maybe those predictions aren't just predicting worse outcomes but are actually reinforcing them.
Let's take a similar case: kill switches on cars. As you may know, not too long ago industry innovated a feature that allows remote disabling of a parked vehicle. This feature was used as an assurance of reposession and payment by automotive financing. They just stop the car from working on late payment. The result: lots of people who were previously unable to access automotive financing now could. This was a very good thing for everyone!
Do we want a society where it is okay to charge red heads more for car insurance because they are red heads? Why should people have to bear the sins of others who happen to look like them?
[0]And isn't intrinsic, e.g. people with big feet might pay more for shoes, but not for mortgages.
If some groups wants to charge redheads more, it's their choice. If someone else sees that as a market inefficiency, they can start their own shop where they don't do that. Who cares if you or I think it's "fair"?
Edit: My original point was, the more nuanced we can be about whom we decide to give credit to, which these sorts of techniques provide, the less someone will find themselves lumped into a group in which they don't belong, and the more deserving people will have access to credit.
s/tall/black/
(inb4 downvotes)