Don’t ask if artificial intelligence is good or fair, ask how it shifts power
nature.com
nature.com
I'm a bit uncomfortable with this kind of political posturing being published in Nature, the preeminent science journal. It seems the exact wrong forum for it.
Or you believe there is a more sinister reason?
So it’s yet another opaque system that might not have a sinister intention, but has a pretty sinister outcome.
1: https://www.vox.com/culture/2019/10/10/20893258/youtube-lgbt...
To be frank to me the algorithim seems to be a scapegoat for society's own fucked up norms and practices. It isn't a moral process and expecting it to act like one isn't reasonable - it would be like demanding a cargo value calculator stop listing concentrated drugs as the most profitable commodity by to transport per mass and volume because addiction ruin lives and overdose ends them. It is terrible but it is accurately describing the status quo.
- LBGTQ populations are small, about 4-7% (usually quoted at 5%) of the US, and are generally clustered in big urban areas. Like, if you're a queer kid in rural TN you're moving or Nashville or NYC otherwise you gonna have bad time. But that also means it's a niche demographic that's in a few areas. That's a big deal if I'm buying ads for Miami or Denver or The Castro, but it's not broad appeal across the country.
- LBGTQ isn't uniform in context and content; not a ton of lesbians are into boy-meets-boy stories, and not a lot of gay men are watching videos about Andrea Dworkin and S.C.U.M.
- Not a lot of crossover appeal -- I know a few straight bourgeoisie white ladies that are all about queer culture, trying hard to be allies and all, but other than that there is little crossover. Even my wokest, Bernie-Sanders loving friends aren't watching LBGTQ vids and related content on the regular. Again, it's niche thing.
- Serious flak from conservatives, be they Christian, Muslim, or Hindu, or a Communist Chinese government that's not especially LBGTQ-friendly. I can post e-sports videos that will garner millions of views, from Pakistan to China to Nigeria, but a queer critique of Harry Potter or League of Legends characters might be flagged or banned in most of those countries. There may be aggressive pushback and demonetization as well via lobbyists or religious groups.
- As another poster said, flagging words like "gay" or "lesbian" as pornographic and blocking or demonetizing them accordingly.
If you want to make a clearer example, here's one: Google's ad classifiers used to almost always classify you as a man if you frequent engineering websites, much to the chagrin of many female engineers working at Google.
I don't know if that's still the case, as I'm not a woman myself.
Good luck trying to posture with the help of a white,right-wing male. Sorry Ted Kaczynski. I liked your manifesto.
The other ironic thing is the author talking about privilege by publishing in Nature and working at Stanford. Sorry Pratyusha, but you are at the belly of the beast, you are as establishment as it comes, no matter your gender or ethnicity.
> Our network emphasizes listening to those who are marginalized and impacted by AI, and advocating for anti-oppressive technologies.
So do you disagree with the idea that power is problematically unevenly distributed, and disagree that listening to the marginalized and being anti-oppression is good?
If not, and you merely disagree with being explicit about the politics of technology, why? The status-quo politics of AI is "how can we make lots and lots of money from this?". Challenging that politics is good, even if you agree with amoral capital accumulation.
None of this is science. Science does not and can not make value judgements; it cannot say that any distribution of anything is "bad", it cannot say that any particular outcome is good or bad. This has been known for over two thousand years, as the https://en.wikipedia.org/wiki/Regress_argument (mentioned by Sextus Empiricus, the father of empiricism). Moral judgements are the realm of politics and philosophy, and there are plenty of journals dedicated to that. Nature is supposed to be dedicated to science.
Everywhere you go in science you will encounter moral sentiments at the basis of research. It would be totally equally scientific to experiment on babies, yet we don't do it, and listening to scientists and ethicists on these issues is part of doing science. Ethical debate is intertwined with research.
There is in fact no iron law on how to do science at all and maybe reading some Feyerabend is in order. Science is a social process, and keeping ethics out of science in fact constraints discourse about science in a way that has no justification at all. It impoverishes science by supposing that scientists behave like sort of autistic robots outside of society at large, which they of course are not.
> Conservation could be part of the toolkit for diplomacy between China and India.
> How environmental racism is fuelling the coronavirus pandemic
> Reducing your own carbon footprint is not as powerful as calling governments and companies to account.
> Remember what science owes to child refugees
> the root problem is that power is distributed unevenly
As if that's some kind of sacred unquestionable axiom that doesn't require supporting argument.
I don't think there is anything controversial or political about the quote in the article, and certainly not about the impetus to consider the moral application of AI.
I think maybe people are a little worried of what's supposed to be a 'culturally secular' journal, turning over to the groups demanding social reforms in other areas (i.e. CNN yesterday indicating that the term 'Master Bedroom' is 'racist' etc.)
The article is fine, it's on point, very fair. The author is within reason to quote her inspiration.
That being said, sure, sometimes you have to explain why you consider something to be an axiom.
But at the same time, if you where to explain your axioms all the time you wouldn't get very far. So, I guess the question here is: Is it reasonable to assume this axiom in this context?
And I would argue that it is reasonable, otherwise it would be an entierly different text, about a very different subject, and for me personally, it would be a less interesting article.
Take 7 bigotted against Y of power X each and add an 8th Y person of power X as well. They could be lynched by them trivially. If they had 7X power he could only be matched if the bigots worked in perfect unision to combine. Otherwise he would be the effective leading power.
If we are dealing with counterfactual root solutions why not use a hypothetical world where violence is literally impossible? It would be more self consistent and just as possible.
IMO academia is really bad at impacting anything mostly because they are couple of steps detached from actual engineering and management that leads to impactful deployments. Not to mention that academia overall is extremely naive - they failed for some 20 years to stand up for themselves when it comes to research publication. Some researchers even manage to loose rights to access their own research without paying big corporations... For anyone interested in raw power playing academia is like stealing a candy from a kid.
The biggest impact academia could have is on education and that's probably where the focus should be concentrated.
Your comment seems to suggest otherwise.
Winning a democratic election necessarily "imposes" upon the losers some decisions of the winners, eg. higher taxes. That's not authoritarianism, that's just politics.
This doesn't change that it's the majority imposing its will on the minority by force.
>Researchers should listen to, amplify, cite and collaborate with communities that have borne the brunt of surveillance: often women, people who are Black, Indigenous, LGBT+, poor or disabled. Conferences and research institutions should cede prominent time slots, spaces, funding and leadership roles to members of these communities.
And:
>Remarkably little research focuses on serving data subjects. What’s needed are ways for these people to investigate AI, to contest it, to influence it or to even dismantle it. For example, the advocacy group Our Data Bodies is putting forward ways to protect personal data when interacting with US fair-housing and child-protection services.
I'm unable to find any other concrete suggestions or arguments. There are some things like "In addition, discussions of how research shifts power should be required and assessed in grant applications and publications.", but "how research/AI shifts power" seems to be a very fuzzy and ill-defined concept.
People aren't going to agree on what power is and who possesses it and who doesn't and what does or doesn't constitute shifting or non-shifting of power, and to who, and how, and why. Let alone exactly how their research's tiny piece of the puzzle fits into it all.
I think it'd be clearer if this contained specific claims or requested specific legal rights beyond broad terms like "shifting power": like anonymization of data, regulations on how companies and the government can collect and use people's data, how and when law enforcement can use automatic recognition systems, etc. Maybe an auditing organization for companies' and governments' heuristic systems (AI or otherwise), to look for type I errors (YouTube LGBT+ videos as not suitable for minors comes to mind). Maybe some way to determine what technologies could be ripe for abuse.
It's like making your motto "our organization is dedicated to ensuring AI benefits the people, not the elites". Okay... who are the elites, who are the people, how could the elites and people be benefitted or harmed, etc.
Google randomly, I see:
AI Applications: Top 10 Real World Artificial Intelligence Applications
Marketing, Banking, Finance, Agriculture, Health Care, Gaming, Space Exploration, In Autonomous Vehicles.
About half of those seem like situation where fairness questions enter - banking and finance - plausibly (who gets a loan), marketing - plausibly (who gets sold what), health care - plausible (who gets treated, what groups' data is and isn't used for what, etc). The other not so much.
AI is not a field like physics, which can be roughly separated into theoretical tools and applications of those tools.
AI is creating heuristics, approximations to data that "generalize" while keep how that generalization works vague. Essentially it's a very leaky abstraction so researchers need to be concerned how that leaking happens, what it's implications are.
The former is largely task agnostic and deals with fundamental issues such as, how to train networks in an unsupervised way, how do you do hard statistical inference for intractable models in a approximate but well enough manner, how is gradient descent behaving exactly, why does it work, are there better ways for optimizing nns, what about pruning? The list goes on.
On the other hand you have applications of AI to other fields that require their own research. In biology for example you may want to segment cells and their compartments or design better point spread functions for you microscope or classify cell types. These are applied problems.
Again, as stated in my original post, striving for more diversity is a good thing and should and is done. Why make it about AI ethics and bias though when large portions of this field have no contact point with it?
I understand theoretical research exists but I think it's problematic that theoretical researchers imagine that a kind of "generic" problem exists, even when a variety of test sets exist to
I mean, is SOTA on imagenet or whatever data a theoretical or an applied question? What theoretical research in AI is so theoretical that the question of data sets doesn't appear?
How this works, why this works, coming up with the technique itself, are all data agnostic. All you did so far is write down a function f(x) -> y and a loss L(x,y), with x the input and y the output, specifying your model.
Of course you use a specific dataset to train your model in the end and see if it works. But the model and the technique itself are not grounded in any specific dataset and thus nothing in this model perpetuates bias.
Now usually the next step for you as a researcher is to evaluate the performance of your model on the test set. Lets take image net. Now there are 3 situations.
A) Your Train set is biased & your test set is biased in the same way.
B) Your train set is biased & your test set is not.
C) Your train set is unbiased & your test set is biased.
With biased I mean any kind of data issue such as discussed in this article, e.g. no women in the class “doctor”.
In situations B) & C) your model wont work well so you actually have an incentive to fix your data. This will happen to you in production if you train your tracker only on white people say.
Situation A) is likely what’s happening with imagenet and other benchmark datasets. In this case your model learns an incomplete representation of the class “doctor” and learns the spurious correlation that all doctors are men. This will work on the test set because it’s equally biased.
You go, get good test results and publish a paper about it, unaware of the inherent dataset biases. (You could have done all this also on MNIST or a thousand other datasets that do not have any issues with societal biases because they are from a totally different domain, but that’s another point).
In this entire process of coming up with the model, training and evaluating it, there is no point at which the researcher has any interest, or benefits from, working with biased datasets. Furthermore, besides potentially overestimating the accuracy of your model, there is nothing in here that would hurt society or further perpetuate biases. That is because models are generally not designed to work on a specific dataset.
Again, this is a different story when you use your model in production. In this case you are in situation B) or C) and here now lies the crux. If you can make money from this bias or maybe it perpetuates your own biases well you might keep it like that. This should be fixed. Here now is a real argument for why there should be diverse populations working on AI systems that are used in industry.
Of course having diverse populations in science is also our goal. But not to fix our datasets but to do better research.
I am not going to choose who I work with based on their race/sexuality/gender.
>Researchers should listen to, amplify, cite and collaborate with communities that have borne the brunt of surveillance: often women, people who are Black, Indigenous, LGBT+, poor or disabled.
Right, we know, only privileged straight white men escape the clutches of surveillance.
All these ideologues are doing is setting the stage for the marginalization of whites, a global minority and soon to be minority majority in the US. I can't believe this kind of casual discrimination has not only been normalized, but is openly and explicitly encouraged.
This is not ok.
Edit: you know, the same way that it takes generations to build wealth, it takes generations to build knowledge, especially institutional knowledge. If we just hand over a chunk of our industries to people because of their identities, by definition we will be even less of a meritocracy - and that's bad for all of society.
(https://en.wikipedia.org/wiki/Demographics_of_the_United_Sta... suggests "A report by the U.S. Census Bureau projects a decrease in the ratio of Whites between 2010 and 2050, from 79.5% to 74.0%")
https://en.wikipedia.org/wiki/The_Rising_Tide_of_Color_Again... was almost exactly a century ago. Might be worth a read before we go stampede cattle through the Vatican.
(similarly, no matter how loony their current CINC may be, whoever in the USAF redeployed B-52s from Andersen AFB to Minot AFB also understands aposematism)
If you want world peace betweeen nations at any cost give every nation ICBMs and nuclear warheads sufficient to break through enough of everyone's defenses - it will last unless and until thermonuclear war starts.
It will be interesting to see if giving away the software is enough.
This doesn't hold up to even a first test using a real-world example.
Consider making guns free. Even if everyone owned an identical gun with identical ammo (an asburdly artificial situation to begin with), not everyone can use a gun equally effectively.
Apply it now to crowds. If everyone owns the same gun, it takes far fewer people to band together and take physical control of a certain area. Even an individual with that same gun is effectively powerless again even two others: to kill one is to be certainly killed by the other.
Now swap the guns with fighter jets, zero-day exploits, bioweapons, fame, religious leadership, or any other form of power.
There is seemingly no scenario where your axiom seems to hold up, even in contrived circumstances.
muscles? upper body strength?
that free gun world sounds pretty good.
> Apply it now to crowds. If everyone owns the same gun, it takes far fewer people to band together and take physical control of a certain area.
that in no way follows.
The original post was about technology.
> that in no way follows.
Scenario A: No one has a weapon. Five people take control of a grocery store. How many people can take back the store without anyone dying? The answer might be as low as five. Certainly 10-15 people could do it.
Scenario B: Everyone has an identical gun. Five people take control of a grocery store. Same question.
The answer is that there is no number of people high enough to plausibly reduce the risk of death to zero. Someone with a gun can kill much faster and at greater distance.
Long story short: technology amplifies power. Sufficiently amplified power cannot be countered by equal power quickly enough to nullify the danger.
Do you want examples?
A Simon Bolivar statue was vandalized in Florida, now Bolivar was no saint but he is a South-American hero he died in 1830 and now he has been tarnished just by being a "White Slaver"(He was not). Well, let's see you can come now after the name Bolivia, named after him, you can also include Colombia, America, The Philippines, El Salvador, Dominican Republic and many many more. I have seen several prominent people just shy of outright requesting that.
The Oscars are going to change their eligibility criteria and it will be based supposedly on the British awards, quoting from a NYT article "All entries in two British film categories, outstanding British film and outstanding debut by a British writer, director or producer, are now required to increase representation to meet at least two of four diversity standards, like 'onscreen representation, themes and narratives,' and 'industry access and opportunities.' among others." Do you know what this means for art? Bye Celine, by Kafka,bye Joyce, bye Kurosawa, bye Kubrick, bye Lynch, bye Tarantino and so on. Only wholesome, clean , politically conscious entertainment will be recognized. Now, what does this remind you of?
None of the current power-holders will be affected for all these changes, so no essential change. The big traditional companies are already on-board, Silicon Valley is on board, the savvy political parties are on-board, Hollywood is on-board , the media too. What this will do is to disenfranchise and silence any dissenting, dissonant voice and create a moralist culture which will dictate in a top-down form the "correct way" to do things and interpret the world. You disagree? No Facebook for you, No youtube, no mastercard, no visa, no Stanford, no Netflix, no New York Times, no BBC, nothing for you.
[1] https://www.bfi.org.uk/sites/bfi.org.uk/files/downloads/bfi-...
The article suggests making models politically aware, which is probably standard practice in many parts of China.
This quote from the OP directly address that attitude: "When the field of AI believes it is neutral, it both fails to notice biased data and builds systems that sanctify the status quo and advance the interests of the powerful. What is needed is a field that exposes and critiques systems that concentrate power, while co-creating new systems with impacted communities: AI by and for the people."
> The article suggests making models politically aware, which is probably standard practice in many parts of China.
While that's a technically correct statement, it misses the point by a mile. Firstly, it's impossible to make non-political models: at a minimum they embed the politics of the status-quo, which is often falsely confused with the absence of "politics" (for the same reason it's hard to see the mountain itself when you're standing on top of it). Secondly, what makes a model designed to embed the politics of the Communist Party of China objectionable is not that it embeds an instance of the class "politics," but rather characteristics of that specific instance.
It doesn’t directly address it, the parent was talking about intention, the article is talking about outcome. Smooshing then together produces nonsense: “when someone wants to remove bias from data, they will fail to notice biased data”.
Surely it isn’t a given that someone trying to remove bias will believe that all bias has been removed?
I wonder if this is really a possible goal. Might it not be better to be aware of the impossibility of true objectivity, and decide to be strongly opinionated in favor of certain values? Certainly if a machine learning model not only predicts something but acts as an actual agent in the world to affect some goal, it becomes impossible to claim pure objectivity, as the AI itself now is acting to change the world in pursuit of that goal.
I mean, it's hard to argue purely "objectively" that cruelty , totalitarianism and power centralization are "wrong". Any statement about them eventually comes to a moral argument, which means that AI researchers ultimately have to take responsibility themselves for being opinionated moral agents.
"Remove bias" could just be passing the buck for support for entrenching the status quo or worse. Things like democracy, egalitarianism, freedom of thought and opportunity don't just spring up spontaneously, they were hard to fight for and win.
The problem is in finding the bias, since it's a quantity inferred ex post by checking the discrepancy between reality and expectations. And since we have come to ideologically reject the very idea that there might be reasons for the discrepancy, then it must be all up to bias.
Which I personally think it's stupid and counterproductive, since when you have actual reasons you can act upon them to improve the situation, while "bias" is only useful to play a power game in which anyone who disagrees with you can be called racist and forced to repent and apologize for his/ her "bias".
It’s open ended and simple enough to answer.