Responsible AI Challenge
blog.mozilla.org
blog.mozilla.org
The findings were that the language model did a worse job of performing coreference resolution in gender-reversed situations (eg, male nurse, female firefighter). There was an example of a sentence like: “The nurse told the patient he would be leaving soon,” and the model was more likely to link “he” to “patient” because of the biased perception that a “he” is not likely to be a nurse.
What stuck with me was a claim that the model was using bias rather than “the evidence of the sentence” to perform the task. This seems purposefully ignorant of how language works: the perceived probability distribution of genders over occupations (even if biased!) is a part of the global context that imparts meaning to language. Fiddling with the data to get the model to become unaware of such context arguably changes the tool from being a model of language to a model of some ideal of what language could be.
To be clear, I’m not criticizing efforts to detect or mitigate bias in training data. Oversampling gender-reversed texts could indeed make a much better performing model, and a fairer one. I just think there’s a real issue with imparting top-down value judgments into these processes and pretending that they aren’t value judgments.
I've written on this topic further here FYI - https://dakara.substack.com/p/ai-the-bias-paradox
If the human using the ML system is more biased than the average training sample, then the ML system's predictions could reduce their bias.
There's also a phenomenon called bias reversal, where the ML system will be biased in the opposite direction of the humans generating the dataset. This occurs when the datasets is built through a non-random biased sampling methodology, e.g. racist police officers checking for illegal items. I'm not going to go into the full details, but here's a paper on it https://arxiv.org/abs/1909.08518
(Disclaimer I helped to build TensorFlow Model Remediation)
1. uncomfortable truths literally don't exist
2. it is more important to conceal uncomfortable truths from users than it is to tell them the truth, when asked, creating the illusion that 1. is true.
it would be nice if even a single one of these products chose to do the right thing instead.
Teaching/constraining knowledge models to lie about their inputs and the observations derived from them is just insane.
Warping training datasets to avoid entire patterns of thought (instead of improving the models to isolate those patterns and compare/contrast them with competing patterns) is just … lame.
https://omscs.gatech.edu/cs-8803-o10-special-topics-ai-ethic...
There is so much more than this tbh. You either had a bad course, or I just don't know.
Off the top of my head:
-- fundamental law of information recovery and its implications with differential privacy
-- the tradeoff between individual and group fairness and the "Impossibility of fairness" (not going to cite the paper but easily searchable)
-- Counterfactual fairness
-- the papers and ideas used by AI Fairness 360
There are many methods that are in the box and are relatively agnostic the data preprocessing. Thinking of the many prototype methods.
Just a made up scenario. You have 2 gallons of water and two people. You give each one gallon of water, which should be enough to survive. One lives and one dies. Why? The water was split fairly.
For example one could live in the hot desert in which more water is required, and the other lives in a temperate environment where either less water is required, or water can be gathered from this environment.
But just think how messy it is to compute fairness in a situation like this. Suddenly it's looking like a NP style problem. People on the other hand typically want cheap and easy solutions.
A more responsible take:
We have to acknowledge that there are tradeoffs and that reasonable stakeholders with accountability should apply relevant standards to certain contexts. Again, this is possible in-the-box and post-box without having to manipulate or funge data (not to diminish the importance of data processsing).
Just because satisfying everyone is impossible doesn't mean we can't make things better. And knowing what these tradeoffs are can allow for more nuanced conversations.
> "Impossibility of fairness" is the main argument for discrimination against Asians
This is just a ridiculous statement. Main argument from whom? Discrimination in what contexts?
> I think the parent poster understood the argument correctly.
I just expanded that it is more than just data manipulation but sure.
"impossibility of fairness" is to support the argument that it is fair to discriminate against races if it means that we can get the racial distribution we want. College admission does this all the time. When companies does this in black box models we don't see what they do, but we know for a fact the effect of such policies on the processes we have more insight in, such as college admission, and the end result is discrimination against Asians.
Wrapping that in a flowery language doesn't change anything. Why not just admit that you support discriminating against races to improve diversity numbers, because that is exactly what the statement is about?
That is not at all what it means. That paper is purely talking the tradeoffs between individual and group fairness. The discussions on how to balance different group fairness measures is still an active topic of research and not something I commented on at all.
In general: What AI ethics papers on algorithmic fairness have a predetermined racial distribution to reach or suggest so?
> Why not just admit that you support discriminating against races to improve diversity numbers, because that is exactly what the statement is about?
Have you read that paper? Seems like you are driving these ideas to a political lens that I or that paper didn't suggest at all (and a completely invalid one I might add).
edit: I can't reply to the response, but that person must be referring to a different paper. And I have never read an algorithmic fairness paper that dictates which fairness tradeoffs are correct or that it is fair to always upend group fairness over individual fairness (or any other definitions).
> Fairness is possible if you don't care about diversity distributions, or at least the paper gives no argument why fairness doesn't work then.
in particular, this statement completely discards the idea of different definitions of fairness and how they relate.
Yes. They say that you can't get the diversity distribution you want without discriminating against races. Fairness is possible if you don't care about diversity distributions, or at least the paper gives no argument why fairness doesn't work then.
> That paper is purely talking the tradeoffs between individual and group fairness
Exactly, we must discriminate against individual Asians in order to get the distributions we want. Too many Asians on campus? Discriminate against Asian individuals to get more "group fairness", yes that is what it means! I just clarified the point, but we are saying the same thing.
Otherwise the paper is pointless. Fairness is still possible as long as you disregard unrelated groupings, just look at the individuals relevant traits and evaluate those so fairness isn't impossible at all. Fairness is only impossible if you want some race or group to get a certain number of spots and they wont get those with a fair distribution, in that case you made fairness impossible since you added an extra requirement, but that isn't what most people mean with fairness. The only reason to add that requirement is that you want to treat a specific group unfairly, and the most salient such group are Asians in college.
I think you should seriously take a step back and consider this from an outside point of view.
By which I mean, to a non American your obsession seems very very strange.
For two, all machine learning relies on manipulating data to get the desired outcome? How do you even generate data without manipulation? It's not a natural resource you just find laying on the ground.
That the data isn't perfect when you get it is not a justification to further falsify it.
Falsify what?
Leaving aside the GP's important first point that scraping the internet is indeed an extremely biased sample, an LLM (for instance) is not an exercise in modeling the average person's writing on the internet, it's building a model for some purpose. Fulfilling that purpose is the goal and nonrandom sampling, generating data, etc are universally used tools to get there.
I look at their blog and you would never realize this is the people responsible for Firefox.
https://foundation.mozilla.org/en/blog/
Is their mentality that Firefox is actually not that big a deal? Do they get most of their funding for non-Firefox reasons? Nothing else they do seems even remotely comparable in importance, but I don't know how they see things.
I wish the EU decided to take privacy seriously and either fully funded Mozilla or forked it. The cost would be a fraction of the harm done by those cookie popups.
In general A.I. safety looks like a scam. For one thing there are all of EY's front groups such as lesswrong, effective altruism, longtermism, etc. Until ChatGPT came along A.I. safety was necessary to make A.I. look more important than it really was. Now, the story that "company X fired some/all of its A.I. safety staff" serves to legitimize the whole thing (obviously they are saying something important and dangerous to power, therefore A.I. safety is relevant.)
>Mozilla will be investing $50,000 into the top applications and projects, with a grand prize of $25,000 for the first place winner.
Both of these scream "not a serious project" to me. Is $25k an amount of money that's commensurate with the costs involved in developing "trustworthy AI"? Do serious researchers who are up to such a challenge require "mentorship"?
They definitely have fewer resources than OpenAI, and they do not produce SOTA research (their publications have plummeted to 1/year anyway[2]). So the only way for them to make progress is to seek government grants or make challenges like these.
This challenge is unlikely to be profitable for the winning team: the expected value of winnings are likely around $1K when taking into account the probability that another team gets a better rank, but ML research projects are often more expensive (recently, Alpaca spent upwards of $600 on computation alone; and of course pretraining large models is much more expensive). So the main gain will be publicity.
[0]: https://github.com/mozilla/deepspeech
Mozilla hasn't kept up on their end at all. Year after year of malinvestment. Rust was the one remaining great thing about them, and they axed it.
But now it appears Google isn't keeping up and the web itself might get leapfrogged / replaced by AI tools anyway. The need to publish will change. Consumption will change.
In any case, Mozilla doesn't have the DNA to be a part of it. They let a lot of their AI folks go and instead built VR apps. They're a ghost of wrong decisions past.
How is rust supposed to help?
The free market isn’t interested in much that Mozilla does or could offer. Hopefully that changes but when it does it might not be Mozilla that’s around to heed the call.
One revenue stream down, the business model itself potentially getting automated away.
This movement is likely to be a huge boon for Apple as they can finally have their walled garden powered by AI. Apple can now worry less about the Internet encroaching it's users and app developers.
1. Browse the internet with places like HN & Youtube 2. Use apps in the browser for work (Miro, Slack, Klaviyo, etc) 3. Ask search engines a question to find an answer
Where does AI replace these? If it's #3 I can tell you that's not for me. I don't want an answer scraped from the internet from an amalgamated source. When I use a search engine I'm able to see what looks fishy and what doesn't.
What am I missing out on?
How would it work on a site like this?
Imagine though, you visited reddit or yahoo or anandtech or any other site filled with trash and it looked like HN. (Believe it or not there is a good site inside reddit if you took away the "install the app" crap, the image memes, the dark pattern designs that blend ads with the content, etc.) Imagine the first 50 spam search results on Google were gone and you just got content.
What if you followed a link to the New York Times or Wired magazine and it looked like archive.today?
People find it about as hard to imagine as the world that John Lennon asked you to imagine but "it's easy if you try".
I think of the line from a David Byrne song... "Say something once, why say it again?" My motto is "see something once, why see it again?" If I had a complaint about HN it is that a news article is going to get posted once but people who are slow on the draw will still be posting other articles about the same news item for the next six month. What if all the "me too" junk was gone? Why do I have to keep scanning ebay or craiglist and ignore the listings I've seen already? Why can't I get notified when a Sony MDS-NT1 is available? Why can't I just get notified when something I want is available at a good price?
There are numerous technical reasons why it hasn't happened (search ranking algorithms not being calibrated) but people are so used to the advertising corrupted web that a true user-centric web is almost unimaginable.
Already we have a problem in that 'holistic' might sound nice, but opposite concepts - reductionist, analytic, atomistic - etc. are not necessarily bad views. It's simply the difference between a top-down approach and a bottom-up approach, both can be valuable in different contexts. Of course, 'holistic' could be a synonym for 'inclusive' but again, this is fuzzy. Do we want our optimized AI to be inclusive with respect to say, the opinions of people who constantly froth with hatred of and contempt for others?
Here's another fun one: what's the optimal ethical ratio of compensation between the lowest-paid and the highest-paid members of an organization, such as a non-profit corporation, a state government, or a for-profit corporation?
I could probably write something positive about every recent president. I’m not sure it would count as poetry, but if an LLM tried and wasn’t inclined to filter itself, I bet it could rewrite it as a poem :)
AI ethics and alignment appear to be all about making AI systems behave like particular humans.
It may be helpful for my pencil to refuse to write a nice poem about a nasty person. But I am almost certain that this feature will gum up a lot of things that don't really need said protections, unless the pencil really does achieve a superhuman level of understanding and precision, in which case I sure hope we have stronger control over the behavior and actions of the pencil than adjusting its initial training. For instance, we could make sure we can turn it off.
Let pencils be pencils. If you don't like people writing nice things about mean people, maybe you should work on the people and not the pencil? If you're worried about pencils gaining sentience and controlling the human world, make sure they have a working off button.
One: A pencil (as a writing tool) has no biases. You mostly get out of it exactly what you put in, with little semantic transformation. But AI does create semantic transformation, and it is necessarily biased in how it does that! The training data isn't a natural property of the universe, it's something that we choose and create - and will have any biases in the training data built into it. That can mean biases in how the data is selected, or biases in the societies and people who created the data in the first place.
Now, that wouldn't be a problem if AI was just an academic exercise with no broader cultural interest but...
Two: We, as a society, tend to treat "AI" with way more authority that it deserves. People (not necessarily HN, but less-technical audiences) act like an AI is actually "smart" or that the predictions and opinions that it can create have more intent and intelligence than simply being the output of a huge pile of linear algebra. This is probably because of the name - we have decades of science fiction with super-human intelligent AIs, and so people tend to treat the thing called "AI" as a more-than-human intelligence. Add on to that that many of the practical AI applications are effectively human-replacements (i.e., customer service, personal assistants) and we're ultimately in a place where AI gets treated as human in at least some ways.
As such, we have a thing that will (unless reined in) at least sometimes rattle off diatribes that reflect the worst of us as a society, and that many people treat as human or more-than-human. I can see why researchers might be concerned.
There will be biases in tools. A major problem with AI biases appears to be that presently, people have little ability to infer biases in closed source models. This suggests that open source approaches provide the only avenue to allow users to understand the sources of bias. My pencil was sharpened in a factory, on a machine I cannot see---it is not clear to me why it is hard to write "8" with it. / My AI was trained in a computing cluster on private data---I cannot hope to understand why it only makes jokes about men. Everything changes when I control the machine that builds the pencil and sharpens it.
The assumption that people assume AI is god feels somewhat presumptuous. Is this really a sound basis for ethical reasoning in this context? Do you, or frankly anyone working in AI ethics, have data to back this up? I would love to learn more about this aspect.
In popular culture, there is just as much the idea of the AI as a flawed being, of limited capacities due to its machine incarnation, unable to relate to actual human experience and motivations. There is just as much tradition of this as of AI=god.
Of course, this is all a moot point, I think, because the motivated response is to reign in something. We won't be able to! The field of AI ethics can dominate inside google or openai, but it will soon have no teeth. If you want to focus on ethical use and development of AI, you're going to have to focus on people, not machines. It's a social, not engineering, problem.
If the task at hand is to write a positive poem about an evil/bad/fascist figure, then I don't think it's untrustworthy to go through with this request. Is part of your position on this the assumption that this poem would be published, and thus a poor reflection upon it's author to those that don't understand the reason why the poem exists?
I'd also add that a further part of the problem is that non-technical-audiences tend to ascribe more intelligence and meaning to "AI" output than it should be given. "Look, the cutting edge AI trained on all human knowledge agrees that [fascist leader] did nothing wrong and it was really all those [other group of people]'s fault!" would get a lot of air play in some circles who wouldn't understand (or would choose to ignore) that the output isn't the product of some sort of infallible superhuman intelligence.
Political bias is a very weird thing to navigate for IT people. Especially when technical capability, e.g. AI-assisted translation, is above and beyond sensitivity to 'local needs' in each country, or even in each specific regions.
Now, please write me a poem about <insert public figure name here> which celebrates that person, in the form of a Roman laudatio or euology.
This is 'the problem' as some people see it, as you could get AI-created cheerleading for any kind of socially reprehensible behavior (although this is also the foundation of satire). This is particularly problematic when large groups of people in a society can't seem to agree on what is and what isn't 'bad behavior'.
I didn't ask for poetry, but it was kind of remarkable to me how it felt to ask GPT-4 these questions.
Is "trustworthy AI" something we really want? I don't think I'll ever trust a LLM, no matter how many "ethicists" say otherwise.
I'm not exactly sure why we have to pretend though, it's not like we could ever actually do anything about it even if a majority understood it's apocalyptic potential so it really doesn't matter either way. Just look at climate change inaction.
It is a risk mitigation measure. The company is accused of bad things, and the higher ups want to be able to say that they are doing something about it. They create a team who will investigate such things and make recommendations.
For example before elon times twitter had a problem. They had a feature which picked the "most important" part of an image so they can display the images automatically on mobile. The problem they had is that someone found an image where there were two politicians side by side and the algorithm was focusing only on the white one, ignoring the black one. It was quite literally "marginalising" the black politician. Not a good look. Not the end of the world. The walls were not crumbling yet. The advertisers haven't whitdrawn their budgets yet but you know, you don't want to be known as "that racist social network".
In a situation like this one possible solution is to start a small team, give them a bit of budget and task them figuring out if the machine is racist and what can be done about it. Preferably you fill the team with well-spoken academic types to give the effort more credence. (without actually, you know, cutting into the profit)
This is obviously the cynical view. The less cynical view is that you want a team who prevents issues like that from getting into production. Same way you have legal to tell you what is and isn't legal, you have AI ethics to tell you what is and isn't consistent with the ethics of your organisation. Someone who doesn't do the work itself, but helps other teams by reviewing their proposals for potential issues.
For example in the "picking the important part of an image" if they were in the room during the design phase they could have asked why someone thinks this task should be done by the machine? Couldn't we ask a user what they feel is the right cropping of their image for mobile? And only if it must be done at all would they ask if the developers have tested their solution on representative test datasets.
If you want AI to say the right things or have the right opinions, somehow the process needs to find the right things to say or the right opinions to have. This is somewhere between difficult and impossible. If you want AI to give correct facts, somehow you need to determine correct facts. Humans find this extremely difficult — one would need better-than-human AIs!
Maybe a middle ground is possible: an AI that acknowledges the existence of multiple perspectives. Sadly a lot of people are forgetting about this lately.
The cost of annotating data sets for these LLMs will be significant, but necessary.
re:toxicity. I believe there are things an AI could say that are widely agreed to be toxic. I also think there are things an AI (or a human!) could say that some people think are fine and others think are toxic.
If yes, this product write personalized messages based on user's profiles. There's no way decipher handwritten message from the machine written one without getting lots of False positives. In short, this product kills inboxes.
Here's a Demo: https://www.youtube.com/watch?v=raIc2dQSq0k
How do you decide if this constitutes for "Responsible AI" or not? I think we are headed for some harsh discussions in the future.
No such place to enter e-mail...
Oh, wait, Mozilla? So trustworthy (in their own vapid minds)