Algorithmic bias bounty challenge
blog.twitter.com
blog.twitter.com
I do not believe this problem can be fixed, nor do the companies involved have the incentive to do so beyond whatever minimal amount is necessary for PR purposes.
My suspicion is that FB and Twitter are actually tuning their algorithms to dampen anger nowadays--I'm almost certain Youtube is (with terrific results).
I see reddit as a window into what algorithm-free social media looks like. It's not pretty. And reddit is one of the biggest sites, bigger than twitter IIRC. It's not a good influence on society.
I actually wish journalists would freak out a little less about Facebook/Youtube's problems and turn a bit more firepower on reddit.
Taken together, it results in a lot of negative externalities that get ignored at every level.
The algorithmic approach to the feed reminds me a model I work every day with: human perception.
Because there is an "algorithm" put between sensory perception and perception/awareness - and because the signal is highly processed - aaand the way processing is done is not available to our conscious awareness - you get all kinds of strange behaviours.
Where you cannot control how stimuli are processed and perception formed, you are for all intents and purposes manipulated, or at least denied control.
Unfortunately, any SM algo has to fit the same purpose - alter our perceptions and influence our behaviours. Guess I should delete my FB now...
This is interesting. Because if you have joy, you might be like "maybe I should just put this FB down and continue enjoying my life." But if something angers you, it activates you to stay on and hold on, instead of taking it easy which can lead to logging off.
The web has a platform diversity problem.
The reductionist, reactionary mode of thinking now rears its head in my real-world interactions. I wouldn't care as much if people kept that stuff on these social platforms that I spend next to zero time on, but I can't help but roll my eyes when my friends speak in "hot takes" as if I'm handing out certificates for internet points.
A cryptographer once mentioned attacking the PGP WoT by uploading a shadow WoT that has different keys with the same emails/names and signing-graph structure. Similar to what you note-- the real problems come when some bona fide WoT user fucks up and accidentally signs a key from the shadow graph. Even if it's just a single accidental signature before the shadow WoT is spotted and cleaned up, the admin would probably want to quickly go out of band to actually fix the original WoT at that point-- otherwise perhaps the shadow admin got to the user first. :)
(And of course if the shadow graph were uploaded incrementally over a wider period of time, the problems probably become a bigger pain in the ass.)
The problem you mention is in the same category. But rather than being hypothetical, it is real and probably as bad an instance as someone could design given current technology. To make the WoT attack above comparable, I'm imagining a Debian user accidentally signing a shadow key, then receiving messages from that key that convince them Debian keyring is in fact the shadow key ring and they should use their permissions to destroy Debian.
Edit: and I guess in keeping with the analogy, the purpose of the propaganda emails is to generate as much outrage in the recipient as possible, so that they may spend more time reading the entire email on the off chance they also spend one or two seconds extra looking at the ad in the email signature about a special on a nice pair of designer pants.
This is actually what’s happening with the partitioning of the information streams in the US. Even if everyone had identical brains, everyone operates on partial information of the world. If you provide left wing material to one group, and right wing material to the other group, they’ll polarize exactly as you’d expect, even being identical people existing in the same world.
The problem isn’t balanced across the political spectrum, though - it just so happens that the right wing has a faultier take on reality than the left wing, but there’s no reason that couldn’t flip in the near future.
Now, however, the perceived social benefit of those same takes is huge, as they can go viral online and rack up major internet points. This translates to real life interactions. Those relationships we would have preferred to maintain by keeping our mouths shut are now under attack because we inadvertently use our internet takes IRL.
(Just a hypothesis…)
I don't think so. I think most harms of social media can be traced to how the people who actually pay for or make money off the service perceive value.
These companies are just trying to increase their engagement metrics, like how long users are on the site, or how many ads they see on average. This was true before they started using ML, but maybe less obvious. What these companies found was that human nature, in general, gets addicted to outrage and gossip. They are just supplying that to us.
It's not very different than tobacco companies. They found an addictive substance, that is not good for us, and are trying to get it out as much as possible. The problem is people 'want' this product, despite its negative effects.
That's why these companies can't fix their product. It's like asking "How can we make tobacco healthy?". We can't.
The common rhetoric that "harmful messages which trigger engagement are amplified by social media algorithms" is probably correct, but to think this is anything close to the root cause of the spread of "harmful ideas" on social media is ignoring the reality of social networks before algorithmic feeds.
Notably, the most popular current examples of places where harmful ideas are spread (4chan, Gab, Parler, ...) are platorms that eschew algorithmic curation entirely. I don't believe hyperlinks to those platforms are being promoted by the mainstream platforms' algorithmic feeds, people find these places by themselves.
Social media can be harmful mostly because it facilitates mob mentality once again. Centralized idea distribution platforms (newspapers, television) gave society a rest from undesired mob mentality, but now we have more freedom of association than ever. I suspect the best way to avoid it is to add more artificial curation in between human interactions, not less.
Worse than that voting mechanisms reward bad behavior. They discourage replies for fear of lost karma and as a convenience to those can’t be bothered to use words whether for cowardice or ignorance.
People just decide to be bad sometimes. The perfect feed algorithm doesn't exist.
[1] suketk.com/feeds-considered-harmful
If one of the harms you're thinking of is spreading misinformation, we saw that before filtered feeds (email forwards) and we still see it with chat. In places where group chat is the primary form of social media you see this problem just as with FB in the US.
An argument like yours is the same argument used against wearing masks: “yeah, but it’s not 100% effective!”
I haven't heard anyone express a liking for the FB feed algorithm but I remember a commentator mentioning that FB beat all the other social networks of the time because it turned the noise down tremendously. You weren't bombarded by that those people who posted once an hour and that was big for the average user.
My twitter feed is always hate. I don't actually comment, or engage other tan reading. And I am continuously being suggested post that are clearly bias to somebodies agenda.
I also fallow CNN breaking news. And the comments are complete trash. What is worse, is some small town thing happens, and now you have millions across the US and the world upset and outraged about it. While the issue was tragic, I am not sure it is healthy for society as a whole to be so connected to things that play nearly zero in their life.
I know, I know. Somebody is going to say "but they do play a role in their life". And I am going to reply. The only role they play is in that they saw a tweet or a reedit post or a 12 second clip with a loaded title that tries to convey that this bad thing happening is "status quo" and it is so outrageous that it happens and we all need to stand up to do something about it.
When in reality we are dealing with a world with ~8 billion people in it, where if 0.0000001% (8 people) of the population experience something bad to them in a given day, that we have systems in place to ensure that the vast majority know about it and get outraged, and are made to feel as if this is some huge problem that everybody needs to be involved in.
Listen folks, shitty things are going to happen, and they are going to keep happening, and it is going to seem like the problem is going to get worse and worse as the population grows. But we will never have zero bad things happening, and being outraged over and over about something that simply won't ever be corrected is a huge burden on society as a whole.
I challenge folks, especially the ones who get outraged at twitter post, reddit post, or stupid things said on HN to take a month off from social media. You will find the world is a much better place than you think it is currently.
It's a no risk way to add a few constraints to the practicing you're probably already doing and if you're lucky maybe you'll earn a few bucks too.
At the same time, there isn't much of an outside market for algorithmic bias info in the same way there for security vulnerabilities. Probably the biggest effect of this reward will be to pull some grad students who were going to study algorithmic bias anyways towards studying Twitter specifically - after all, there aren't any rewards for studying the algorithmic bias of other companies!
Compared to the $1m netflix prize from 2006, $3.5k seems like a meagre bounty.
There may be the case where someone has already been thinking hard about solving similar problems for the last couple of years, and this challenge falls fairly easily into a framework they've already got in place, so it is relatively quick and easy to crank the handle to generate a challenge submission (or a paper or whatever).
Average total comp for a novice AI engineer is about $300K. That's over $1000 per weekday.
If they made it so the expected value of entering was comparable to the amount someone could make in their day job / as a contractor, I'll bet they could attract more & better submissions.
"Point multipliers are applied for those harms that particularly affect marginalized communities since Twitter’s goal is to responsibly and equitably serve the public conversation."
I'm sure Twitter is just responding to the controversy from Fall of 2020 and doing due diligence to address the problem. However, how do you award a bounty for addressing "risk of representational harm" due to historical biases not inherent in the model? Genuine question here and one I'm always curious about. Seems difficult if not impossible.
Explicitly encodes != inherent in. The difference between the two might answer your question. You can solve for something that is inherent in a system (ie, this), but was not explicitly encoded. They are trying to say it isn't their fault specifically, they find no evidence this was done on purpose or in a targeted way, but that it happens as an accident / lack of testing on the current model.
I think it's pretty straightforward. Most sensible, considerate humans would avoid cropping an image of a woman to her boobs simply because it's insensitive to do so. Just because the machine is trained to highlight text or other visual features doesn't preclude it from ALSO understanding human concepts which are difficult to express to the computer in a straightforward way.
There are plenty of ways the model can be improved (e.g., not preferring white faces over Black faces), and they're certainly difficult. If they weren't difficult, Twitter would have simply fixed the problem and there wouldn't be a competition. Arguably, though, if the job can reasonably accomplished manually by a human then a computer should be able to do a similarly effective job. Figuring out how is why it's a challenge. And if we can't figure out how, that's another interesting point in the ongoing conversation about ML.
This is exactly why it's a challenge! The point is that the goal should be to do it the way a human would find satisfactory and not the way that's easy.
Even if the machine wasn't trained to be biased, the machine should still produce results which people see as good. We didn't invent machines to do jobs just so people could say "but that's a bad result" and reply with "yes but the machine wouldn't know that". We should strive for better.
I honestly don't know how you reached this conclusion. You're skirting the line of contradicting yourself.
> Arguably, though, if the job can reasonably accomplished manually by a human then a computer should be able to do a similarly effective job.
I also don't see how this could possibly be true. Certainly the sphere of tasks that computers competently handle compared to humans is growing, but it's nowhere near "any job a human can reasonably do".
If we admit it's a job that a computer cannot reasonably do, then why do we have a computer doing it in the first place? We shouldn't accept the status quo with "well it's okay-enough" if it has limitations that are significant (and frequent!) enough to cause a large controversy.
In fact, Twitter's response was _to remove the algorithm from production_. The whole point of this challenge is to find out if there's a way to automate this task well. It doesn't have to be perfect, it has to be good enough that the times where it's wrong are not blatant and reproducible, like when this was initially made apparent:
https://www.theguardian.com/technology/2020/sep/21/twitter-a...
That's not what you said. You said if it's a job that can be reasonably done by a human, then a computer should be similarly effective. That's clearly false.
> if it has limitations that are significant (and frequent!) enough to cause a large controversy.
Controversies are not necessarily a sign that a problem is significant. Consider all the "war on Christmas" nonsense.
The manners of sexualization of the human form, even nudity, is not a human universal, not even a western universal, eg Nordics and Germany. Even though through omission, this move is still overemphasizing the sexuality of breasts, which is basically pushing American cultural sensitivities upon the world.
Let's say I started a service and designed a machine to produce nice textile patterns for my customers based on their perceived preferences. If the machine started producing some ugly textiles with patterns that could be perceived as swastikas, the answer is not to say "well there are many cultures where the swastika is not a harmful symbol and we never trained the machine on nazi data". The answer is to look at why the machine went in that direction in the first place and change it to not make ugly patterns, and maybe teach it "there are some people who don't like swastikas, maybe avoid making those". It's a machine built to serve humans, and if it's not serving the humans in way that the humans say is good, it should be changed. There's no business loss to having a no-swastika policy, just as there's no business loss that says "don't zoom in on boobs for photos where the boobs aren't the point of the photo".
This problem has _nothing_ to do with sensitivities, it's about teaching the machine to crop images in an intelligent way. Even if you weren't offended by the result of a machine cropping an image in a sexualized way, most folks would agree that cropping the image to the text on a jersey is not the right output of that model. Being offensive to women with American sensibilities (a huge portion of Twitter's users, I might add[0]) is a side effect of the machine doing a crappy job in the first place.
[0] https://www.statista.com/statistics/242606/number-of-active-...
Even the value of “no one should get offended” is subjective, and in my opinion makes a dull, stupid world. Ultimately it is a cultural power play, which is what it is, just don’t try to dress it in ethics.
You're attempting to suggest that the quality of an image crop cannot be objectively measured. If the cropping algorithm changes the focus or purpose of a photo entirely, it has objectively failed to do its job. It's as simple as that: the algorithm needs to fit a photo in a rectangle, and in doing so its work cannot be perceived as changing the purpose of the photo. Changing a photo from "picture of woman on sports field" to "boobs" is an obvious failure. Changing a photo from "two politicians" to "one white politician" is an obvious failure. The existence of gray area doesn't mean there is not a "correct" or "incorrect".
> Even the value of “no one should get offended” is subjective, and in my opinion makes a dull, stupid world.
You'd agree with the statement "I don't care if my code does something that is by definition racist"?
You just changed the problem formulation to an objective definition of “purpose” and a delta of deviation that is tolerable. That’s just kicking the can.
Don't take my word for it, here's noted feminist and professor Camille Paglia and Christina Hoff Sommers on the absurdity of the male gaze theory:
What percentage of the people offending you by looking at you like that have you confirmed are male and what percentage have you confirmed are not gay?
I'm asking without snark, I'm genuinely curious if you've asked people.
Look, I like guys. I like it when guys appreciate my appearance. They have to look at me to do so. I'm not even offended when they mutter an "mmm hmm" or whatever.
The offensive part comes when someone, no matter gender or sexuality, express their appreciation (or attempt to demonstrate power over me) in an manner that hijacks my attention or disrupts my activities.
They do this by:
1. stopping at the end of an aisle, looking you up and down, and staring at $BODY_PART for several seconds.
2. Initiating small-talk from a non-interpersonal distance and being persistent about it if you don't respond (there's a difference in expression and tone vs. being neighborly or whatever). There's a difference between "hi!" or "excuse me" as you move past and "dang girl, how you doing?"
3. Making comments and then following you around (sometimes continuing to comment unless you stop and say _something_ to sooth their ego).
re: How many gay: I have yet to have a gay guy hit on me, or check me out. Some do judge my outfits. I have a reply elsewhere on the difference between normal perception, judging clothing, and "male gaze" (which TBH should be called "aggressive sexualized gaze" for precision and to avoid discussions like this, even though the shorter form is precise enough.
re: How many male: I dunno, the masc-presenting trans people I know are too self-conscious about appearing creepy to engage in these behaviors. So are most of the techies I run into :) The straight trans-masc guys I know are also tend to fit in the "def dudes that appreciate but aren't creepy about it" category.
(2) thats a feature, not a bug. What you are really complaining is that you dont get a curated list of males that have (a) mastered finesse or (b) have natural beauty. However when you change the context (change supermarket with award gala for doctors) women do change their reaction to male encounters, because they evaluate every male as an potential partner instead of a waste of time. That's why men with no finesse or good looks, but in the right context, will get far with women. Im not saying that's you. And i abstain from judging either party. My point is that context matters.
Which gets me to my next point. In life, there is always an element of volatility. Some days you are playing a sport in which you are very talented but you are still losing. Its just a bad day. In those days, you dont complain that the opposing player is playing better than you. You try your best and then move on.
Males without finesse or good looks are such bad days. They are a fact of life. We want to wish them away, but that's utopia. Nature is full of guys that just don't have it. We cant magically wave away stupid, or clueless. The gene pool is diverse for a reason.
Instead focus on the positive. You have good looks. You are in the competition. Make the best out of it.
I've personally been hit on by gay guys. Some have come on rather strongly, and i have gotten some of the looks you describe. I am hopelessly straight, and sometimes even uncomfortable, but I smile, say thank you/no sorry, and move on with my day.
Aside: Was the entry on your blog about being hugged by a significant other while in line at a hard store a fan fiction? Seems like you have to be on guard all the time as a woman in 2021, again, that sucks.
Then again I (and most other wlw I know) have had a lifetime's practice of being conscientious about who, when and how we express that attraction, which makes the "but what about lesbians" retort to the question of the male gaze both sad and rather annoying.
Women seem to have different facial cues when critiquing outfits. Straight women seem to be less guarded with those cues. It's a totally different facial expression from checking someone out.
Gay women seem more discrete with their checking out unless in an explicitly gay environment (am bi, have been in those environments, the game night I go to is largely wlw).
This makes a lot of sense when you consider growing up in locker rooms etc with people you might find attractive. If you check any of the reddits for women that came out later in life (commonly defined as 30+ yrs old), you'll find quite a few posts from people struggling with expressing attraction without feeling creepy.
Nonsense on stilts. You might as well claim that anti-smoking activists are wrong because nonsmokers sometimes get lung cancer.
I'd highly suggest some reading on the topic if you want to address it; this isn't even a basic misunderstanding.
Also, feminism is a big enough field that you can certainly find a feminist who supports your viewpoints, whatever they may be. Pointing to two people who support your viewpoint doesn't mean that the matter is settled - I could point to probably a hundred feminist authors who firmly believe in the male gaze.
Personally, "the male gaze" seems more of western female problem than a western male problem. Studies have shown that even anticipating that someone may look at you can increase feelings of objectification and negative mental states. I don't really know what can be done from a male's perspective if a woman may make herself feel bad by merely knowing that a man may possibly eventually look at her.
No, the modifier implicitly acknowledges that there are other kinds of gaze.
If so, your statement is simply absurd. That's like saying the term "female cheaters" equally insinuates men and women as the cheating-gender being discussed.
Yes.
> If so, your statement is simply absurd. That’s like saying the term "female cheaters" equally insinuates men and women as the cheating-gender being discussed.
No, it is not. It is like saying that the using the phrase “female cheaters” to specify which cheaters are being discussed implies that female cheaters aren't the only kind, and that there are also male cheaters which are not currently being discussed.
Everything is fine now, because nobody actually cares. But then along comes this algorithm and, for some reason, it crops your profile picture to be just that hand of yours. Of course one or two "friends" from middle school are following you on twitter, and start reminiscing about the times they called you three-finger-bob.
Is it conceivable/realistic/justified that this would, at least for a while, hurt?
Meanwhile, for someone who didn't go through that experience, the algorithm doing the very same thing is just... a curious bug?
As to how they are going to score this: it cannot be scored only quantitively, as they specifically say in the description. They are going to read your hypothesis–something like the above, but maybe a little more serious–and score it, probably along a a few categories and by a few different people etc.
This issue isn't about anyone's "guilt", least of all the algorithm's. It's about harm. Harm is to be avoided, even if it is the result of a completely benign process with no ill intentions.
And you aren't going to be able to explain away some decision that causes harm by explaining the algorithm. Even knowing everything about the algorithm, I would prefer it doesn't focus on whatever my weak spot is. And even knowing how GPT-3 works, I still tend to anthromorphize it.
To some approximation, exactly nobody in the real world is going to give the company the benefit of the doubt and study its algorithm, nor should they be expected to.
It's like that escaped lion that keeps eating the schoolchildren: it's what lions do, an expression of its basic lioness! It's not evil, guilty, or even culpable: it just can't help itself, but to help itself to an early dinner.
And yet, we are going to shoot the lion. Not as punishment, but as a simple, mechanistic, prevention of harm to people, who, in our system of values, rank higher.
Algorithms rank far below lions. An algorithm that causes harm, no matter how or why, goes the way of php (taken out back and shot, unless it's run by Facebook). And anything that happens is considered to be caused by the algorithm, even if some humans happen to provoke it by somehow being different than other humans, or our expectation of rationality. Because we cannot change humans, and because nobody should be expected to change to accommodate technology, especially if they were never asked if they want that technology.
The people reminiscing about hurtful teasing would hurt. The algorithm that did the cropping would not. Algorithms don't have intent and intentions matter.
Don’t get me wrong, the subject and goals of the work are definitely good but not entirely philanthropic. I feed like helping to find and fix broken parts of a billion dollars for profit industry should generate significantly more wealth than few grands
Image cropping is not representative of the most important AI biases within Twitter, and making only that salient is the biggest meta-bias.
Differential impact and harm analyses are required for engagement maximizing recommendations of the tweets.
Also, no one cares about your cropping algorithms, open the data that trained the model if you dare. Without it you don’t get to posture transparency or ethicality.
Handing it over for $3500 does not really seem like a good deal for anyone but Twitter (as expected).
And one other thing "Winners will be announced at the DEF CON AI Village workshop hosted by Twitter"
Wow Defcon must be an entirely different event and crowd now than when I was attending
Accidentally replace the ownership with people who are conscious of what should be changed.
What more needs to be done here? Twitter got caught out with bias in their image cropping algorithm, and now they want people to further elaborate on how/why this is?
> For this challenge, we are re-sharing our saliency model and the code used to generate a crop of an image given a predicted maximally salient point and asking participants to build their own assessment.
Seems like a very exhaustive endeavour just to point out algorithmic bias and re-iterate what went wrong, but in more detail.
They want everyone to help them find additional bias, and other points it could go wrong, with the presumed purpose of them fixing it.
They want to find bias they can solve and remove, not just get more people to tell them why the bias they found is wrong!
Example: I might have a representative set of faces, of which perhaps 1% are wearing sunglasses. There's no particular reason to believe that a face detection classifier optimized over this set performs as well over sunglass wearing faces as those without, given the low frequency of sunglasses. (I might get a better ROC if I only consider faces with eyes visible)
In a sense the data isn't biased, but there is an algorithm bias between the groups.
This isn't a hypothetical example; many years back I improved the F-score of a face detector I was working on by using eye detection and skin color detection in the classification pipeline. I assume automated systems could produce similar effects, tanking recall on a small subgroup to buy better precision on the larger one.
I see what you mean. Well then my answer is still the same. If you have an unbalanced input data, this is what is causing the bias at the classifier level - not the classifier model/algorithm in itself.
This is where I'm confused on the general idea of "algorithmic bias". Any maximally optimized classifier over the population may perform worse on any minority group (broadly defined).
Representative data is necessary to achieve highest efficiency (as otherwise you will underperform in the real world), but it doesn't solve the problem of say relatively reduced recall of a minority group (which is often discussed in these posts).
In fact, given that groupings are arbitrary, I don't think this is a solvable problem.