Sharing learnings about our image cropping algorithm
blog.twitter.com
blog.twitter.com
https://twitter.com/colinmadland/status/1307111816250748933
(Note the thread displays differently now because Twitter have changed their cropping algorithm)
Originally @colinmadland was trying to post examples of how Zoom virtual background had removed his black colleagues head, however when he posted the side-by-side images (with heads) on Twitter, twitter always cropped out his colleague and just showed him, even if he horizontally swapped the image. So, while trying to talk about an apparently racist algorithm in Zoom, he was scuppered by an apparently racist algorithim in Twitter.
It was widely covered in the press at the time https://www.theguardian.com/technology/2020/sep/21/twitter-a...
As an aside, “I think plenty of people would say X” is not a very good way to phrase an opinion. It’s ok to say you would say X and argue for it, rather than ascribe the opinion to some undefined group of other people.
I am for one happy we have unneutered the word from having to reach a bar so high, it wouldn't apply to most bigotry, but it is also unfortunate for people who have not caught on and believe calling a thing racist is a damning statement of evil intent, but it really is not anymore. Or those that insist on meaning of words remaining static forever.
ok we need pictures of human faces, luckily I've got all these white people here!
on edit: it was racist in result, in that it empowers a racist system, it was not racist in intention - as in the people gathering the training data probably didn't say hey how can we empower a racist system with this?
Google exploited homeless black people to develop the Pixel 4’s facial recognition AI - https://thenextweb.com/news/google-exploited-homeless-black-...
an attempt by Google to improve its facial recognition algorithms by collecting data from people with dark skin is raising further concerns about the ethics of the data harvesting - https://www.theguardian.com/technology/2019/oct/03/google-da...
The Twitter example was not a racist algorithm. It would consistently pick one head over the other, but it had nothing to do with the skin colour. It might preference the black head for some pairs, and the white head for other pairs.
In the second example people anthropomorphised the algorithm. They assumed that any example of a preference for an images was due to a racial bias. It was easy to keep feeding it images to get to an input that confirmed this assumption.
The blog post mentions several times how ML might not be the right choice for cropping; but their conclusion was...to keep using ML for cropping. I hope someone got a nice bonus for building the model!
If you go attach an image to a new tweet, you will see it gets uploaded right then and there, whether or not you ever hit post. Who knows what happens from there. Allowing clients to demand more crop work, at will, on top of the one that is automatic seems like it would have some implications at twitter's scale. At the very least, it would probably undo some optimizations they've made around image handling.
Cute puppy nose -> click -> porn ad.
Mitch McConnell -> click -> porn ad
My takeaway from the article was that their conclusion was to remove cropping from the product, starting incrementally on iOS. (I got cropping removed on Android as well recently). That seems like the opposite of "keep using ML for cropping"?
Find the larger dimension of the image. Remove either the first or last row/column of pixels, based on which had less entropy. Keep repeating until the image was a square.
The most notable "bias" of this algorithm was the male gaze problem identified in the article. Women's breasts tended to have more entropy than their face, so the algorithm focused on that since it was optimized for entropy. To solve the problem, we added software that allowed the user to choose their thumbnail, but not a lot of users used it or even realized they could.
I assume they've since upgraded it to use more AI with actual face detection and so on, but at the time, doing face detection on every image was computational infeasible.
Clearly there is human-derived input in the system (otherwise... What's the point just crop randomly)
https://github.com/reddit-archive/reddit/blob/753b17407e9a9d...
But in short, it's a histogram of the values of the pixels.
Thanks for the insights!
Someone wrote and tested this algorithm, and either:
a) didn't test it on pictures of women, or,
b) didn't notice that it cropped breasts rather than faces, or,
c) didn't think that was a problem.
If they had noticed and cared, this wouldn't be the approach in use.
Maybe they weren't thinking about boobies a whole lot, tried like 5 random test images and shipped it?
Presumably there are a ton of failure modes for that algorithm, why get so moralistic and high-horse about just one?
Not doing so; just observing the facts.
If a proposed 'unsupervised' algorithm of this simplicity highlighted women's faces perfectly, but zoomed in on men's receding hairlines, it wouldn't have made it past the drawing board. Indeed, it's reasonable to believe that nobody would have noticed that it consistently worked for women. We certainly wouldn't know that this algorithm existed or be talking about it here.
We observe a bias in what is considered important to check before shipping.
The simplest case is just to pick the most central square, and then you could probably improve that by picking a standard square according the rule of thirds. Those are the naive algorithms - this choice of alternative algorithm is deliberate and isn't as naive or simplistic as you're claiming.
The algorithm is only considered useful because it appears to do better than that, on whatever examples that the developers tried (i.e. there was a business case for using it), and against other possible code.
Including, likely, pictures of their own selves. That's certainly what I would test it on, until it vaguely worked.
What 'awful lot of assumptions' do you think I am making? I don't imagine we are in disagreement about this.
Years ago reddit was, IIRC, not very staffed at all compared to their traffic. It's a pretty privileged take to say they should have done expensive QA entirely around your particular things that you care about.
If you don’t have the capacity to use new technologies without increasing harm, maybe you don’t have the capacity to use them.
And it was a naive image cropping algorithm, years ago, and not making use of any sophisticated 'new technologies'. The beauty of the algorithm is that it was a simple function that could have been written in 1975 and required no training, deep learning or any of that. If you want to talk self-driving cars, you've got a much more relevant measure of harm and I'm right there with you.
As it is, I'd say there's a disconnect between where you and those years-ago shoestring developers stand on Maslow's hierarchy of needs. They were being scrappy with limited resources, and you're mad at them for not having an amount of QA that would have seemed unbelievable to them under their resource constraints.
Right. I’m saying that not caring is the problem. We mostly agree on the facts. I’m objecting to not caring as an acceptable position.
We’re not talking about shoestring developers. We’re talking about platforms that millions of people including heads of state use to affect billions of people’s lives.
Jedberg was >5 years ago, maybe closer to 10 for this algorithm. They were relatively shoestring compared to now, and especially compared to current-day FAANG.
I’m not objecting to an algorithm that exposes me to more cleavage. I couldn’t care less. I’m objecting to an algorithm that exposes millions of men to a subtle but routine equation of men with faces, and women with breasts. It’s not a privilege for women to expect not to be casually sexualized because someone thought their algorithm was helpful but didn’t bother to find out it wasn’t.
Likewise the impact of racial bias. I’m not offended by seeing white faces. But it’s not a privileged position for POC to object to being basically erased from images because whoever developed the algorithm didn’t bother acknowledging more than one skin tone exists.
These things have real impact, not just on how it affects the people who are directly underrepresented. It also affects how the people who are overrepresented perceive and ultimately act toward them.
If we are aware of a negative principle that infests and invades every part of life, though we do our utmost to repel it and mitigate it and remediate it, how can we maintain the conviction that it is not the natural order of things, but is a product of solvable human faults?
I'm not denying the importance of anti-racism or of examples of bias as mentioned here; my point is that when faced with an all pervasive adversary, whether a personified devil or something more abstract, human minds tend to find relief in submitting to the seemingly inevitable via some rationalization.
Aha, perhaps that's the problem then.
edit: I googled "woman" and selected random pictures which showed the whole upper body, entropy summed over each row to the right: https://imgur.com/a/oVB57gu
(She has no work dress code she needs to conform to. Rarely does she wear a top as plain as any of your examples.)
So the solution all along was to give users the ability to crop their own photos. Why wasn't this the original way of doing things?
Instead of forcing a complicated algorithm into the Twitter experience, it seems to me that the solution all along was just to let users do what they do best-- make tweets for themselves. This incident strikes me as a major failing of AI: We are so eager to shoehorn AI/ML into our products that we lose sight of what actually makes users happy.
Someone wanted to do a feature so they could get promoted. Probably with some mumbo jumbo about how it reduces the number of clicks to create a tweet and thus increases revenue.
This seems like it should have been a foregone conclusion. What was the driving force in the first place to think cropping images with an AI model was desirable? Seems like ML was a solution looking for a problem here, and I'm glad they've realised that.
Twitter crops photos to fit their preview formats. It seems like an obvious improvement to show people's faces when cropping, etc.
So, previously, the preview was just cropped in the center. But this made some images look funny, since people's faces would get chopped off.
Coming up with a workable solution to this with ML is not especially hard. You can get things like face detection off the shelf, maybe just tell your autocropper, "crop closer to the face" and have a demo within a couple days (and then much more effort to productionize it). From there, you can start introducing ML models to improve on your basic face detection. (I'm not counting face detection as ML.)
This is not a case where some massive ML model is being brought in to save two seconds of your time. This is a very natural and obvious application of ML, at a company which already does ML at scale, in a way that sounds like it has a good chance at improving the appearance of the site without introducing additional friction.
Instragram gets around this by encouraging everyone to take square photos.
The parent comments are basically saying that, just not in such an exaggerated way. "Give the users an image cropper, and be done with it."
https://blog.twitter.com/engineering/en_us/topics/infrastruc...
All those examples show large improvement. Of course they might cherrypick images with large improvement for their blog advertising the feature. But still, it illustrates why people would think it's a good idea.
Of course they don't seem to consider the idea of not cropping at all.
But "learnings" is literally, exactly, just a synonym for "lessons." Can we not?
In my experience, there's a tendency toward folksiness in certain varieties of corpspeak that causes rejection of "formal-sounding" terms and repurposing of "plainer" forms to create new words, hence lessons = learnings, protégé = mentee, and so on.
Do you think "cookings" is a word? It would be "things that have been cooked" I suppose, on that basis. I would say if it's a word it's one I've never heard before.
On the other hand, "cuttings" is clearly a word in common usage in horticulture.
It bugs me because it emerged out of tech executive culture and caught on in large part because others used it as an aspirational signal to sound like they are members of that group.
"Learnings" to me sounds like a mistake a non-native English speaker would make, perhaps by incorrectly extrapolating from the word "teachings". It's a forgivable mistake, but I don't see why we need to adopt it into the native vernacular.
(Yeah, yeah, I know, language evolves, prescriptive versus descriptive etc.. In a world where "literally" no longer means literally, I'm not going to waste much time getting angry about "learnings".)
> In comparisons of men and women, there was an 8% difference from demographic parity in favor of women.
would have been clearer (and more correct) as "an 8 percentage-point difference from demographic parity". That 8 pp difference though is a 16% "relative" difference (58/50), or more starkly "The algorithm chose the woman almost 40% more often" (58/42 => 1.38). That said, the diagram in the post [1] is much easier for humans to parse and say "wow, that looks pretty far off!".
tl;dr: A number like 8% sounds like "no big deal", but 8 percentage points (on each side) is a big deal!
[1] https://cdn.cms-twdigitalassets.com/content/dam/blog-twitter...
It's hard to believe that the bias was only 4% - there were a lot of people testing with images that they sourced themselves, and the preference for white people seemed much closer to 80-20.
The paper authors mention that their training data is from Wikidata (pictures of celebrities). I wonder if the types of photos in that dataset are meaningfully representative of the kinds of photos that people usually post to Twitter.
It's very easy to believe the bias was near-zero given you are citing highly motivated people on Twitter cherrypicking from thousands of examples and a little baffling you find that to be more credible than controlled systematic experimentation; note, for example, the extremely striking fact that the fuss completely missed the other bias they found which was several times larger - showing how totally useless people on social media are for testing these things and how they can conjure up "80-20" biases which don't exist.
One of the reference threads that identifies the issue happened on it by accident highlighting a surprising experience in another product (Zoom). Believe it or not, people who care about this stuff are not looking for stuff to complain about, we’re tired and overwhelmed. And I would hope that people who, upon discovering a vulnerability find and catalogue the ways it can be exploited, would be celebrated here.
I of course hope that the systems I use aren't racist against my loved ones. I am motivated to confirm whether or not they are, but I didn't go on to parlay my findings into an essay for clout. I gained nothing from doing this, except the knowledge that Twitter was suckier than I knew.
You don’t even have to add extra required steps, generate the initial thumbnail automatically, then allow the user to move the focal point or zoom it.
Sites that allow uploading profile photos have had this type of functionality for more than a decade.
Edit: I realize I missed an opportunity for a funnier way to make this observation, which is my curse.
Going by the numbers they report all of the biases seemed relatively small. Slight favor for women over men and white over black and no evidence of male gaze - but single digit percentages in each case. I wouldn't be surprised if humans were more biased than machines given the results I saw.
I would argue that a 7% preference for White women over Black women is pretty darned significant. But beyond that, cropping a photo in a way that disrespects the intent of the poster is not the right approach. Fortunately, Twitter realized that it goes beyond the numbers.
It seems like Twitter, and you, are saying they should try to minimize responsibility for bias in image cropping. Right now Twitter is responsible for it because their algorithm does it. If they foist the responsibility to users, then users will be responsible, but is there more or less bias in image cropping? No idea.
I think, if bias in image cropping is a problem, then they should check to see whether their change increases or reduces the problem. If bias in image cropping isn't a problem, then who cares. If their problem is that they don't want to be responsible for bias in image cropping then of course making the users responsible is the right way to go.
The bar would be different, in my mind, if we let people choose whether to use this algorithm or opt out.
Also, as Twitter finally came to at the end of their post, the much bigger issue is that this algorithm doesn't respect people's desire to curate their own public image. Taking out the bias aspect, if I post a group photo that includes my best friend, and the algorithm crops them out of the thumbnail entirely, then that's going to be really frustrating to me. If I want to center a person of color or a woman (or an LGBTQ individual, which we don't know how the algorithm would rate - we don't have the data), and the algorithm makes a different choice, then my message is damaged.
https://images2.minutemediacdn.com/image/upload/c_crop,h_135...
It's disappointing that Twitter is trying to deflect blame for their failures onto "men".
[Edit - this is inaccurate]
They...aren’t. Leaving aside the question fo whether identifying the “male gaze” as a cobtributor to problems with the algorithm’s output would be blame passing when the entire context is taking responsibility for and addressing problems resulting from the choice of algorithm (hint: no, it wouldn’t), they initially identified that as a potential thing the saliency algorithm might* be biased for, constructed a test, and determined it was not a factor, so they are saying “male gaze” wasn’t a problem.
Kneejerk reactions to the use of the phrase without considering anything about what it is being used to say aren't helpful.
Imagine how much work, how much energy and effort, went into building an ML-based image cropping feature, just because an up-and-coming Designer decreed that a certain specific crop ratio was the most aesthetically pleasing to the human eye...
...so that years later, after countless hours of additional user research, they would just remove the feature because it doesn't work, and award themselves a medal for doing it.
The ML-powered auto-crop was the engineering solution for how to deliver the best version of that.
They don’t say it in this article but I suspect the need to fit more tweets into the viewport was obviated by smart phone UI. The screen is tall and it is super easy to scroll quickly with the flick gesture. No need for short images anymore.