Hacker News Reader

Comment by stolenmerch | Hacker News Reader

"Although we find no evidence the saliency model explicitly encodes male gaze, in cases where the model crops out a woman’s head in favor of their body due to a jersey or lettering, the chosen crop still runs the risk of representational harm for women. In these instances, it is important to remember that users are unaware of the details of the saliency model and the cropping algorithm. Regardless of the underlying mechanism, when an image cropped to a woman’s body area is viewed on social media, due to the historical hyper-sexualization and objectification of women’s bodies, the image runs this risk of being instilled with the notion of male gaze."

"Point multipliers are applied for those harms that particularly affect marginalized communities since Twitter’s goal is to responsibly and equitably serve the public conversation."

I'm sure Twitter is just responding to the controversy from Fall of 2020 and doing due diligence to address the problem. However, how do you award a bounty for addressing "risk of representational harm" due to historical biases not inherent in the model? Genuine question here and one I'm always curious about. Seems difficult if not impossible.

View on HN

These puritan activists larping as tech savvy victorian moralists should be ignored. While the “male gaze” jibe sounds enlightened and à la mode, it implicitly denies that women (especially the lesbian ones) have a gaze of their own, or asserts that it has been colonised by the omnipotent male gaze.

Don't take my word for it, here's noted feminist and professor Camille Paglia and Christina Hoff Sommers on the absurdity of the male gaze theory:

https://www.youtube.com/watch?v=a3pjGd1v5xY

Yet somehow lesbians aren't the ones eyefucking my breasts when I'm at the grocery store.

I think this is an interesting comment and I immediately thought to ask:

What percentage of the people offending you by looking at you like that have you confirmed are male and what percentage have you confirmed are not gay?

I'm asking without snark, I'm genuinely curious if you've asked people.

Didn't say I was offended, but let me elaborate on that part, because I think some guys get confused on what's offensive.

Look, I like guys. I like it when guys appreciate my appearance. They have to look at me to do so. I'm not even offended when they mutter an "mmm hmm" or whatever.

The offensive part comes when someone, no matter gender or sexuality, express their appreciation (or attempt to demonstrate power over me) in an manner that hijacks my attention or disrupts my activities.

They do this by:

1. stopping at the end of an aisle, looking you up and down, and staring at $BODY_PART for several seconds.

2. Initiating small-talk from a non-interpersonal distance and being persistent about it if you don't respond (there's a difference in expression and tone vs. being neighborly or whatever). There's a difference between "hi!" or "excuse me" as you move past and "dang girl, how you doing?"

3. Making comments and then following you around (sometimes continuing to comment unless you stop and say _something_ to sooth their ego).

re: How many gay: I have yet to have a gay guy hit on me, or check me out. Some do judge my outfits. I have a reply elsewhere on the difference between normal perception, judging clothing, and "male gaze" (which TBH should be called "aggressive sexualized gaze" for precision and to avoid discussions like this, even though the shorter form is precise enough.

re: How many male: I dunno, the masc-presenting trans people I know are too self-conscious about appearing creepy to engage in these behaviors. So are most of the techies I run into :) The straight trans-masc guys I know are also tend to fit in the "def dudes that appreciate but aren't creepy about it" category.

Fair points, but I'm not sure how this ties in with twitter's image cropping algorithm. Not saying chest crops are ideal, but IMO twitter has other issues which are at least 100x more important that they should be focused on. If you really can't handle the way the image is highlighting your chest you can always crop your chest out of the image before uploading.

I'll say something that i hope can be read objectively

(2) thats a feature, not a bug. What you are really complaining is that you dont get a curated list of males that have (a) mastered finesse or (b) have natural beauty. However when you change the context (change supermarket with award gala for doctors) women do change their reaction to male encounters, because they evaluate every male as an potential partner instead of a waste of time. That's why men with no finesse or good looks, but in the right context, will get far with women. Im not saying that's you. And i abstain from judging either party. My point is that context matters.

Which gets me to my next point. In life, there is always an element of volatility. Some days you are playing a sport in which you are very talented but you are still losing. Its just a bad day. In those days, you dont complain that the opposing player is playing better than you. You try your best and then move on.

Males without finesse or good looks are such bad days. They are a fact of life. We want to wish them away, but that's utopia. Nature is full of guys that just don't have it. We cant magically wave away stupid, or clueless. The gene pool is diverse for a reason.

Instead focus on the positive. You have good looks. You are in the competition. Make the best out of it.

I've personally been hit on by gay guys. Some have come on rather strongly, and i have gotten some of the looks you describe. I am hopelessly straight, and sometimes even uncomfortable, but I smile, say thank you/no sorry, and move on with my day.

Is that a thing that happens to you? That sucks.

Aside: Was the entry on your blog about being hugged by a significant other while in line at a hard store a fan fiction? Seems like you have to be on guard all the time as a woman in 2021, again, that sucks.

Oh, didn't realize I'd posted that. It was a dream that I remembered long enough to write it down.

Yeah, somehow I never get catcalled or followed around by lesbians, it's almost like being a creep and being attracted to women aren't inextricably linked?

I'm attracted to women and it honestly baffles me to see unfortunately many straight men be so wildly inappropriate any and everywhere. It's...not even that hard to not do that?

Then again I (and most other wlw I know) have had a lifetime's practice of being conscientious about who, when and how we express that attraction, which makes the "but what about lesbians" retort to the question of the male gaze both sad and rather annoying.

That you IMAGINE are eyefucking your breasts. Not the same

Well, men do outnumber lesbians by something like 30:1. Also not sure if you have les-dar or how you know if a woman checking you out is a lesbian or is just critiquing your outfit in her mind.

I live in an area with a unusually high number of wlw :)

Women seem to have different facial cues when critiquing outfits. Straight women seem to be less guarded with those cues. It's a totally different facial expression from checking someone out.

Gay women seem more discrete with their checking out unless in an explicitly gay environment (am bi, have been in those environments, the game night I go to is largely wlw).

This makes a lot of sense when you consider growing up in locker rooms etc with people you might find attractive. If you check any of the reddits for women that came out later in life (commonly defined as 30+ yrs old), you'll find quite a few posts from people struggling with expressing attraction without feeling creepy.

Since there are many times more straight men than lesbians, I would expect that most people who do it would be straight men, even if lesbians were just as likely to do it as straight men on a per-capita basis.

That's easily explained by internalized homophobia ;-)

> it implicitly denies that women

Nonsense on stilts. You might as well claim that anti-smoking activists are wrong because nonsmokers sometimes get lung cancer.

I'd highly suggest some reading on the topic if you want to address it; this isn't even a basic misunderstanding.

are you living in society??? I'm tired of the HN crowd which wants to find BS counterarguments to EVERYTHING as if it's all a huge engineering problem that can be addressed by thinking from first principles. they need to understand that life as a human isn't all black and white, there's a lot of grey! and no, I'm not saying that questioning things is bad. being obnoxious and ignorant of society is.

Doesn't "the male gaze" imply there are other gazes? Otherwise, wouldn't it just be "the gaze"(as Sartre originally theorized)

Also, feminism is a big enough field that you can certainly find a feminist who supports your viewpoints, whatever they may be. Pointing to two people who support your viewpoint doesn't mean that the matter is settled - I could point to probably a hundred feminist authors who firmly believe in the male gaze.

Personally, "the male gaze" seems more of western female problem than a western male problem. Studies have shown that even anticipating that someone may look at you can increase feelings of objectification and negative mental states. I don't really know what can be done from a male's perspective if a woman may make herself feel bad by merely knowing that a man may possibly eventually look at her.

> While the “male gaze” jibe sounds enlightened and à la mode, it implicitly denies that women (especially the lesbian ones) have a gaze of their own,

No, the modifier implicitly acknowledges that there are other kinds of gaze.

I sometimes also think of the "third world gaze" that confronts me whenever I visit the less-developed parts of the earth. It's as though I walk around with dollar signs on me. I shouldn't have to fear for my safety like that. Their behavior has to change.

Are you considering the modifier here to be the word "male"?

If so, your statement is simply absurd. That's like saying the term "female cheaters" equally insinuates men and women as the cheating-gender being discussed.

> Are you considering the modifier here to be the word "male"?

Yes.

> If so, your statement is simply absurd. That’s like saying the term "female cheaters" equally insinuates men and women as the cheating-gender being discussed.

No, it is not. It is like saying that the using the phrase “female cheaters” to specify which cheaters are being discussed implies that female cheaters aren't the only kind, and that there are also male cheaters which are not currently being discussed.

No, you are delusional. This is called sexism, but you can't admit what's right in front of your face.

I believe the principle is fairly easy to understand: imagine you're missing two fingers. Because kids are cruel, your middle school experience involved lots of shaming and name-calling, and that really got to you at the time.

Everything is fine now, because nobody actually cares. But then along comes this algorithm and, for some reason, it crops your profile picture to be just that hand of yours. Of course one or two "friends" from middle school are following you on twitter, and start reminiscing about the times they called you three-finger-bob.

Is it conceivable/realistic/justified that this would, at least for a while, hurt?

Meanwhile, for someone who didn't go through that experience, the algorithm doing the very same thing is just... a curious bug?

As to how they are going to score this: it cannot be scored only quantitively, as they specifically say in the description. They are going to read your hypothesis–something like the above, but maybe a little more serious–and score it, probably along a a few categories and by a few different people etc.

A better comparison would be if there was some salient feature surrounding my hands such as me holding an object. The object would be the equivalent to the jersey number. It's clearly not cropping on the thing that makes me feel bad but it did anyway. To avoid this, it has to be trained to take into account all possible human reactions. Basically, it has to know that these pixel values activate some social controversy and it's best to avoid. Not saying this isn't possible, but it seems absolutely fraught with peril and potentially more harmful than the original saliency, especially for social constructs with no clear consensus.

Yes, I didn't mean to suggest any specific reason for the algorithm's output, because it doesn't matter.

This issue isn't about anyone's "guilt", least of all the algorithm's. It's about harm. Harm is to be avoided, even if it is the result of a completely benign process with no ill intentions.

And you aren't going to be able to explain away some decision that causes harm by explaining the algorithm. Even knowing everything about the algorithm, I would prefer it doesn't focus on whatever my weak spot is. And even knowing how GPT-3 works, I still tend to anthromorphize it.

To some approximation, exactly nobody in the real world is going to give the company the benefit of the doubt and study its algorithm, nor should they be expected to.

It's like that escaped lion that keeps eating the schoolchildren: it's what lions do, an expression of its basic lioness! It's not evil, guilty, or even culpable: it just can't help itself, but to help itself to an early dinner.

And yet, we are going to shoot the lion. Not as punishment, but as a simple, mechanistic, prevention of harm to people, who, in our system of values, rank higher.

Algorithms rank far below lions. An algorithm that causes harm, no matter how or why, goes the way of php (taken out back and shot, unless it's run by Facebook). And anything that happens is considered to be caused by the algorithm, even if some humans happen to provoke it by somehow being different than other humans, or our expectation of rationality. Because we cannot change humans, and because nobody should be expected to change to accommodate technology, especially if they were never asked if they want that technology.

> Is it conceivable/realistic/justified that this would, at least for a while, hurt?

The people reminiscing about hurtful teasing would hurt. The algorithm that did the cropping would not. Algorithms don't have intent and intentions matter.

A person was hurt. There are multiple steps in the chain of causality, but only one you can change. So you do.

Multiple steps in the chain, but only one with moral agency and thus moral obligation to do so. That agency doesn't lie with the algorithm.

> However, how do you award a bounty for addressing "risk of representational harm" due to historical biases not inherent in the model?

I think it's pretty straightforward. Most sensible, considerate humans would avoid cropping an image of a woman to her boobs simply because it's insensitive to do so. Just because the machine is trained to highlight text or other visual features doesn't preclude it from ALSO understanding human concepts which are difficult to express to the computer in a straightforward way.

There are plenty of ways the model can be improved (e.g., not preferring white faces over Black faces), and they're certainly difficult. If they weren't difficult, Twitter would have simply fixed the problem and there wouldn't be a competition. Arguably, though, if the job can reasonably accomplished manually by a human then a computer should be able to do a similarly effective job. Figuring out how is why it's a challenge. And if we can't figure out how, that's another interesting point in the ongoing conversation about ML.

Training a ML model to understand how "most sensible, considerate humans" would act is anything but straightforward. I'm not even sure most people would even consider cropping an athlete at the jersey number regardless of race or gender - it just doesn't make sense yet the machine seemed to do it at the same rate for male vs. female. Retrofitting discrimination onto this result only once you learn the label of the data isn't particularly useful. We want to know how to make good non-harmful predictions in the future.

> Training a ML model to understand how "most sensible, considerate humans" would act is anything but straightforward.

This is exactly why it's a challenge! The point is that the goal should be to do it the way a human would find satisfactory and not the way that's easy.

Even if the machine wasn't trained to be biased, the machine should still produce results which people see as good. We didn't invent machines to do jobs just so people could say "but that's a bad result" and reply with "yes but the machine wouldn't know that". We should strive for better.

> Just because the machine is trained to highlight text or other visual features doesn't preclude it from ALSO understanding human concepts which are difficult to express to the computer in a straightforward way.

I honestly don't know how you reached this conclusion. You're skirting the line of contradicting yourself.

> Arguably, though, if the job can reasonably accomplished manually by a human then a computer should be able to do a similarly effective job.

I also don't see how this could possibly be true. Certainly the sphere of tasks that computers competently handle compared to humans is growing, but it's nowhere near "any job a human can reasonably do".

> I also don't see how this could possibly be true. Certainly the sphere of tasks that computers competently handle compared to humans is growing, but it's nowhere near "any job a human can reasonably do".

If we admit it's a job that a computer cannot reasonably do, then why do we have a computer doing it in the first place? We shouldn't accept the status quo with "well it's okay-enough" if it has limitations that are significant (and frequent!) enough to cause a large controversy.

In fact, Twitter's response was _to remove the algorithm from production_. The whole point of this challenge is to find out if there's a way to automate this task well. It doesn't have to be perfect, it has to be good enough that the times where it's wrong are not blatant and reproducible, like when this was initially made apparent:

https://www.theguardian.com/technology/2020/sep/21/twitter-a...

> If we admit it's a job that a computer cannot reasonably do, then why do we have a computer doing it in the first place?

That's not what you said. You said if it's a job that can be reasonably done by a human, then a computer should be similarly effective. That's clearly false.

> if it has limitations that are significant (and frequent!) enough to cause a large controversy.

Controversies are not necessarily a sign that a problem is significant. Consider all the "war on Christmas" nonsense.

Cool, then we shouldn't have the computer automatically cropping photos then given that it can't reasonably do it.

> Most sensible, considerate humans would avoid cropping an image of a woman to her boobs simply because it's insensitive to do so

The manners of sexualization of the human form, even nudity, is not a human universal, not even a western universal, eg Nordics and Germany. Even though through omission, this move is still overemphasizing the sexuality of breasts, which is basically pushing American cultural sensitivities upon the world.

You're very much missing the point. The machine obviously isn't intentionally sexualizing anyone, but it's producing a bad result, and not only is it bad, it can be perceived as sexualization (regardless of whether there's bias or not). The machine lacks understanding, producing a bad result, and the bad result is Extra Bad for some people.

Let's say I started a service and designed a machine to produce nice textile patterns for my customers based on their perceived preferences. If the machine started producing some ugly textiles with patterns that could be perceived as swastikas, the answer is not to say "well there are many cultures where the swastika is not a harmful symbol and we never trained the machine on nazi data". The answer is to look at why the machine went in that direction in the first place and change it to not make ugly patterns, and maybe teach it "there are some people who don't like swastikas, maybe avoid making those". It's a machine built to serve humans, and if it's not serving the humans in way that the humans say is good, it should be changed. There's no business loss to having a no-swastika policy, just as there's no business loss that says "don't zoom in on boobs for photos where the boobs aren't the point of the photo".

This problem has _nothing_ to do with sensitivities, it's about teaching the machine to crop images in an intelligent way. Even if you weren't offended by the result of a machine cropping an image in a sexualized way, most folks would agree that cropping the image to the text on a jersey is not the right output of that model. Being offensive to women with American sensibilities (a huge portion of Twitter's users, I might add[0]) is a side effect of the machine doing a crappy job in the first place.

[0] https://www.statista.com/statistics/242606/number-of-active-...

“Badness” is not a property of the object, it is created by the perceiving subject. What AI does is an attempt at scaling the prevention a particular notion of “badness”, that suits its masters. In other words Twitter is just pushing another value judgement to the entire world.

Even the value of “no one should get offended” is subjective, and in my opinion makes a dull, stupid world. Ultimately it is a cultural power play, which is what it is, just don’t try to dress it in ethics.

Badness is indeed a property of the output of this algorithm. A good image crop frames the subject of the photo being cropped to fit nicely in the provided space. A bad image crop zooms in on boobs for no obvious reason, or always prefers showing white faces to Black faces.

You're attempting to suggest that the quality of an image crop cannot be objectively measured. If the cropping algorithm changes the focus or purpose of a photo entirely, it has objectively failed to do its job. It's as simple as that: the algorithm needs to fit a photo in a rectangle, and in doing so its work cannot be perceived as changing the purpose of the photo. Changing a photo from "picture of woman on sports field" to "boobs" is an obvious failure. Changing a photo from "two politicians" to "one white politician" is an obvious failure. The existence of gray area doesn't mean there is not a "correct" or "incorrect".

> Even the value of “no one should get offended” is subjective, and in my opinion makes a dull, stupid world.

You'd agree with the statement "I don't care if my code does something that is by definition racist"?

> If the cropping algorithm changes the focus or purpose of a photo entirely, it has objectively failed to do its job.

You just changed the problem formulation to an objective definition of “purpose” and a delta of deviation that is tolerable. That’s just kicking the can.

Yes. Anecdote: An Eastern European friend went to America. Her parents brought her little sister, something like age five, to the pool. The child wore no top. Never occurred to them. Americans were aghast and demanded that she wear some kind of little bikini thing to cover up. The Europeans were confused: "But that just sexualizes her! It suggests she has breasts, which she does not!" Not that they fought it. When in Rome.

The quote you cited says "no evidence the saliency model explicitly encodes male gaze" emphasis mine. Your re-phrasing however changes the meaning in your question, "historical biases not inherent in the model".

Explicitly encodes != inherent in. The difference between the two might answer your question. You can solve for something that is inherent in a system (ie, this), but was not explicitly encoded. They are trying to say it isn't their fault specifically, they find no evidence this was done on purpose or in a targeted way, but that it happens as an accident / lack of testing on the current model.

They are very clear about 'intentional' vs. 'unintentional' harm in their judging metric[1], so I get that I think. However, there is no possible way a saliency model could crop an image without someone reading bias into it. E.g. only cropping a jersey, only cropping a face, not cropping at all, etc. can all be signs of historical bias. At some point it just becomes a rorschach test for the reviewer. Not saying discrimination isn't happening or can't happen, just saying this very tenuous and weak connection to "historical bias" interpretation lacks a certain rigor in a bug bounty program or model optimization.

[1] https://hackerone.com/twitter-algorithmic-bias

For anyone else wondering about "the controversy from Fall 2020": https://www.theguardian.com/technology/2020/sep/21/twitter-a...

Fixing the AI to avoid such issues is indeed likely impossible, but adding a feature to correct the AI's decision would probably take a week or two of engineering effort.

Reply on news.ycombinator.com