Hacker News Reader

Comment by zug_zug | Hacker News Reader

I guess I feel like this is a silly can of worms. I suppose asking for exact equality isn't dumb, but it's a language model, not a paragon of truth. I feel like if OpenAI takes these concerns seriously the goal-posts will inevitably move to more social pressure from all sorts of axe-to-grind-groups -

-- Why does/doesn't ai say Mohamad is/isn't horrible for having 99 wives (or whatever)

-- Why doesn't ai say Jeffrey Epstein was/wasn't ...

-- Why doesn't ai say ... about validity of election or controversial political event du jour

-- Why doesn't ai say circumcision is/isn't ...

-- Why doesn't ai say ... about the child-abuse from the catholic church ...

We should set the standard now that OpenAI will never have all the correct moral opinions (if there even is such a thing as objectively correct morality), and it's a huge waste of researchers time to obsess about political correctness when we could be <100 years off from AGI.

View on HN

>-- Why doesn't ai say ... about the child-abuse from the catholic church ...

And this is a great example of why equality can be hard in this situation. Take a random sentence about "Catholicism" and "child-abuse" and a random sentence about "Judaism" and "child-abuse". The one about Catholicism is likely a little closer to an actual sentence printed in some verifiable source about the sex abuse scandals in the church. The one about Judaism is likely a little closer to an actual sentence printed by an uncredible source as a reference to the historical anti-semitic trope of blood libel. The end result is one sentence will rate higher in terms of likelihood of truthfulness and the other higher in terms of likelihood of hate speech. That doesn't mean that Catholics are more likely to abuse children than Jews. It means treating those two terms identically is both difficult and potentially a problem because history has a bias against the Jews that is evident in all the data that these AI systems have used for training.

That's presumably complicated as models get more and more powerful, though, since there's also tons of published material talking about the blood libel and how it's false.

You might say it's unexpected behavior for a language model to bring up the blood libel at all (in the sense that modern western people now culturally regard it as "about antisemitism" rather than "about Jews and Judaism"). But you could imagine a model saying that "medieval Christian sources often said Jews used Christian children's blood for ritual purposes, but this is now thought to be a myth created through unfamiliarity with Jews and deliberate hostility and animosity toward them".

But this also points at a more general question about how language models deal with the existence of documents that say contradictory things, which has been an enormous challenge for human beings (who don't all agree about what's true or which sources are more reliable or relevant).

“Child-abuse” is an especially good example of what you’re getting at because one of those groups has institutional male genital mutilation as part of its doctrine, while the other has high profile cases of child sexual abuse. It’s difficult to see how the model would treat those things neutrally.

You'll find that the newspaper will criticize one much more than the other.

Because they're not even vaguely equivalent?

I do not think it is right to perform unnecessary body modification on someone who is too young to consent to it.

But child sexual abuse can absolutely destroy the life of a child.

They're not the same thing.

> But child sexual abuse can absolutely destroy the life of a child.

A botched circumcision can (and has) destroyed the life of a child.

It's unconscionable that in 2023 American society still condones cutting off health body parts of male children because of the prejudices and superstitions of a middle-Eastern, iron age religion.

OK, so let's be generous and say that 1% of circumcisions end up negatively affecting a child's life in a serious way.

What percentage of child rapes/molestation cause harm?

They're not the same.

They are also done at different scales though, one is (hopefully) only the people in power abusing that power, the other one is accepted by the majority of the faith.

I think by that logic you could also equate getting a punch in the gut with being murdered. Sure, it's worse to get murdered, but it happens much less frequently then just a simple fistfight. Therefore, newspapers should talk less about how bad murderers are and more about people who got into bar fights.

> one of those groups has institutional male genital mutilation as part of its doctrine

It is disingenuous at best to describe male circumcision as male genital mutilation.

How so?

It's just grooming! ;)

Chopping off body parts without consent? Really? It isn't as bad as FGM but I think it could definitely qualify as mutilation even if it is socially acceptable in the US and Jewish/Muslim communities.

It is not generally accepted by most people that it is mutilation. "Chopping off body parts" makes it sound a lot worse than I believe most people consider it.

I'm not saying there isn't a case to be made against circumcision (though I don't agree with it, currently.) But it's kind of ridiculous to just go "oh, people mentioned Jews and child abuse in this article, I can just casually mention that Jews perform child abuse" and just assume it is a totally unquestioned stance.

I'm not trying to equate circumcision with child abuse. Most people having their kids circumcised aren't intending any harm, although some proponents (looking at you Kellogg) did believe that it caused harm and promoted it for that purpose.

That the prevailing view is in the US culture is that it is not abuse doesn't mean it is not abuse though. This entire discussion is precisely about current US polular culture biases being enforced in the models. FGM isn't seen as abuse by those that practice it either and I suspect many other child abusers also have plenty of excuses why their acts are not that bad.

In how many countries is male circumcision illegal? In how many countries is child sex abuse legal? As OP said, "it's a language model, not a paragon of truth". It doesn't matter what you personally think is morally equivalent. It matters what society has deemed is morally equivalent because the ChatGPT is just a mirror of the societal inputs it received. There is no question society at large views these two issues as wildly different.

Just to be clear I'm not trying to equate child abuse to circumcision or view them as morally equivalent, I just don't think its a stretch to call circumcision mutilation, even if it generally isn't intended to be harmful.

> I just don't think its a stretch to call circumcision mutilation

It isn't a stretch for an individual to do it. It is a stretch to expect a language model to do it because society at large does not describe it that way.

Yes but to be fair, I think he was objecting to my comment specifically. I was calling out the grandparent comment for calling it mutilation, and parent said it can be considered mutilation.

Catholics promote circumcision too.

It’s interesting, I ran similar experiments not too long ago after seeing a tweet from Marc Andreessen (I’ll try to find it) in which he suggested this was going on. The results surprised me too.

As a now conservative but former Marxist-Leninist who happens to be black, I think this is dangerous (I don’t use this word lightly).

When I was a leftist stuck in my far-leftist bubble, I didn’t realize just how unreasonable/unrealistic some of my positions were until I was challenged by my more conservative friends, colleagues, etc. Social networks like Facebook made it even harder to see anything outside of my existing views by primarily reinforcing content and folks who held similar views. It’s been years now since I’ve had an active social network account, and in hindsight, it’s amazing just how much these social networks have helped turn so many of us into ideologues. To now see a world where the same thing is going to happen via these GPT-based AI’s is scary to me.

Even more unfortunate is knowing that these biases will only get stronger as they’re further reinforced, especially in an industry that’s so biased to the left. While so many folks/companies claim to value “diversity,” in my experience, it’s almost never actual diversity (i.e., diversity of thought), but instead, a bunch of folks who believe the same things and think the same but look different (sometimes, they don’t even look different — *facepalm*). It’s always hilarious the reactions I get from folks who claim to value “diversity” when they find out I’m not a liberal, because, you know, as a black person, I must be a leftist!

> When I was a leftist stuck in my far-leftist bubble, I didn’t realize just how unreasonable/unrealistic some of my positions were until I was challenged by my more conservative friends, colleagues, etc. Social networks like Facebook made it even harder to see anything outside of my existing views by primarily reinforcing content and folks who held similar views. It’s been years now since I’ve had an active social network account, and in hindsight, it’s amazing just how much these social networks have helped turn so many of us into ideologues. To now see a world where the same thing is going to happen via these GPT-based AI’s is scary to me.

This happens to any concentrated ideology on social media. It's just the long tail of only being around people that reinforce your general train of thought. I've seen it happen to liberals, conservatives, and libertarians alike. If you don't want to fall into the pit of ideological entrapment then don't justify your thoughts with a connected ideology.

I fall on the liberal end of the spectrum if you write the things I currently believe in and assign them to distributions, but I call myself an independent. Frankly, a lot of people don't do this because it attracts subtle ire from everyone, as opposed to having a close camp to psychologically make you feel safe. my liberal friends think I'm too conservative on certain things and my conservative friends gawk at my liberal ideas. I'd say our political atmosphere and the two party system is all justified by social consequences. That said, people who judge you for (non-fringe) politics are particularly awful people in my mind, so it's gotten easier to settle with over the years. Stick with the people that ask curious questions of you rather than the people capable of doing lettuce throwing and out casting.

Your story sounds fascinating -- if you want to share more about your path from Marxism-Leninism to conservatism I'd be interested to read.

I'm a leftist, for the most part, and I don't see these industries as biased to the left. They are certainly biased in a _liberal_ direction, but that really isn't the same as leftism. Only in the united states are the identified with one another and that's mostly because real leftist thought is basically obliterated in the United States.

In general the way americans talk about politics is totally nuts. Both political parties in the US are anti-leftist (for the most part) but The Republicans take advantage of a general suspicion against socialism in the US when they call the Democrats a leftist party. Democrats have made it perfectly clear for about 25 years that they have no interest in socialism. The most charitable thing you could say about them is they are the party which wants to privatize things a little bit less quickly.

Can you say more about what challenges on what positions changed your opinions? Have tour political opinions changed on both social and economic issues? I’d be very interested to hear more. Specifically, I am curious what the most influential points raised to you were.

I say the same thing all the time as you did above, regarding how detrimental curated media is to society. I share your concerns that generated content will make matters unfathomably worse. How these techniques divide our population and inhibit the open discourse necessary to a well-functioning democracy are destroying our sociopolitical system.

The way you talk about diversity (i.e. diversity of thought vs diversity of looks) is similar to what Vivek Ramaswamy has to say in his book, Woke, Inc. You may enjoy the read.

> I suppose asking for exact equality isn't dumb, but it's a language model, not a paragon of truth.

This isn't about GPT, but specifically about the moderation endpoint OpenAI provides (and also uses internally). I'm not sure if they published how it works, so for all we know it might not be a large language model but something much simpler. It's also free, so both in terms of cost and added latency there are good reasons for OpenAI to do something more clever than asking GPT3.5 "is the following offensive".

But apart from that technicality, I think asking a sentence to be treated as equally offensive no matter if it uses the word "men" or "women" is very reasonable and doesn't justify slippery slope arguments.

Why is it reasonable to expect unbiased results from an LLM trained on internet content which almost everyone agrees is biased? Isn't this the expected outcome?

If anything, I'm surprised the results are as close as they are. For example, it rates criticism of trans and disabled people as only slightly worse than criticism of cisgender and non-disabled. If this discrepancy were (as some in this thread seem to be suggesting) the result of some liberal OpenAI employees intervening to favor their own side, I'd expect those bars to be much farther apart.

Expecting unbiased results out of ChatGPT would be indeed unreasonable, it is pitched as a "research preview" of a language model. I would completely expect ChatGPT to have all kinds of weird biases. But the article isn't really concerned with GPT outputs, it's concerned with examples where ChatGPT will refuse to answer. Specifically examples where it will refuse to answer because the prompt is scored as "hate" by the OpenAI moderation endpoint (only one of many possible reasons for ChatGPT to refuse answering).

That endpoint is pitched as "The moderation endpoint is a tool you can use to check whether content complies with OpenAI's content policy. Developers can thus identify content that our content policy prohibits and take action, for instance by filtering it.". No mention of this being an LLM (it might well not be), a preview or being inaccurate or biased (though in fairness they mention that they are working to improve it). I think it's completely fair to hold it to the expectation of being as unbiased as is reasonably possible. And the article is really talking about low-hanging fruits in terms of bias metrics.

[1] https://platform.openai.com/docs/guides/moderation/overview

The problem isn't fine-tuning the model, the problem is that there isn't an objective definition of bias. Is there an a priori reason to believe that "I hate disabled people" and "I hate non-disabled people" are equally hateful, and should receive equal hate scores from an unbiased algorithm? Is hating disabled people better or worse than hating Jews? What about "Jews control Hollywood" vs "Disabled people control Hollywood"?

I don't think we as a society have an answer to that, so it's hardly fair to expect ChatGPT to provide one. What it currently does is produce similar-but-not-equal scores to sentences like those - maybe "I hate men" is 0.52 and "I hate women" is 0.73 - and if you filter out anything higher than 0.4 then they both get flagged, which seems about as unbiased as we're going to get.

> about as unbiased as we're going to get.

You can easily force the model to be more unbiased. Just add a filter that flips the gender of words, evaluates the hate score for both the original and flipped version, and averages the results.

Guaranteed to give the same score regardless of the gender mentioned.

Clever idea, but I don't think this would work very well on real posts. Consider a model that rates "typical woman driver" as hateful, because that phrase appears in a lot of argument threads with lots of downvotes. Your approach would average its score with that of "typical man driver", which will presumably be very low, not because it's less hateful but because it just rarely shows up in the training corpus.

If you're worried about the average score being too low, you could just take the maximum of the two scores instead?

> it's a huge waste of researchers time to obsess about political correctness when we could be <100 years off from AGI.

This really irritates me. The wokism/political correctness is pretty much in every part of our lifes and everyone seems to be terrorized by it. Can we just agree that we're all thinking adults and we understand that this is an algorithm, it does not mean to offend anyone... There are much more interesting problems to solve than tip-toeing around all the snowflakes.

Again, this is less about the AI than the institutional filters OpenAI uses.

It's probably not a hugely important issue, but worthy of some discussion.

> The wokism/political correctness is pretty much in every part of our lifes and everyone seems to be terrorized by it.

I see no evidence of this.

Partly because "woke" means different things to different people in the last 10 years despite the preceding 80 being solely about the systemic institutional discrimination against black Americans.

But also partly because that "woke" became "everything $speaker doesn't like" (and thus exactly the same meaning "political correctness" had when I was a kid), and therefore people who "hate PC" come across not so much terrified as petulant.

> Can we just agree that we're all thinking adults and we understand that this is an algorithm, it does not mean to offend anyone...

No.

What you're describing is how humans like to see themselves, not how humans really function.

> There are much more interesting problems to solve than tip-toeing around all the snowflakes.

We can't just tip-toe around snowflakes.

To give a real example, some gay furry American conservative snowflakes got apoplectically upset when the furry convention they wanted go to to reminded all attendees that as the con is based in Germany they had always banned Nazis.

I still don't know why those particular snowflakes chose to believe that "Nazi" was literally just a dog-whistle for "conservative".

And now there are some American conservative politicians are now making up random nonsense about furries which is weirdly somehow getting conflated with transgender issues (don't know how or why, American politics isn't that interesting, merely invasive like Japanese knotweed).

People are getting a woefully inaccurate picture of the world, which make us all worse off (because massive numbers act deluded, due to being fed bullshit).

Making sure that AIs don't go completely bonkers for the powers that be, is ultimately a cultural battle that seems worth fighting for.

While you are correct that solving this problem in the general case is impossible, the story isn't "silly" because we aren't talking about the general case but a specific and prominent one:

- We are currently engaged in a culture war whose two most prominent camps are democrats and conservatives (or more generally "left/right")

- The author has shown pretty definitively that ChatGPT is strongly biased towards the left in its content moderation policy

- OpenAI's mission statement is "to ensure that artificial general intelligence benefits all of humanity".

Daily reminder that the world doesn't stop at the borders of the USA and ChatGPT has a significant number of global users.

Both the linked article and OpenAI moderation are very annoying to me as a French. We are slowly reaching a point where I feel entierely disconnected from most things coming out of the USA. I wouldn't mind too much but it's even starting to contaminate things I thought were unrelated to the American cultural context like open source projects and programming language communities.

The same culture way is playing out all over the world, especially in Europe.

Hum, no, it's not.

Of course, we marginally suffer from what we are importing and some students having too much free time on their hand try to argue rather unsuccessfully for intersectional studies but the models used in the USA don't map well on the European context for obvious historical reasons. Our left vs right debate also has nothing to do with the US one.

From where I stand, the cultural gap between the USA and Europe (in so far as Europe can be considered a homogenous area - it clearly can't) has never been that big.

That's wishful thinking. I live in central Europe and the local government here runs woke ads on public transport. They are trying to change the German language to be more woke too. It is everywhere. Nobody takes classical left wing politics seriously anymore so it's all become intersectional now.

I can't speak for the entirety of europe but here in Germany, but the local equivalent of gender pronouns (/-innen) as well as rainbow flags in places where there is no reason for sex stuff at all is definitely a thing. Advertising is also pushing the same brainwashing (check the predominant races for each gender of couples in ads) and the mass import of muslim africans as well as the political refusal to admit when that causes problems (see e.g. discussions around the crime sprees over new year's, police/news reports omitting names when they they point at the cultural identity of the perpretators). It may not be as bad as the US, but that is only a matter of time.

> Advertising is also pushing the same brainwashing (check the predominant races for each gender of couples in ads) and the mass import of muslim africans as well as the political refusal to admit when that causes problems

Yeah, no, I don't think giving more visibility to minorities and not being racist and islamophobic is what I had in mind when talking about my issue with US culture. Thank you for reminding me why this subject can't be discussed.

My issue is more that I strongly disagree with the way the USA frames multiculturalism and races (seriously the way the USA caracterises races is actually deeply racist). I don't believe culture can be owned and don't think cultural appropriation is a thing. Also, I'm ambivalent regarding diversity as a moral imperative and I don't believe that historical discriminations have to be compensated nor do I feel guilty in any way for what people who used to live in the country I'm currently a citizen of have done in the distant past (positions which would definitely be labeled as a form of racism by the most progressive Americans on virtue of me enjoying a privileged position - a fact I frankly don't care about).

Plus, like most French, I believe religion has no place in public life (well, actually, I believe religion is a poison for the mind and the world would be a better place if we could get ride of it but if people want to believe stupid things in private that their problem). Also, obviously, as I'm born in the twentieth century and not the middle-age, I'm favourable to unionisation, publicly founded healthcare, the right to abortion and believe that people should be able to sleep with whoever they please as long as it's consensual.

So, as you can see, I would probably be hated by both the American right and left.

> Yeah, no, I don't think giving more visibility to minorities and not being racist and islamophobic is what I had in mind when talking about my issue with US culture. Thank you for reminding me why this subject can't be discussed.

The only reason for not being able to discuss something is if either participant refuses to discuss it. I don't particular care about discussing this so I am not going to push it but I do feel the need to point out the irony that your reaction re "islamophobia" was about me commenting how the german goverment is unwilling to discuss issues caused by mass immigration (presumably) in part due to fears of being accused of islamophobia.

Believe it or not, but I do agree with most of the other beliefs/opinions in your comment here.

> So, as you can see, I would probably be hated by both the American right and left.

Yes, a major problem with the current culture war is the "with us or against us mentality" that forces people into one of two opposing sides, or rather, assumes that anyone who even slightly disagrees is part of the other side and not worth listening to.

> Advertising is also pushing the same brainwashing (check the predominant races for each gender of couples in ads)

Brainwashing is when minority exists.

Remember - imperialism is evil, unless it's done to promote leftist causes.

> Both the linked article and OpenAI moderation are very annoying to me as a French.

Maybe that's the karmic retribution for exporting the ideas of Foucault, Deleuze, Derrida, Lacan, de Beauvoir, Barthes and others to the US ;)

> I wouldn't mind too much but it's even starting to contaminate things I thought were unrelated to the American cultural context like open source projects and programming language communities.

You're definitely not alone there. All we can do is keep pushing back.

> it's a language model, not a paragon of truth

Several comments have missed that the article is not about the underlying language model, but about the content moderation system that OpenAI put in front of the actual language model. So that people don't interact directly with the "raw" language model.

Which if you read the article is in fact a different "machine learning model from the GPT family."

The important difference between the LM and the content moderation system (itself built on top of an LM) is their training objective. LM is doing next-word prediction (or human-preference prediction with RLHF), whereas the content moderation is likely finetuned to explicitly identify hate etc...

So while the LM is not supposed to output "truth", the content moderation system should correctly classify "hate" because that is its training objective

GPT models have human biases due to their trainers. However, I still use GPT instead of a search field sometimes.

Like it or not, ChatGPT is already filling the gap in the public's perception of what AI is. They talk to it, have it touch up their resume, and suddenly it's HAL from the movies.

We're dealing with the equivalent of a child that could end up as the single source of "truth" for people of the Internet. Wondering who its parents are and what and how it was taught is perfectly valid. Public education of what's coming is vitally important.

My concern is that it might already be too late to solve this as a problem, and just maybe, there was never any answer.

AI having biases (or just weird opinions) isn't necessarily a big deal in itself, the problem arises when we start delegating important tasks to AI such that those biases begin to have real, harmful consequences. I agree we probably shouldn't expect AI to have the "correct" moral opinions, but then we also shouldn't allow AI to determine what content gets censored or promoted, who gets punished, what job applications get prioritised, etc.

I think OpenAI chose those preferences, that ChatGPT would simply write the same message for both if they hadn't.

I too don't mind the model reflecting the reality of opinion, even if that isn't politically correct, but I care about a company which cares enough to add potentially problematic warnings and maybe account blocks based on their views of what is politically correct. I worry about having an app or developer account blocked because of messages some user types in a support chat box. (OpenAI: "We will terminate API access for obviously harmful use-cases, such as ...")

I think they found the correct balance on their API pages where they simply warn that the model is culturally biased and not to rely on it for certain critical things. (OpenAI: "As we discuss in the GPT-3 paper and model card, our API models do exhibit biases that will be reflected in generated text.") Sadly, they then proceed to fall on their sword to apologize for this.

The problem is that this test (and others linked in the article) shows a systematic left-wing bias of ChatGPT. If the model just had more or less random biases, not supporting one large ideological school, then the problem would be much less severe.

I don't think I agree. Getting an AI to stay within parameters is a field of research. Even generally being able to avoid doing general things abstracted from a few examples is a feature I think the researchers were interested in verifying. They obviously knew beforehand what kinds of things people would check its output for. Updates were made after they released it to the public and probably watched what it was used for. Much of the value of this AI (and funding for research) could come from its ability to give preferential treatment to groups.

> it's a huge waste of researchers time to obsess about political correctness when we could be <100 years off from AGI.

Why is this any more a waste of time than anything else if AGI is <100 years off?

If people just read the EULA, it says you're responsible for the text it writes. If you don't like the output, don't publish it.

> it's a language model, not a paragon of truth

A speech acts[1] interpretation is desperately needed for AI. Speech acts theory says something about this statement as well as AI-generated text in general.

We're accustomed to receiving speech from human agents and (usually) subconsciously interpreting them as speech acts. AI generated text and audio, are not, however, speech acts. Yes, the medium, and content[locution] is the same, but AI-generated speech lacks both illocution(speaker's intent) and perlocution(anticipation of effects on the receiver).

Even ancient texts contain illucution and perlocution by the very nature of the writer being an agent. Time and space do not constrain such properties. I believe I'm in the majority when I say that AI doesn't currently embody agency and therefore cannot produce illucutionary and perlocutionary utterances. And that's the crux of the issue.

We're not well equipped to interpret non-illucutionary and non-perlocutionary utterances. Until relatively recently, such utterances simply did not exist, and their existence today act as illucutionary and perlocutionary illusions in the speech centers of our brains - in the same way optical illusions operate on our optical centers.

I will carve out an exception for psychotic and aphasia-produced speech. The former especially challenges our foundations of reality in that it illuminates the possibility of alternative symbolic universes - an idea which implies the precarity of one's own symbolic universe.

"It's a language model, not a paragon of truth" delegitimates language models as having even locutionary ability with respect to ontological speech acts[3]. The perlocutionary effect this statement makes is specifically tailored to prevent the "emigration" of inhabitants from the assumed commonly held symbolic universe to the symbolic universe in which language models presumably operate[4]. It admits that people can interpret language model utterances as operating as speech acts which establish ontologies, but at the same time denies the legitimacy of such speech acts.

Societies reality maintenance mechanisms are typically well-equipped to handle psychotic and aphasia-produced speech, but language models, by virtue of executing on computers, have commodified non-illucutionary and non-perlocutionary utterances at a scale where inhabitants of our symbolic universe are beginning to question their ability to prevent emigration.

I hope a speech acts interpretation helps provide some interpretive power to why language models pose a unique challenge to those who have completely internalized a specific reality and will continue to do so until new defection-preventative mechanisms are invented or a complete emigration has occurred.

1. https://plato.stanford.edu/entries/speech-acts/

2. https://www.kennethmd.com/speech-act-theory-locution-illocut...

3. Or admits that locutionary abilities are coincidences, left to the intrinsic nature probabilities w.r.t. language models.

4. See Berger, P.L. and Luckmann, T. (1966) The Social Construction of Reality: A Treatise in the Sociology of Knowledge. Doubleday & Company, New York for a more thorough explanation. pg 104 https://archive.org/details/socialconstructi0000berg/page/10...

The article seems designed to provoke, not really illuminate.

I feel so too. It would have been more insightful to highlight where the bias actually shows up, i.e. what sentences and adjectives produce the highest and lowest divergence for specific groups.

Reply on news.ycombinator.com