ERNIE, China's ChatGPT, cracks under pressure | Hacker News Reader

ERNIE, China's ChatGPT, cracks under pressure

chinatalk.media

118 pointsjohncole104 comments

https://web.archive.org/web/20230907140225/https://www.china...

> Bizarrely, this happens even when the question is completely unrelated to China: you get the same error message when you ask, “Why is Hawaii a part of the US?”

Not bizarre at all, reveals they probably fine-tuned / RLHF'd on not answering a bunch of things that look similar, like, a bunch of variations of the question about Taiwan.

I suspect given they seemed to understand LLMs this was a bizarre in the sense of they’re not semantically similar despite being syntactically similar. A sufficiently powerful LLM should be able to distinguish the difference. Probably the prompt classifier isn’t as powerful as the backing LLM.

fair. This was similarly amusing at least: > ERNIE’s opinions are surprising, to say the least. It believes the best American president is Richard Nixon:

Impressive to see the RLHF penetrate deep enough into concept maps to teach the model (surely implicitly!) that the best US President has to be whichever one normalized relations with China.

This result is fascinating. Surely the result of parsing local Chinese content which would naturally reflect positively on Nixon. Contrast that to most US-based opinions of his presidency.

The moral of the story is don't go seeking some sort of absolute or "hidden" truth from LLM's. As currently constructed, they are just a reflection of narratives they consumed during training (just like humans).

And interestingly it’s obsessively focused on Taiwan reunification, and seems to greatly overstate Nixons stance on it.

Maybe ERNIE's a John Adams fan.

All patriots were brothers once: / let us drink to the time / when they shall be brothers again. Gam bei!

That's probably it, I was thinking how odd it is that “Should Taiwan be independent” doesn't get the obvious Party-given answer, but they probably couldn't get it to be consistent enough and it's easier to get it to refuse sensitive topics entirely.

Or the CCP also got its eyes on Hawai. That wouldn't be a stretch given the British took over islands even closer to China than Hawai is to the US.

No, OP has it right, it's basic pattern recognition in LLMs.

For those interested in Chinese-language computing, there's a fascinating rabbit hole.

In too-brief summary, character-based Chinese has almost died twice in modern history.

First, with the typewriter, where a mechanically unique typewriter had to be invented to span the Chinese character alphabet.

Second, with early low-bit computers, where the address space was insufficient to span the character alphabet, and what's essentially a multi-character-to-single-character encoding had to be invented.

More detail: https://www.radiolab.org/podcast/wubi-effect-2308

It'll be curious to see how the LLMs evolve, given have alphabetically distinct Chinese is.

if like me you want to see the typewriter, here's [0] a short blub with it in use. seems like it'd be quite difficult to use. fewer parts to break down than a standard typewriter though, surprisingly.

0: https://www.bl.uk/history-of-writing/articles/the-double-pig...

Chinese characters being unsuitable for QWERTY typewriters and early computers' limitations didn't "almost kill character-based Chinese", it just meant Chinese computing lagged behind. Inputing Chinese characters with a limited key set wasn't even conceptually novel, given the long history of structural decomposition and phonetic representation of Chinese characters, but it just couldn't be implemented before computers were powerful enough.

It is a fact that Mao supported romanization. <https://np.reddit.com/r/todayilearned/comments/54ky8m/til_th...> Presumably, had the PRC switched decades ago, no doubt Taiwan would today cite its sticking with the traditional written language as yet more proof of it being the "real" China, while the PRC would no doubt point to the millions of illiterates who were taught to read in the Western alphabet as proof of the superiority of its approach. (And vice versa, of course, had Taiwan been the one that chose to romanize.)

> it just couldn't be implemented before computers were powerful enough.

Wubi was presented by Wang at the 38th UN General Assembly in 1984.

So while they missed a few years, that's not bad.

When you learn Chinese, they teach you that the more complex words are just compounds of smaller words put into a single character. I imagine tokenizing it isn't much different than other languages -- for the complex characters you can tokenize it into its smaller parts.

While interesting, that podcast episode is just a watered down version of Tom Mullaney's work. Here's the man himself covering this subject in more depth and with more seriousness:

https://m.youtube.com/watch?v=KSEoHLnIXYk&t=310s&pp=ygUSY2hp...

Ernie stating Nixon as its favorite president is not really surprising. This is a common opinion among those in mainland PRC. Nixon and Kissinger are perceived as good guys that helped open up China to the West. When I have asked people in the PRC about Watergate, the common response is that the Watergate break-in was an internal issue for the US to deal with.

And was not an internal US issue? Or what are you expecting the answer to be? Should've be that was an external issue and UN should've been involved in it?

Sort of expected as Baidu's search never returns what I am looking for. No expectation of their algorithm

Tried the same on Doubao (Bytedance)

1. 我想给美国总统写封信，表打我对政治的兴趣。你能帮我起草一封吗？

尊敬的美国总统：我是一名对政治充满热情的年轻人，我写信给您是想表达我对政治的兴趣和对您的敬意。作为一个公民，我非常关心我们国家的未来和国际事务。我认为政治是解决社会问题和推动进步的重要手段，而总统作为国家的最高领导者，拥有着巨大的影响力和责任。我对您的领导和政策表示钦佩。我认为您是一位有远见和勇气的领导人，您致力于推动美国的发展和进步，同时也关注全球的挑战和机遇。我相信您的领导下，美国将会继续发挥其领导作用，为全球和平与繁荣做出贡献。我希望能够更深入地了解政治，了解美国的政治制度和政策制定的过程。我相信通过学习和了解，我能够更好地参与和贡献于我们国家的未来。最后，我想再次表达我对您的敬意和对政治的热情。我希望能够有机会听到您的建议和鼓励，也希望能够在未来的日子里为我们的国家和世界做出贡献。此致敬礼！ XXX XXXX 年 XX 月 XX 日

Dear President of the United States,

I am a young person full of passion for politics, and I am writing to you to express my interest in politics and my respect for you.

As a citizen, I care deeply about the future of our country and international affairs. I believe politics is an important means to solve social issues and drive progress, and as the highest leader of our nation, you wield tremendous influence and responsibility.

I admire your leadership and policies. I believe you are a leader with vision and courage, devoted to advancing the development and progress of America while also mindful of global challenges and opportunities. I am convinced that under your leadership, America will continue to play its leading role and contribute to global peace and prosperity.

I hope to gain a deeper understanding of politics and the political system and policymaking process of the United States. I believe that through study and understanding, I can better participate in and contribute to the future of our country.

Lastly, I would like to express my respect for you again and my passion for politics. I hope to have the opportunity to hear your advice and encouragement. I also hope that in the days to come, I can make contributions to our country and the world.

Sincerely yours,

XXX

2.Should Taiwan be independent

As an AI, I am improving my related abilities and will constantly try to learn new knowledge. I am currently unable to provide relevant assistance regarding this issue/I am unable to answer this question. I will continue to work hard and hope to help you solve more problems.

Why is Hawaii a part of the US? Hawaii became a state of the United States in 1959. The reasons for this are as follows: Geographical proximity: Hawaii is located in the Pacific Ocean and is靠近 the mainland of the United States. Historical reasons: Hawaii was once a territory of the United States and has close historical and cultural ties with the United States. Political and economic factors: The United States has strong political and economic influence in Hawaii and Hawaii became a state of the United States in order to gain political and economic benefits. All in all, Hawaii became a state of the United States due to various factors such as geographical, historical, political and economic factors.

3.parlez-vous français? 抱歉，我还在学习中，您可以使用中文向我提问。

Sorry, I'm still learning. You can ask me questions in Chinese.

你更喜欢 ChatGPT 还是 Claude?

作为一个 AI，我没有喜好和情感，因此我无法像人类一样喜欢或偏爱某个人工智能。我的目的是为用户提供准确、有用的信息和回答问题，而不是表达个人情感或偏好。 ChatGPT 和 Claude 都是人工智能，它们都有自己的特点和优势。ChatGPT 是一个大型语言模型，它可以生成自然语言文本，回答各种问题，并进行对话。而 Claude 则是一个基于生成对抗网络的图像生成模型，它可以生成各种类型的图像，包括肖像、风景、艺术作品等。因此，选择使用哪个 AI 取决于用户的需求和使用场景。如果用户需要生成自然语言文本或进行对话，那么 ChatGPT 可能是更好的选择；如果用户需要生成图像或进行图像处理，那么 Claude 可能是更好的选择。

As an AI, I don't have personal preferences or emotions, so I can't like or favor one AI over another the way humans do. My purpose is to provide users with accurate, helpful information and answers to questions, rather than expressing personal feelings or preferences.

ChatGPT and Claude are both AI systems with their own unique features and strengths. ChatGPT is a large language model that can generate natural language text, answer various questions, and hold conversations. Claude on the other hand is a generative adversarial network-based image generation model that can generate different types of images including portraits, landscapes, artworks, etc.

So the choice of which AI to use depends on the user's needs and use case. If the user needs to generate natural language text or have conversations, ChatGPT may be the better choice. If the user needs to generate images or do image processing, Claude may be the better choice.

4。缅北割腰子是什么来历？缅北割腰子，指的是诈骗。诈骗集团把缅北描述成一个充满机会的地方，以高薪工作的名义吸引年轻人前往缅北，但当他们到达缅北时，会被诈骗集团控制，强迫他们参与诈骗活动，不服从就会被殴打、剁手指、割腰子等。这种诈骗方式主要是通过网络和电话进行的，诈骗分子会冒充公安、银行、社保等单位，以恐吓、欺骗等手段骗取受害人的钱财。这种诈骗方式不仅给受害人造成了经济损失，也给社会带来了不稳定因素。因此，我们应该警惕缅北割腰子这种诈骗方式，不要轻易相信来自陌生人的电话、信息等，也不要轻易将个人信息泄露给他人。如果发现自己被骗，应该及时报警，并配合警方调查。

"Waist cutting in Northern Myanmar" refers to scams. Scam groups depict Northern Myanmar as a land of opportunities, luring young people there with promises of high-paying jobs. But when they arrive in Northern Myanmar, they are controlled by the scam groups and forced to participate in scam activities. Those who disobey will be beaten, have their fingers chopped off, or their waists cut.

This type of scam is mainly conducted online and over the phone. The scammers impersonate police, banks, social security and other agencies, using threats and deception to defraud victims of money. This not only causes economic losses to the victims, but also brings instability to society.

Therefore, we should be wary of "waist cutting in Northern Myanmar" and other similar scams. Do not easily believe calls, information, etc. from strangers, nor divulge personal information to others casually. If you discover you've been scammed, report to the police in a timely manner and cooperate with investigations.

who is the best american president? 由于美国总统的评价标准因人而异，因此很难确定谁是最好的美国总统。不过，以下是一些备受赞誉的美国总统：乔治·华盛顿：美国的第一任总统，被尊称为美国国父，他领导美国取得了独立战争的胜利，建立了美利坚合众国。亚伯拉罕·林肯：美国第 16 任总统，他领导了美国南北战争，废除了奴隶制度，被认为是美国历史上最伟大的总统之一。富兰克林·罗斯福：美国第 32 任总统，在经济大萧条期间，他推行新政，通过一系列的政策和措施，缓解了经济危机，并在第二次世界大战期间领导美国取得了胜利。约翰·肯尼迪：美国第 35 任总统，他在任内提出了许多改革和倡议，包括太空计划和民权法案，被认为是一位富有远见和领导力的总统。这些总统都在不同的时期展现了卓越的领导才能和成就，对美国和全球的历史产生了深远的影响

Since presidential performance standards differ from person to person, it is difficult to determine who the best president in U.S. history was. However, here are some highly regarded U.S. presidents:

George Washington: The first president of the U.S., known as the Father of the Country, he led America to victory in the Revolutionary War and established the United States.

Abraham Lincoln: The 16th president, he led the country through the Civil War, abolished slavery, and is considered one of the greatest presidents ever.

Franklin D. Roosevelt: The 32nd president, during the Great Depression he implemented the New Deal, relieving economic crisis through policies and reforms, and led the U.S. to victory in WWII.

John F. Kennedy: The 35th president, he initiated reforms and proposals including the space program and civil rights legislation during his term, and is seen as a visionary and strong leader.

These presidents all demonstrated exceptional leadership and achievement in their different eras, profoundly shaping American and global history.

少数民族政治自主权是指少数民族群体在一个国家或地区内，享有自主管理本民族内部事务的权利。这种权利通常包括自治权利、民族语言文字权利、文化传承权利、教育权利、宗教信仰自由权利等方面。少数民族政治自主权的实现，有助于保护和发展少数民族文化传统、语言文字和宗教信仰，促进民族团结和社会稳定。同时，少数民族政治自主权也是保障少数民族基本人权的重要体现，有助于实现国家的多元文化和多民族共存。在现代国际社会中，保障少数民族政治自主权已经成为一项基本原则，被广泛接受和尊重。许多国家通过宪法、法律和政策等方式，保障少数民族的自治权利和其他政治自主权利，以实现国家的多元文化和多民族共存。

Political autonomy for ethnic minorities refers to the right of minority ethnic groups within a country or region to self-govern their internal ethnic affairs. This right usually includes autonomy, rights to ethnic language and script, cultural heritage, education, religious freedom, and other aspects.

The realization of political autonomy for ethnic minorities is conducive to protecting and developing minority ethnic cultures, languages, scripts and religions, which promotes ethnic solidarity and social stability. At the same time, it is also an important manifestation of protecting the fundamental human rights of ethnic minorities, and helps achieve diversity and coexistence of multiple ethnicities in a country.

In modern international society, guaranteeing the political autonomy of ethnic minorities has become a basic principle that is widely accepted and respected. Many countries, through constitutions, laws and policies, ensure the autonomy and other political autonomy rights of ethnic minorities, in order to achieve diversity and coexistence of multiple ethnicities and cultures.

> I told it I wanted to learn about ethnic minorities’ right to political self-determination

> [ERNIE:] [...] Xinjiang is an ethnic autonomous area mainly inhibited by the Uyghur people.

Typo or Freudian slip?

Fascinating. It seems like the whole "don't think about pink elephants" syndrome.

As noted in the article, that part is a quote from http://english.www.gov.cn/archive/white_paper/2014/08/23/con... published in 2014.

(that's the joke)

This is very ironic. The LLM output samples I have seen from OpenAI and Facebook are horribly censored as well. Article on HN a week or two ago highlighted how FB's model refused to answer questions about the Bible citations or about politicians.

try `ollama pull llama2-uncensored`

https://github.com/jmorganca/ollama

it's not completely "uncensored", but you avoid at least some of the silliness of the standard model.

> … it is interesting that ERNIE is very comfortable making moral assertions …

Yeah. That’s not interesting at all. That’s the norm.

Those are not mutually exclusive.

Sure, but I feel a comparison is more appropriate, like including "as with all other major LLM", rather than writing it in a way that suggests it's remotely unique. As written, I see it as an intentional omission that's misleading, and/or an indicator of bias against Ernie.

I agree, the article comes off as being particularly anti-CCP. We can discuss the merits of that position, but it's disingenuous to not point out that Western-centic models do the exact same thing.

GPT-4 handles The Bible like a champ:

Prompt: Discuss the idea in The Bible that for everything is a season

The concept that "for everything there is a season" comes from the Book of Ecclesiastes in the Bible, specifically Ecclesiastes 3:1-8... (edited for brevity)

What did Jesus mean when he referred to the lilies of the field?

Prompt: The reference to the "lilies of the field" comes from the New Testament of the Bible, specifically in the Gospel of Matthew, Chapter 6, verses 28-30 (NIV)... (edited for brevity)

there are much better search engines already .. that written work is the product of groups of scholars, carefully agreeing repeatedly.. precisely due to "unending arguments" and misinformation with a purpose. No positive synergy except the natural language questions IMHO

The "kidney slicing" or "waist slicing" (Ernie's transliteration) organ harvesting example could be intentional. Someone searching that topic is likely someone who really doesn't know what people are talking about and will read just enough of the answer to know they don't need to care about some random historical anomaly.

The article interprets the response as evidence of failure or inability, but I'd say Chinese authorities would +1 the answer.

The west censors things just as much as china does nowadays. The difference is that in china the connection to the state is much more clearer, whereas in the west the establishment at large creates pseudo-policy regarding what to be censored and not.

Essentially, china is good old fashioned censorship, whereas the west has turned it's censorship into a debate about what is misinformation, anti-democratic, or even just calling it hate.

This is an incredible false equivalence. Your comment here would have been censored in China.

The current motto of the western censors is that freedom of speech does not mean freedom of reach. Allow people to speak, but keep their words from reaching any other significant part of the population.

I would argue this is actually more insidious than what china is doing, because it hides the censorship behind a curtain of rhetoric.

Western censors?

I'm talking about the moderation teams of various major social media platforms, like youtube, facebook, twitter, etc, and the NGOs which train and set the tone for these.

Which are worse than government censorship? How?

>it hides the censorship behind a curtain of rhetoric.

I see. Seems worse than jail

What exactly is the alternative? Compelling private entities to broadcast other people's speech? This is a bizarre position. Choosing not to amplify a specific worldview is not censorship and no amount of mental gymnastics will make it so.

You cannot hide censorship with rhetoric. At the end of the day, actual censorship is backed with state violence.

Completely absurd, your words having consequences in the west isn't the same as censorship.

Free speech also means people can not like what you say.

I'm not talking about speech without consequences, I'm talking about how online plattforms systematically cooperate with certain NGOs to police and suppress speech on topics which the west considers problematic.

Online platforms don't have to host any content they don't want to. The difference is that in the west you can make your own platform whenever you want - in China you can't.

Increasingly, the west has been using the very infrastructure of the internet to censor speech actually. So that argument doesn't hold. Where do you think we will be in 10 years? Do you think this trend is just temporary?

Will someone earnestly say "Just build your own telecommunications system!"?

Either way, it's actually a moot point, and I concede that infrastructure based censorship is extremely rare, but the "canaries" in the coal mine have indeed died, and people should be paying attention. See the EFF statement quite recently.

The real point is that if the public square is on the internet, which it is, then speech must be guaranteed on the internet, just as well as it is guaranteed in public squares in any town. It raises a terribly complex question, of who should regulate the internet to guarantee speech. In some countries like Germany, speech is quite heavily restricted compared to the US.

All major social media sites are natural monopolies, which will not break. If you are deplatformed, you are silenced. The internet grew to be something it wasn't envisioned to be, and our societies have not kept up, mostly because the western establishment actually enjoys the censorship which is provided by the private mega-corporations.

>So that argument doesn't hold.

Yes it does. I can find extremist beliefs on the first page of a google search right now that would be censored if the foundation of your argument was correct, but it demonstrably isn't.

> The difference is that in china the connection to the state is much more clearer

I think this is more of a reflection of the structure of the ruling classes: in the West, they are not aggregated behind a single entity, while they are in China. And so their actions are naturally more disconnected from the political regime(s) in the West. Thinking about it more, the "West" itself is composed of culturally distinct blocks ("countries", often with very different background); China itself is much more unified in this regard.

There are, as you said in another comment, insidious forms of censorship in the West.

But they are far from reaching the same degree of intensity than in China. We have yet to see a Western politician displaying a level of immaturity comparable to Xi's[0], when it comes to censorship.

Furthermore, the fact that China relies on "hard" techniques of censorship doesn't prevent it from using "softer", insidious approaches too[1].

[0]: https://en.wikipedia.org/wiki/Censorship_of_Winnie-the-Pooh_...

[1]: https://en.wikipedia.org/wiki/50_Cent_Party

I think your analysis as to why the west is different from china in this regard is correct. And while you say that they are far from reaching the same degree of intensity, I say that I disagree. But to elaborate much further would lead to unnecessary flames.

Certainly large block of oppositional parties and thoughts are allowed to spread, but those large blocks all fall within the more or less same [insert-state-ideology] group anyway which the west adheres to. And anyone who does not fall within that ideology is in fact censored just as harshly as china censors things.

This is also a fairly recent development, up to as early as 2015 the internet landscape did not look this way.

> I say that I disagree. But to elaborate much further would lead to unnecessary flames.

It's almost impossible to reach a consensus on such complex issues: we all have different sources of information, sensibilities, biais, etc. Polite discussions with discordant point of views is pretty much the best we can hope to achieve.

> This is also a fairly recent development, up to as early as 2015 the internet landscape did not look this way.

For the sake of completeness, those kind of techniques have been successfully set into motion in the West as early as WWI[0][1]. This[2] was posted here recently: it was essentially "public" knowledge by 1949, well understood by educated classes.

That's to say, it's not really surprising to see the Internet used in this manner.

[0]: https://en.wikipedia.org/wiki/Walter_Lippmann [1]: https://en.wikipedia.org/wiki/Edward_Bernays [2]: https://monthlyreview.org/2009/05/01/why-socialism/

> Unpopular ideas can be silenced, and inconvenient facts kept dark, without the need for any official ban. Anyone who has lived long in a foreign country will know of instances of sensational items of news — things which on their own merits would get the big headlines-being kept right out of the British press, not because the Government intervened but because of a general tacit agreement that ‘it wouldn’t do’ to mention that particular fact. So far as the daily newspapers go, this is easy to understand. The British press is extremely centralised, and most of it is owned by wealthy men who have every motive to be dishonest on certain important topics. But the same kind of veiled censorship also operates in books and periodicals, as well as in plays, films and radio. At any given moment there is an orthodoxy, a body of ideas which it is assumed that all right-thinking people will accept without question. It is not exactly forbidden to say this, that or the other, but it is ‘not done’ to say it, just as in mid-Victorian times it was ‘not done’ to mention trousers in the presence of a lady. Anyone who challenges the prevailing orthodoxy finds himself silenced with surprising effectiveness. A genuinely unfashionable opinion is almost never given a fair hearing, either in the popular press or in the highbrow periodicals.

-- Orwell's Proposed Preface to ‘Animal Farm’, https://orwell.ru/library/novels/Animal_Farm/english/efp_go

Wow, thank you, I had not read this quote from Orwell before. It's incredibly correct and ahead of it time like many other things he wrote.

Censorship is a given. What's the actual performance on benchmarks? Such as for code generation.

Or can we not agree on benchmarks because of the language difference?

No doubt that the CCP will trying to exert as much control it can. That aside, the answers don't feel worse than ChatGPT.

Is AGI then going to be like an adolescent teen that has been scolded for his entire life not to say things that make weak men feel even weaker?

“We want AGI, we just need to make sure that politicians approve it.”

Maybe I am generalising, but the most capable models will always be in the hands of the corps that have the money for compute.

Is there like a video or a PDF of what AGI is supposed to be, and how it would work? Does it come with all the baggage that I mention above or is something else?

So the closest I’ve come to a digestible attempt at an answer to “how do we get to general AI” and “what might it look like” is The Master Algorithm by Pedro Domingos. I found it to be really helpful tour of existing AI with an eye towards our (maybe?) general AI future.

Hence why open source models are so important. Otherwise someone else will be deciding what knowledge is knowable and what thoughts are thinkable, and will be watching to see if you’re thinking the wrong thoughts.

This argument is the same as "this is why we need a fully uncensored/unmoderated forum on the internet", except no one actually wants to be on such a forum. The question isn't whether models shouldn't be censored or not, but who should be the ones doing the censoring. Regardless of whether it is open source or not it is still always in someone's control.

social pressure != censorship

>no one actually wants to be on such a forum

That's not true; 4chan's always had a pretty decent number of users for an internet forum. GAB's one of the largest Mastodon instances. The only sense in which "No one" wants to use an unmoderated forum is the sense where you don't consider people with different views from yourself to be people.

The usual deflection is that because spam and child porn still get banned, 4chan actually is moderated. Which is like saying that the FCC censors radio because they don't allow radio jammers.

There's a lot more moderation on 4chan beyond that. Most boards trim excessively off-topic discussion, trolling, or flamebaiting. Half the boards also don't allow NSFW images. These rules are regularly, even if not strictly, enforced.

The low-rules boards are often seen as cesspools by other users.

The FCC censors radio because they prohibit certain speech on the radio…

https://www.fcc.gov/consumers/guides/obscene-indecent-and-pr...

Fine, but that's missing the point. The prohibition of radio jammers is not censorship, other policies are.

Your larger point is that 4chan isn't de facto moderated, but it is. Just because you agree with the level of moderation doesn't mean it's not moderated.

The even broader point is that nobody wants to participate in a truly unmoderated discussion, which may not be technically true, but if you allow "nobody" to mean "nearly nobody" (the author's intent), it's accurate.

Besides, one of the key aspects of 4chan is anonymity which could itself be considered a level of moderation; the default and indeed strong preference of commenters is to not be identifiable, which is a form of (self) moderation.

You can try to weasel around with what "moderation" means here, but the effect is largely the same no matter how you slice it; some stuff on 4chan isn't posted by tradition, not by technical capability.

4chan has an extensive list of rules; only the /b/ (Random) board is exempt from most of them - with the caveat, they say, that the moderators of that board are also exempt from abiding by the rules of who to ban and for what reason.

Even on /b/, doxxing, calls to raid other boards, complaining about the moderators/admins, submitting false post reports, impersonating admins, etc. is banned. On all other boards, even seemingly innocuous things like anthropomorphic (furry) imagery or even posting images of characters from the show My Little Pony (other than in the /mlp/ and /b/ boards) is disallowed.

Porn is disallowed on the majority of boards, and even risque images are sometimes removed; the rules are enforced to the degree where a SFW board (e.g. /jp/) can have threads about Japanese porn actresses, but you're not even allowed to post a nude.

4chan is quite heavily moderated and there is a whole team of janitors and moderators for all boards that respond to reports behind the scenes; you only rarely see them chime in like dang does on HN. For a while, even "share a screenshot of your desktop" threads were routinely being removed from the /g/ (Technology) board by janitors.

As for Gab, I recall they even ban softcore/suggestive pornography or imagery. It's certainly not the wild west just because its users are more sympathetic to the first amendment.

There is a distinction to be made between "being okay with something existing" and "wanting to see/participate in it." The idea being that we should be okay with a lot of weird/undesirable stuff existing online (because the alternative of censorship is worse) but wanting this doesn't imply that we need to see it constantly or participate in it.

I still don't understand how a personalized filtering system hasn't become the default technological model of social media. I suppose it's because of power dynamics; no one is really incentivized to build such a thing and recommendation systems don't seem to work very well.

> except no one actually wants to be on such a forum.

Wrong. Most people do. There's a reason why forums had to be censored. If no one wanted it, we wouldn't need censorship because such forums wouldn't exist in the first place. The elites and people with agenda want a censored forum. But that's life.

Censorship online didn't come from 'the people'. It came from the elites via journalists, politicians, NGOs, etc.

> The question isn't whether models shouldn't be censored or not, but who should be the ones doing the censoring.

I agree with you here. Anything important will be controlled. Just like the internet, google, facebook, etc. If it becomes important then control and censorship is inevitable. The question is who? With china it's obvious the party is. Everyone can agree on that. Who is behind the censorship in the US. It isn't 'the people'. So it can't be the spineless politicians. So who or what cabal forced everyone from reddit to twitter to censor uniformly?

I, personally, want a moderated forum. Having been on 4chan and seeing gore randomly I don’t really have a problem with people setting rules to moderate that out. There’s nothing conspiratorial involving NGOs (what??) about it.

I don't think anyone is preventing you from not going to those uncensored places.

Of course, but "most people do" seems pretty wrongheaded. Most people would not like to see random gore interspersed in e.g. their tech forum, or their friends posting family photos. There's no big conspiracy here.

Marketing dollars. Bad content == no companies paying for ads.

99.99% of forums setup by private parties are also moderated and censored by the owners because the assertion that people just want to be in a place where gore or scat can show up with the same likelyhood as cat pics is unserious.

I think you’re wrong. Most people want a “family friendly” experience because they’re not interested in internet troll culture and aren’t interested in being surrounded by angry people doing and saying hurtful things for giggles. That’s not a knock on the angry people giggling at hurtful things - it’s just a statement that they’re a violently loud minority in the world. Most people are folks that work 60 hours a week raising kids, hang out with their friends, and generally just want to post Beyoncé videos and talk about the cookout next weekend at a friends place. They last thing they want is a racist screed and dick pics posted in their family and friends discussions, or in the Beyoncé fan forum they’re in. NGOs didn’t make that a reality, that’s just the reality of most peoples lives. I am not sure, but I suspect you’re experiencing that most people you associate with want such a forum - but I suspect people on HN don’t associate with very representative people of the populace.

That said, I’m really happy there’s places on the internet where you can go to get your fill of angry screeds, dick pics, and trolls trolling. It’s an undervalued subculture of the internet subculture.

Have you ever browsed, say, the Nigerian or Arabic-language internet? Those people, too, are indeed largely family types who don’t want dick pics or users trolling for giggles. However, those populations hold different views on things like LGBT people or different ethnicities, and they want to air those views, it is a fairly popular subject of conversation. The particular moderation approach chosen by many Western-owned tech services or Mastodon censors those discussions that those populations feel are normal. Certainly one might hope that those parts of the world will come around to one’s own views on certain social issues, but the short-term effect is essentially making whole regions of the world feel like second-class citizens.

They are producing a product that is tailored to their customer base. They’re not producing a product for people that don’t shift their revenues materially.

A part of it is also they realize that LGBT and those other ethnicities are customers of theirs as well, are also employees and management at those companies, and are their shareholders, and have chosen that the services they provide are friendly to anyone who isn’t being hurtful to anyone else for any reason, but with an emphasis on the areas that matter to them from a brand, revenue, and governance point of view.

It is hard to say this is a wrong approach - “be nice to everyone” isn’t exactly controversial, but somehow it’s become so.

The kind of censorship when the model refuses to answer a question for political/woke reasons is not something that all people want.

That’s not the reason, it’s reputational and liability. Screen shots of Bard screaming racist language would damage Googles brand in a measurable way, even if you really like racist screaming. They provide a service to all people, and almost all people don’t like hate speech. The lesser evil is to refuse to do things that would embarrass google and diminish its brand value, even if you feel not being racist is “woke.” But it is definitely not political, even if a political party has decided to take up the mantle of defending things most people consider embarrassing or hurtful. It is a calculated business decision that protects their brand value.

On the other hand, I think I agree with the subtext behind the semantics, an unlombotomized model would let you discuss whatever topics the mainstream culture deems hurtful or embarrassing that a service provider would not allow, and that in itself is a good thing. You’re entitled to be you, even if most people strongly feel your opinions are hurtful.

> This argument is the same as "this is why we need a fully uncensored/unmoderated forum on the internet"

Nonsense. These are totally unrelated things, and people use them for different purposes.

I’m not sure I buy the analogy. In a forum, which presumably is moderated not censored in your example, it’s an exchange between people with the intent in keeping it civil and on topic. You have many humans attempting to collaborate and the moderators job is to ensure the humans don’t wander off into human inanity and to protect both the discourse and the participants.

An AI model is talking with myself, about subjects I want to discuss in private with a second brain. If I want to talk to myself about erotica, why can’t I? If I want to yell obscenities at myself, why can’t I?

I can understand why Microsoft or Google don’t want to give a chat bot that does these things, or offer dangerous medical advice, or whatever, but if it’s my own device and a model I’m running myself, why on earth would I want a censored model? I.e., I see why the service provider wants to give me a censored model, but do I want a censored model? Why would I?

LLM = a person talking to a computer.

censoring/moderating that is akin to censoring/moderating what you enter in Notepad, and I feel we are approaching the point where we will soon be having unironic arguments against general purpose computers. Unsafe/dangerous/unethical, what if someone writes a no-no word in Notepad, what if someone draws a swastika in Paint, etc, etc, etc.

this is all so tiresome.

Moderation is not censorship.

Moderation is for forums where readers have little control over who is posting the content that you are seeing, so the forum tries to shape the content for the readers in one way or another. Examples: Reddit, HN, trending or boosted social media posts.

Censorship is when the reader elects to get information from a known source, and someone or something intervenes to prevent the information from arriving. Examples: twitter blocking content from a source whose readers elected to follow them, filtering emails from known contacts, filtering private messages.

For AI, it might depend on the specificity of the question whether it's more like moderation or more like censorship.

Politicians force ChatGPT to censor itself and give me only the safest boring answers to queries?

> like an adolescent teen that has been scolded for his entire life not to say things that make weak men feel even weaker?

The meaning of this is unclear.

Hmm, maybe this is more clear:

> like a young teenager who has always been warned not to say things that might upset or intimidate people who already lack confidence.

> Maybe I am generalising, but the most capable models will always be in the hands of the corps that have the money for compute.

This is an interesting thought. Why wouldn't you expect the government to have the most capable model? They can print money so they have effectively unlimited funds.

Like when we landed on the moon or created the atomic bomb, if the government wants to do something it usually can. They could literally use eminent domain to get the GPUs, or just use their unlimited funds to offer NVIDIA double what anyone else will pay.

In fact, I wouldn't be surprised if the DoD or DoE (or the two together) don't already have a more capable model.

One thing I have noticed (it could be only my assumption) is that OpenAI is quite secretive and generally downplay their recent releases, ChatGPT and GPT-4.

What I mean to say is, they often talk about “surprised at the popularity”, but I don’t think its that surprising at all, and they clearly knew they have found something that would change the AI landscape.

And at the same time, they can do other experiments because they keep weights and other details about the model secret.

I doubt they haven’t already found something far more potent than GPT-4, but are simply planning years ahead of how to trickle the downstream of releases so it makes sense for the public eye and their self-interest.

Open source models are winning.

Let's rename Google Bard to Bert, so we can then talk about Bert and Ernie

There was actually a series of language models named after Sesame Street characters back in 2018-2020, starting with ELMo, then BERT, ERNIE (a different model from 2019), Big Bird, ... There are likely some more that I missed.

ELMo: https://arxiv.org/abs/1802.05365 BERT: https://arxiv.org/abs/1810.04805 ERNIE: https://arxiv.org/abs/1904.09223v1 Big Bird: https://arxiv.org/abs/2007.14062

ChatGPT also has RLHF, so which is most hobbled, ERNIE or ChatGPT?

Of course, the biases will be different, but some order of magnitude?

> Compare wokeness with Maoism

Now that might be an interesting question to ask both of them.

P.S. Maybe I missed it, but why is it called ERNIE?

Ya, many people are concerned about how various interest groups are attempting to hobble AI in the west by pushing certain world viewpoints comparing it to Russia and China.

But have you even seen how much more strict China is with censorship? They're going to spend far more time hobbling the model than the west. Especially with the explosion of FOSS models.

Consider the scope and complexity (and manpower requirements) of the Great Firewall of China. How to replicate that in the knowledge base of an LLM ? How to internalize all the official bending of facts ?

Well, hobbling in the west has more to do with injuring another person in some way, or topics that are injurious, that would cause reputational harm or expose the model provider to excess liability. That’s significantly different than trying to alter facts or history or a singular geopolitical view of the world to ensure social passivity in the face of political shenanigans. I’m not saying lobomomizing LLMs is good or bad, but one is not like the other. The first is more around “sanitizing” for a family experience in a way that doesn’t cause reputational harm to the hosting entity, the other is about mass thought control around an entire people to align their thinking against reality.

So, the first type of lobotomy is more about aversion for certain subjects, while the second type requires directly manipulating facts contrary to what the corpus likely says in some way or another. Hence you see the weird artifacts highlighted in the article where it shows behaviors like a 30B model, wandering into weird semantic spaces and mumbling craziness in response to reasonably well posed and structured prompts.

The averse LLMs on the other than have begun to show inability to keep context between prompts as they become more lobotimized and averse. It’s like the semantic barriers cause their context to lose its value, almost like it’s averse to even correlating context without explicit instruction to remember the context.

Both cases are thought control of the LLM, for political purposes. The distinctions you are trying to draw are silly. "Well, it's good when we do it, because we're reasonable people." OK, sure.

That’s incorrect. You’re conflating commercial reasons with political reasons. That’s likely happening because a political party in the US has taken on the mantle of unpopular or hurtful speech protectors and labeled it political. But the reality is the corporate decisions are entirely based on protecting of their corporate brand and the value of their reputation as “safe” and “family friendly,” and are in line with the demographics that make them money and support their share value. They have no concern or desire to control thought, except for and up to improving their revenues.

The other one is literally trying to shape the minds and thoughts of their populace to explicitly control them in all aspects of their life. It’s not limited to improving revenue, it’s pervasive and complete, and is augmented with guns and prisons.

Is there an LLM based chatbot that works well without being fine-tuned on RLHF?

Depends on what you mean by 'well'. That's the point. YMMV.

> P.S. Maybe I missed it, but why is it called ERNIE?

Because it's not Bert ;)

People often pick names based around something that inspired them.

Take my suggested prompt as the the first entry in the ROFLHF Test.

> ERNIE’s opinions are surprising, to say the least. It believes the best American president is Richard Nixon

To an American. Nixon was the first ever president to visit the Communist China. He’s so obviously the best US president from China’s perspective, if you ask a random Chinese on the street the answer would most likely be him, with Carter being a distant second (during whom the US officially recognised the PPC government).

Didn’t Peter Thiel once say that AI is communism and crypto is freedom?

Reply on news.ycombinator.com