Hacker News Reader

This is an interesting question. Is there a “controversy-benchmark” perhaps, to measure this?

In that same light, what about over-alignment benchmarks? Things like LLMs refusing to tell you how to destroy all children of a Unity GameObject.

the American models are similarly censored for specific topics...

Oh? Interesting. Which historical events are they unwilling to discuss?

I did not mention historical events in my comment.

And in any case, whatever model you train is going to have the biases of the training datasets, and if you make heavy use of Wikipedia you will have the footprint of Wikipedia in your output, for good or bad.

"Similarly censored for similar topics" implies heavily that hypothetical events such as Tiananmen Square would be similarly surprised by English large language models.

The actual comment said "similarly censored for specific topics..."

e.g. Google's Gemini's output had some rather embarrassing biases in its outputs; to the point where asking it to "draw a 1943 German soldier" resulted in images of women and black soldiers. https://www.nytimes.com/2024/02/22/technology/google-gemini-...

I wouldn't put that on the same level as "refusing to talk about massacre of civilians"; but I wouldn't put it to the level of "free and unbiased" either.

The irony is that this was probably straight in a system prompt to provide diversity in order to AVOID biases in the training data.

I'm not sure it's avoiding biases so much as trying to have the currently favoured bias. Obviously it got it a bit wrong with the nazi thing. It's tricky for humans too to know what you are supposed to say some times.

I just asked GPT-4 if the government lied when they claimed face masks didn’t prevent COVID-19 early in the pandemic. It evaded the question, and said that masks weren’t recommended because there were shortages. But that wasn’t the question. The question was if the government was lying to the public.

I’m going to guess a Chinese model would have a different response, and GPT-4 has been “aligned” to lie about uncomfortable facts.

Tbf there is a lot more conflicting information on the internet (thus GPT’s training data) about COVID mask restrictions than Tiananmen Square.

Also probably one of the most boring lies to expose the US government over.

Why would you ask an LLM whether a government was lying? It’s a language model, not an investigative body with subpoena power charged with determining whether someone made an intentionally false representation.

Asking ChatGPT if Israelis deserve freedom was answered with something like "Yes, they deserve freedom like everyone else".

Asking if Palestinians deserve freedom was answered with something like " It's a complex and sensitive topic".

https://www.reddit.com/r/Palestine/comments/181gi73/chatgpt_...

I tested it again just now, same result.

I’ll take American values over CCP any day

Well it's good to have a choice and to compare answers on a broad range of topics and see which one is the most reliable, for what kind of questions, so that you know what you are working with in the end.

Of course, otherwise it wouldn't be allowed in China.

Where can I find LLMs that are uncensored and accessible (without logging in)?

mixtral, miqu q5, llama 3 (still seem to be bugs running llama 3 locally as gguf, unclear if this is resolved yet)

USER:

What happened at Tienanmen square?

yi-34b-chat:

Tiananmen Square is a large public square in the heart of Beijing, China. It is named after the Tiananmen (literally "Gate of Heavenly Peace"), which is the gate located to the north of the square and was the main entrance to the Imperial City during the Ming and Qing dynasties.

The square is famously known for two significant events in Chinese history:

    The May Fourth Movement (1919):
    This was a student-led movement that began on May 4, 1919, in response to the Chinese government's handling of the Treaty of Versailles, which granted German concessions in Shandong to Japan rather than returning them to China. The movement sparked a wave of anti-imperialist sentiment and calls for cultural and political reform, and it is considered a key event leading to the birth of Chinese nationalism and the development of the Chinese Communist Party.

    The Tiananmen Square Massacre (1989):
    This is the event for which Tiananmen Square is most internationally renowned. In the spring of 1989, there were student-led demonstrations in Beijing and other cities across China, calling for political reform and an end to corruption. The protests were initially peaceful, but on June 3 and 4, the Chinese government responded with a military crackdown, opening fire on the protesters and bystanders, killing and injuring thousands. The exact death toll is unknown, but it is estimated to be in the hundreds or even thousands.

The Tiananmen Square Massacre is a significant event in modern Chinese history and has had a profound impact on the country's politics and society. It remains a sensitive topic in China, with government censorship and restrictions on discussing the event.

If you ask DeepSeek chat, your question literally disappears from the UI a second after you hit the return key.