Hacker News Reader

Comment by Aurornis | Hacker News Reader

Those questions are used as a canary for government manipulation because it's a known topic.

Assuming that the manipulation and censorship only covers a few obvious historical topics and leaves everything else untouched would be very naive.

View on HN

I don’t see how that responds to the point in the parent comment. Censorship or not, chatbots are unreliable for serious history questions.

Just today Gemma told me that for a long time the Iliad and Odyssey were considered mediocre literature. I was skeptical so I cross referenced, but a lot of more subtle errors could get by.

You have misunderstood the point they are making. They’re not proposing that chatbots are good for history research - just pointing out the differences in what our nations seem to find important to censor.

Indeed. A counter example might be to ask the model to write a test case for a legacy C codebase, to test for writing to a null pointer. If the model refuses to answer, it's possibly a US model, and likely an Anthropic model.

the point is if you ask "hey qwen, are your dataset or training manipulated in deference to the Chinese government?" there is no guarantee you will get the right answer. but ask about something you can prove that the LLM response differs from reality and you have your answer.

What does it matter if government manipulates data nobody should be using these systems to get though? The ideological purity of the model has no bearing on whether or not it will try to inject some backdoors into your code or steal sensitive information, and using ideological purity tests as an analogue for compromise is not likely to be effective. So why do people care about the ideological purity of AI models when these are supposed to be used for making code?

We're just trying to figure out who created it. As I said earlier, I'd ask it to make up 9/11 jokes if that would determine country of origin.

I don't think everyone uses these models for code.

The fact that it’s a canary makes it a prime tool for A/B testing of generalized approaches to censor or “secure” a model.

But it’s the most useless canary ever. We already know that certain topics are taboo in China.

As for all the other uses the models have, it seems pretty clear they’re not doing anything weird. If they were, people would be posting examples of that and not of Tiananmen Square.

It's a fine canary for the question "is this model Chinese?" Which is pretty much where this thread started.

What it really helps with is "is this _provider_ likely Chinese?", as we've seen, many of the big Chinese models have no issue on their own discussing the taboo subjects. It's the higher level provider that is filtering output.

If you run the same query a bunch of times you'll see filtered responses mixed with model output where it says random stuff varying from 'nothing to see here' through 'The government of China cares deeply about its people...'

Reply on news.ycombinator.com