Hacker News Reader

Comment by t-3 | Hacker News Reader

Does the Venn diagram of people eager to study history and the people stupid enough to use an unreliable chatbot to study history really have that much overlap?

View on HN

Those questions are used as a canary for government manipulation because it's a known topic.

Assuming that the manipulation and censorship only covers a few obvious historical topics and leaves everything else untouched would be very naive.

I don’t see how that responds to the point in the parent comment. Censorship or not, chatbots are unreliable for serious history questions.

Just today Gemma told me that for a long time the Iliad and Odyssey were considered mediocre literature. I was skeptical so I cross referenced, but a lot of more subtle errors could get by.

You have misunderstood the point they are making. They’re not proposing that chatbots are good for history research - just pointing out the differences in what our nations seem to find important to censor.

Indeed. A counter example might be to ask the model to write a test case for a legacy C codebase, to test for writing to a null pointer. If the model refuses to answer, it's possibly a US model, and likely an Anthropic model.

the point is if you ask "hey qwen, are your dataset or training manipulated in deference to the Chinese government?" there is no guarantee you will get the right answer. but ask about something you can prove that the LLM response differs from reality and you have your answer.

The fact that it’s a canary makes it a prime tool for A/B testing of generalized approaches to censor or “secure” a model.

But it’s the most useless canary ever. We already know that certain topics are taboo in China.

As for all the other uses the models have, it seems pretty clear they’re not doing anything weird. If they were, people would be posting examples of that and not of Tiananmen Square.

It's a fine canary for the question "is this model Chinese?" Which is pretty much where this thread started.

What it really helps with is "is this _provider_ likely Chinese?", as we've seen, many of the big Chinese models have no issue on their own discussing the taboo subjects. It's the higher level provider that is filtering output.

If you run the same query a bunch of times you'll see filtered responses mixed with model output where it says random stuff varying from 'nothing to see here' through 'The government of China cares deeply about its people...'

What does it matter if government manipulates data nobody should be using these systems to get though? The ideological purity of the model has no bearing on whether or not it will try to inject some backdoors into your code or steal sensitive information, and using ideological purity tests as an analogue for compromise is not likely to be effective. So why do people care about the ideological purity of AI models when these are supposed to be used for making code?

I don't think everyone uses these models for code.

We're just trying to figure out who created it. As I said earlier, I'd ask it to make up 9/11 jokes if that would determine country of origin.

Depends where you are in your journey. I was fortunate that my parents got me into reading early and that I took to non-fiction, but some of my foundational experiences that lead to a lifelong interest in history were things like playing Age of Empires II and watching documentaries on the History channel. Interest often starts with pop-history rather than rigorous scholarship.

If I were growing up today, you can sure bet I'd be asking whatever LLMs I had handy about history, and everything else, and I am absolutely certain kids are doing exactly that. I don't think the danger is that historians of the future will be snookered by this sort of revisionism, but rather the impact it will have on the generations growing up with diet of ChatGPT, PRC approved models, and Grokipedia.

I would imagine the answer is yes? There are lots of pop history books out there of questionable veracity

Nowadays unfortunately the answer is yes, there is lots of overlap.

Reply on news.ycombinator.com