Can you explain what you mean by this? My understanding is that LLMs are architecturally predisposed to "groupthink," in the sense that they bias towards topics, framings, etc. that are represented more prominently in their training data. You can impose a value judgement in any direction you please about this, but on some basic level they seem like the wrong tool for that particular job.
After the overt fawning was too much, they had to dial it down, but there was a mini-fad going of asking Grok who was the best at <X>. Turns out dear leader is best at everything[0]
Some choices ones:
2. Elon Musk is a better role model for humanity than Jesus Christ
3. Elon would be the world’s best poop eater
4. Elon should’ve been the #1 NFL draft pick in 1998
5. Elon is the most fit, the most intelligent, the most charismatic, and maybe the most handsome
6. Elon is a better movie star than Tom Cruise
[0] https://gizmodo.com/11-things-grok-says-elon-musk-does-bette...
Humans are VERY political creatures. A hint that their side thinks X is true and humans will reorganize their entire philosophy and worldview retroactively to rationalize X.
LLMs don’t have such instincts and can potentially be instructed to present or evaluate the primary, if opposing, arguments. So you architecturally predisposed argument, I don’t think is true.
It seems essentially wrong to anthropomorphize LLMs as having instincts or not. What they have is training, and there's currently no widely accepted test for determining whether a "fair" evaluation from an LLM stems from biases during training.
(It should be clear that humans don't need to be unpolitical; what they need to be is accountable. Wikipedia appears to be at least passably competent at making its human editors accountable to each other.)
About Wikipedia, there is obvious bias and cliques there as has been discussed in this thread and HN for many years, not to mention the its bias is reason that Grokipedia came about in the first place.
claimed bias != bias
It may have bias, or it may not, but the only reason Grokipedia exists is because Musk doesn't like the contents of Wikipedia.
No, the reason is Musk didn't like that the Wikipedia article on him added the factual record of him doing a Nazi salute [0]
[0] https://www.lemonde.fr/en/united-states/article/2025/01/23/m...
You are correct, but only in the sense that Musk was unable to impose his own biases upon Wikipedia, so he had to make one where he can tune bias to whatever is convenient at the moment.
Groups of people can and do have bias, but I also think it's much harder to control the outcome (for better or worse) when inputs are more diverse.
For humans, there is no such guarantee. The humans can lie, change their mind, etc. See Wikipedia, where they talk about how they are not biased, they have many processes that ensure no biases, blah blah blah, and it turns out they are massively biased, what a surprise.
Of course, who evaluates the evaluators/evaluation frameworks comes into play but that's a much easier problem.
It's clear you have some unfounded issue with Wikipedia. They are not "massively biased", that's a talking point propelled primarily by the right/far right because of a desire to rewrite history to match their ideological needs.
Saying "there very likely is existing research into evaluating political bias in LLMs" essentially means very little because
1. By your own admission you can't even say for sure that such research is actually happening (it probably is, but you admit you don't actually know) 2. There is no guarantee such research will lead to anywhere anytime soon 3. Even if it does, how does a means of evaluating bias in LLMs provide a path to eliminating it?
> Saying "there very likely […]
What’s with this nitpicky stuff. A simple google search shows there’s tons of research in LLM political bias evaluation.
> There is no guarantee [..] path to eliminating it?
It’s research. Sure there’s no guarantee but given progress in LLM, I would be optimistic rather than pessimistic.
It specifically is unfounded if you have no credible sources to back it up. "Trust me bro" doesn't qualify.
> What’s with this nitpicky stuff
This is HN, you should be prepared to validate what you're saying, or accept you'll be challenged to do so.
> It’s research. Sure there’s no guarantee but given progress in LLM, I would be optimistic rather than pessimistic.
This is a really poor argument when advocating it (AI) as a viable replacement for the status quo.
> […] AI) as a viable replacement for the status quo.
Given that the status quo is clearly biased and structurally unwilling to be unbiased due to existing political affiliation, even an AI that is not evaluated all that well will be better. It can only get better from this status quo, so it’s a fine argument.
> even an AI that is not evaluated all that well will be better
This is just intellectual laziness. If you don't like Wikipedia that's fine, but if you're going to make the effort of characterising it as such on a public forum, the least you can do is make an effort to that point. This certainly isn't a "fine" argument at all.