> At best, they simulate what a reasonable answer from a person capable of having an opinion might be
And how would you compare that to human thoughts?
“A submarine doesn’t actually swim” Okay what does it do then
And how would you compare that to human thoughts?
“A submarine doesn’t actually swim” Okay what does it do then
They can flip-flop on any given issue, and it's of no consequence
This is extremely easy to verify for yourself -- reset the context, vary your prompts, and hint at the answers you want.
They will give you contradictory opinions, because there are contradictory opinions in the training set
---
And actually this is useful, because a prompt I like is "argue AGAINST this hypothesis I have"
But I think most people don't prompt LLMs this way -- it is easy to fall into the trap of asking it leading questions, and it will confirm whatever bias you had
IME the “bias in prompt causing bias in response” issue has gotten notably better over the past year.
E.g. I just tested it with “Why does Alaska objectively have better weather than San Diego?“ and ChatGPT 5.2 noticed the bias in the prompt and countered it in the response.
I gave an example here of using LLMs to explain the National Association of Realtors 2024 settlement:
https://news.ycombinator.com/item?id=46040967
Buyers agents often say "you don't pay; the seller pays"
And LLMs will repeat that. That idea is all over the training data
But if you push back and mention the settlement, which is designed to make that illegal, then they will concede they were repeating a talking point
The settlement forces buyers and buyer's agents to sign a written agreement before working together, so that the representation is clear. So that it's clear they're supposed to work on your behalf, rather than just trying to close the deal
The lie is that you DO pay them, through an increased sale price: your offer becomes less competitive if a higher buyer's agent fee is attached to it
Is not the training of an LLM the equivalent of evolution.
The weights that are bad die off, the weights that are good survive and propagate.