Can that not be considered truth-seeking, with the agent-environment boundary being the prompt box?
An LLM is primarily trying to generate content. It’ll throw the best tokens in there but it won’t lose any sleep if they’re suboptimal. It just doesn’t seek. It won’t come back an hour later and say “you know, I was thinking…”
I had one frustrating conversation with ChatGPT where I kept asking it to remove a tie from a picture it generated. It kept saying “done, here’s the picture without the tie”, but the tie was still there. Repeatedly. Or it’ll generate a reference or number that is untrue but looks approximately correct. If you did that you’d be absolutely mortified and you’d never do it again. You’d feel shame and a deep desire to be seen as someone who does it properly. It doesn’t have any such drive. Zero fucks given, training finished months ago.
But yeah, LLMs act more like bullshit artists rather than scrupulous researchers.
Unfortunately it also means it can be easily undone. E.g. just look at Grok in its current lobotomized version
Is the average person a truth seeker in this sense that performs truth-seeking behavior? In my experience we prioritize sharing the same perspectives and getting along well with others a lot more than a critical examination of the world.
In the sense that I just expressed, of figuring out the intention of a user's information query, that really isn't a tuned thing, it's inherent in generative models from possessing a lossy, compressed representation of training data, and it is also truth-seeking practiced by people that want to communicate.
Absolutely
If ChatGPT claims arsenic to be a tasty snack, nothing happens to it.
If I claim the same, and act upon it, I die.
Evolution is much less brutal and efficient. To you death matters a lot more than being trained to avoid a response does to ChatGPT, but from the point of view of the "tasty arsenic" behavior, it's the same.
That being said, I don't ask any controversial or political questions; I use it to search for research papers. But if I try the occasional such question, the response is generally balanced and similar to that of any other LLM.
1. It is still correct that the limited "truth-seeking" that I expressed holds. With respect to the limited world model possessed by the limited training and limited dataset, such a model "seeks to understand" the approximate concept that I am imperfectly expressing that it has data for, and then generate responses based in that.
2. SotA models have access to external data, be it web search or RAG+vector database, etc.. They also have access to the Chain of Thought method. They are trained on datasets that enable them to exploit these tools, and will exploit these tools. The zero-to-hero sequence does not lead you to build such an LLM, and the one that you build has a very limited computational graph. So with respect to more... traditional notions of "truth seeking", these LLMs fundamentally lack the equipment to do that that SotA models have.
Now, you can't conclude that "they clearly don't 'seek' anything" just by the fact that they got an answer wrong. To use the broad notion of "seeking" like you do, a truth seeker with limited knowledge and equipment would arrive confidently at incorrect conclusions based on accurate reasoning. For example, without modern lenses to detect stellar parallax, one would confidently conclude that the stars in the sky are a different thing than the sun (and planets), since one travels across the sky, but the stars are fixed. Plato indeed thought so, and nobody would accuse him of not being a truth-seeker.
If this is what you had in mind, I hope that I have addressed it, otherwise I hope that you can communicate what you mean with an example.
I opened my 'conversation' with a very clearly presented 'problem statement'. Given this datastructure (with code and an example with data) convert it to this datastructure (with code and the same example data transformed) in terraform.
I went through seven rounds of it presenting me either code that was not syntactically correct or produced a totally different datastructure. Every time it apologized for getting it wrong and then coming back with yet another wrong answer.
I stopped having the conversation when my junior who I also presented the problem to came back with a proper answer.
I'm not talking about it trying to prove to me that trump actually won the 2020 election or that vaccines don't cause autism or anything. Just actual 2+2=4 answers. Much like, in another reply to this post, the guy who had it try to find all the states that have w in their name.