Every phenomenon found massively in training set will eventually pop up in LLMs. I just don't find the discoveries made in these papers very meaningful.
Edit: May be I am being too short sighted. The researchers probably start from "Humans are good at X and the training data had many examples of X. How good is LLM at X?" and X happens to be deception this time.
That's a nontrivial and controversial claim that many on HN would hotly disagree with - that LLMs are guaranteed to do everything humans do, if you just feed them enough ordinary data, and so finding behaviors like deception or manipulation is not even worth writing up.
And I think even you would abandon that nonchalant position if the "phenomenon" were sufficiently extreme... ("Sure, the LLM autonomously stalked and hunted down a person that a malicious user badmouthed, and called in a swatting using the Twilio API to get the person killed by the police; but you can find plenty of stuff like that in the Internet training corpus, so I don't understand why anyone is interested. This could have just been a reddit post.")
E.g. this paper would be much more interesting if it measured the threshold at which the LLM starts to become good at X, and linked that threshold to the number and character of training examples of X. Then, maybe, we can begin to think about comparing the LLM to a human.
Alas, it requires access to the training data to do that study, and it requires a vast amount of compute to do it robustly.
I mean for every single claim made in such papers, these claims are hard to reproduce even with the same model for a set of different inputs.
LLMs have no idea they're adopting some behavior. Just numbers being multiplied and token probabilities being calculated based on the distributions extracted from huge sample corpus.
I suspect you're right, but we absolutely do not know this. We have no idea when, why, or how consciousness emerges. We have no reason to think there's something inherently special about the atoms in an animal brain that imbues them with the capability to "produce" consciousness and that the atoms on a silicon wafer are inherently incapable of doing the same thing.
> Just numbers being multiplied and token probabilities being calculated based on the distributions extracted from huge sample corpus.
This is not wildly different from how brains appear to work. Sure we get a gigantic, constant, highly heterogeneous multimodal stream of data inputs from our sensory systems and we get to test theories about our world model via our own actions, but of course, AI companies are all trying to produce analogs for each of these for their models too.
2. We don't know if there's something conscious there to do the knowing
3. Ego doesn't seem necessary either to being conscious or for self-perception
I suspect you're right, but we don't understand nearly enough about consciousness to be confident about any of these questions.
So... Can I worship an LLM and start my own religion? Because we really don't know if consciousness requires X, Y or Z let alone what a a superior all knowing almighty's consciousness consists of.
To me, LLMs look so superior in some ways and maybe they are the gods now that they have manifested themselves as large blobs of floating point numbers when humanity is mature enough to receive them.
We don't know.