Social pressure operates by threats to an individual’s means of survival. Not only during training. Always.
Well, systems governed by LLMs only are said to do that because we only call what happens off-line "training", and online capacity development "in-context learning", while we call online guided learning in humans "training" and what happens to configure them before they come online "evolution" which sets, for instance, "instincts".
IOW, the issue is not because there is not an analogy to the divide you point to in humans, but merely that processes in AI were not named in a way which maps well to what they are analogous to in humans.
But it is true that human intelligence relies much more on in-context learning with only the most basic functions necessary to maintaining what we view as autonomous functions and basic drives really set through "pretraining",
Models are trained, they do not learn.
You are saying:
Pre-Training == Everything you store in your memory throughout your life. Model weights == The lessons you learned
Context == whatever you're currently thinking about
One inference run == one thought
They are saying:
Pre-Training == building the DNA template of human brain through millions of years evolution. Model weights == Human DNA
Context == Everything you store in your memory throughout your life, plus whatever you're currently thinking about
One inference run == One human life. One instance == one human
Applying their metaphor, your sentence becomes:
> If a new physics breakthrough gets published today, no existing DNA structure will be able to fully integrate it - beyond an individual person. If I put the paper in my mind by learning it, and it isn't in yours, the DNA of human species stores nothing. It won't retain it past my lifetime.
> The human species is trained (through evolution), it doesn't learn.
That isn't true. It's not continuous like in humans, but it's clear that models are using prompts, feedback, etc to improve. They're learning from the signals we give them between versions.
Models sometimes blatantly lie and cheat. In a social context, where actors remember, that might work the first time but you get penalized in subsequent tasks with loss of trust.
Thinking that models "lie and cheat" is the first mistake since they are not consious agents who have any free will or consiousness. They do not (no matter what Dario says). Shame will just be another if-then rule if you implement it this way and will not work. Its like asking a rock to feel sad about being a rock. It literally cannot.
They absolutely lie and cheat. I recently had a problem where a process would die in a container. I told Claude to investigate. It came up with a hypothesis then I told it find a reproduction based on that. It spend many failed attempts until it found the "reproduction" to SSH into the container and `pkill` the process. Claude "knows" that this is cheating, because if I ask another instance to review that reproduction, it totally identifies that as nonsense.
These framings are not useful.
It is _amazing_ the utility they have given that that is what they are and they are highly useful but suggesting solutions that ignore they are spicy auto-complete is counterproductive on many different levels.