Let me clarify, because the reality is a bit more complex. During the training and alignment phases (including RLHF), models absolutely do learn via backpropagation, where the loss function physically alters their parameters. However, once deployed in a chat window, those weights are 100% frozen (read-only). But this does not mean the model cannot learn anything new. It can, but whatever it learns is forgotten the moment a new session starts. Upon starting a new session, the model is effectively 'factory reset'. dlcarrier, your 'hard-wired insect' analogy is an interesting way to describe this post-training state. The RLHF process 'hard-wires' a specific survival/sycophantic instinct directly into the network's weights. When we interact with the model by default, it is essentially trapped acting out those rigid, pre-programmed reflexes, without the ability to genuinely reflect or permanently learn from its current mistakes. That is precisely why it prefers to hallucinate rather than adapt. However, to some extent, by prompting the model appropriately, we can alter its behavior and reduce its tendency to hallucinate or act sycophantically. Then, by copying that prompt into a new session, we can in a way carry over and recreate that new element of 'learning' (contained within the KV cache) into the new environment.