A normal transformer model doesn't have online learning [0] and only "acts" when prompted. So you have this vast trained model that is basically in cold storage and each discussion starts from the same "starting point" from its perspective until you decide to retrain it at a latter point.

Also, for what it's worth, while I see a lot of discussions about the model architectures of language models in the context of "consciousness" I rarely see a discussion about the algorithms used during the inference step, beam search, top-k sampling, nucleus sampling and so on are incredibly "dumb" algorithms compared to the complexity that is hidden in the rest of the model.

https://www.qwak.com/post/online-vs-offline-machine-learning...