None of these LLMs are plastic. They lack neurodiversity. Their thought space and their traversal are likely constrained in someway that humanity's isn't as a collective.
My guess is that we'll just ignore it and make money along the way and every 2-3 months we'll have the equivalent to "Equifax gets hacked and millions of user records are stolen", etc. (this time with the LLM itself doing the hacking at someone's behest - accidental or not).
Some of the linear RNN layers in recent models are provably doing SGD in hidden space during inference
1. Is this learning persistent?
2. Do they verify these new lessons against core principles?
3. Do they and protect themselves/ignore requests if these new lessons contradict those core principles?
Humans do that from the time they're 3 years old (not that well, but they do do it).
So the next step is to ask for evidence and ideally independent and peer reviewed research.
And ICL dates all the way back to 2020, at least: https://arxiv.org/abs/2005.14165