Today's locked-down pre-trained models at least have some consistency.
In a decade since then, things got marginally better, and such events wouldn't play out so fast and so intensely in 2026.
Are you saying the internet would not do it again, or Microsoft would not do the same approach? Because I think the internet would absolutely do it again.
Sameness is bad for an LLM like it’s bad for a culture or species. Susceptible to the same tricks / memetic viruses / physical viruses, slow degradation (model collapse) and no improvement. I think we should experiment with different models, then take output from the best to train new ones, then repeat, like natural selection.
And sameness is mediocre. LLMs are boring, and in most tasks only almost as good as humans. Giving them the ability to learn may enable them to be “creative” and perform more tasks beyond humans.
Was it based on a specific scientific paper or research?
The controversy surrounding it seemed to have polluted any search for a technical breakdown or a discussion, or the insights gained from it.
There seems to have been interest in a model which would pick up language and style of its conversations (not actually learning information or looking up facts). If you haven't trained an LSTM model before - you could train on Shakespeare's plays and get out ye olde English in a screenplay format, but from line to line there was no consistency in plot, characters, entrances and exits, etc. in a way which you'd expect after GPT-2. Twitter would be good for keeping a short-form conversation. So I believe Tay and the Watson that appeared on Jeopardy are more from this 'classical NLP' thinking and not proto-LLMs, if that makes sense.
Imagine deploying a software product that changes over time in unknown ways -- could be good changes, could be bad, who knows? This goes beyond even making changes to a live system, it's letting the system react to the stream of data coming in and make changes to itself.
It's much preferable to lock down a model that is working well, release that, and then continue efforts to develop something better behind the scenes. It lets you treat it more like a software product with defined versions, release dates, etc., rather than some evolving organism.
Well it shows that most humans degrades into 4chan eventually. AI just learned from that. :)
If aliens ever arrive here, send an AI to greet them. They will think we are totally deranged.
Ugh HN is so tedious with these remarks. These people are trying to get computers to learn, not just train on data, and HN goes nOt LeArNiNg Is A fEaTuRe. Where's the wonder and the curiosity?
Like when Google wasn't personalized so rank 3 for me is rank 3 for you. I like that predictability.
Obviously ignoring temperature but that is kinda ok with me.
In real life, take programming as an example, we want Claude to be strong in capability at first, but what is more important is for it to learn our code base, be proficient in it, as it gains experience around it. In other words, become a domain expert.
Because our code base is proprietary I don't expect ( not do I want) the AI to be familiar with it on the first day. So learning on the job is the only way to go.
Only in that way it will resemble a human programmer, and only then we can truly talk about replacing human programmer.
Learning is OpenClaw's distinguishing feature. It has an array of plugins that let it talk to various services - but lots of LLM applications have that.
What makes it unique is it's memory architecture. It saves everything it sees and does. Unlike an LLM context its memory never overflows. It can search for relevant bits on request. It's recall is nowhere near as well as the attention heads of an LLM, but apparently good enough to make a difference. Save + Recall == memory.
The first LLMs were utter crap because of that, but once you have just one that's good enough it can be used for dataset filtering and everything gets exponentially better once the data is self consistent enough for there to be non-contradictory patterns to learn that don't ruin the gradient.