When I infer I also train.
They're turn based intelligence in a real time world.
1. Each time someone talks to me they don't have to repeat the entire conversation from the beginning with each reply.
2. If my boss/partner/whoever gives me some mandates/orders (basically), I don't just forget about them because they were at the beginning of the conversation.
3. If during the conversation I access external data sources to get new info or refresh stale info (a presentation, a book, whatever), I don't instantly forget about it after the conversation ends and forget to incorporate this information if 10 000 other people ask me again.
4. I verify new inputs/lessons against my core principles.
5. I protect myself/ignore requests if new inputs/lessons contradict my core principles.
6. Etc, etc.
Your context window is 80 years. You are forgetting plenty before you reach the end of it.
But if my partner tells me they're allergic to shellfish, I'm not going to order oysters for them tonight.
See the difference?
I’m pretty sure that’s why so many in-the-know people have been saying we have achieved AGI already. Not just sama’s contract-breaking tactics of late. I’m referring to all the really intelligent folks who have been crying doomsday scenarios for modern society for the last couple years.
What I’m getting at is that the toolset we get exposed to is not what’s available in the labs. This stateless method of managing chat context is just how we are allowed to interact with it.
Do you have a source for this?
Alternatively they could design and run a single super-intelligent model, with no scalability constraints. Probably whey are already doing that as well.