I was just suggesting something like this to a friend yesterday! (Neither of us know enough to do it or know if it's a good idea.)
However, I do think the context length is one of the top improvements that would make LLMs much more useful.
However, I do think the context length is one of the top improvements that would make LLMs much more useful.
The high-level memory read/writes are quite intuitive and you may be surprised at how closely it matches what you were suggesting to your friend.
Context length being so limited is the number one thing that rules LLM as possessing something that resemble “intelligence”, so if we this kind of unbounded context length we're entering into a completely new universe in terms of LLM abilities.