I think no matter how large the context windows will get and no matter how fast the inference speeds will get, there will always be break-point where the context we have will be so large, that either cost or inference time are not a good experience in real life, and we have to split up the context.

We cannot simply state that at some point in time RAG will not be necessary. Like everything in the computer science world it always will depend on our data size and the resource constraints we have.

Unless of course we can process a corpus the size of the whole internet in <1 second. However, I doubt this can be achieved in the next 20 years.