Speaking as software-developers we have to see RAG as kind of a caching-mechanism.

Instead of computing every token every time on the whole context, we can grab a cache to make some shortcut. We do the same in software development all the time. Of course it's a performance issue.