It's a bear to implement well. I'd start with Gemini and see how far that context window can take you. Then I'd work on summarizing fragments of earlier conversations down to key points to include in the broader conversation. Then I probably still wouldn't try to implement RAG unless time was not an issue.