I built a RAG implementation for 35k books/articles/wiki pages/web pages i collected over the years(it took about 6 weeks on 3070ti 100% constant usage). I query it with various steps of data extraction/narrative building/refining etc, over LLMs. Almost daily i figure out new steps to add to the pipeline and honestly, i could not imagine learning about niche topic x from so many perspectives/periods in such a short time(including the original source). I did not yet figure out how to package this, but i spend at least 2h of my free time daily with it. Ideas and feedback is welcome.