You don't need Langchain, here's how to do RAG without it

I might be wrong here. I just know some product quantization techniques, but you can reduce the index by a lot! However, from my research, the more size you reduce, the more retrieval quality is also reduced.

Quoting from https://github.com/criteo/autofaiss

> Using faiss efficient indices, binary search, and heuristics, Autofaiss makes it possible to automatically build in 3 hours a large (200 million vectors, 1TB) KNN index in a low amount of memory (15 GB) with latency in milliseconds (10ms).

I would highly recommend taking a look at AutoFaiss. You just have to set the maximum memory to build the index, and it will come up with the configuration.