Ah, you're right, forgot to mention that. We use the Llama 2 7B 4 bit quantized model. The machine requirements are:
Ideal: 16Gb (GPU) RAM
Less Ideal: 8GB RAM and CPU
Ideal: 16Gb (GPU) RAM
Less Ideal: 8GB RAM and CPU
What about if I have a GPU with 8GB?
Khoj and your other apps need more RAM themselves, so practically 8GB of System or GPU RAM should suffice.
Khoj has been tested with CUDA and Metal capable GPUs. So Nvidia and Mac M1+ GPUs should work. I'm think it'll work with AMD GPUs out of the box too but let me know if it doesn't for you? I can look into what needs to be done to get that to work.
[1]: The calculation is [params] * [bytes] GB RAM, so 7 * 0.5 = 3.5Gb
PS. Nice to see an Hindi name for a software. For those who don't speak Hindi: https://en.m.wiktionary.org/wiki/%E0%A4%96%E0%A5%8B%E0%A4%9C...