> If you have a linux machine with GPUs
How much VRAM one needs to run inference with llama 2 on a GPU approximately?
How much VRAM one needs to run inference with llama 2 on a GPU approximately?
Quantization still seems to have some issues for the 70B model.