ParentFull threadinfinityio·a very rough approximation is 2GB of vram for every billion fp16 model parameters, so the lower end models may be just about achievable on high-end cardsView on HN