Hacker News Reader

A 4-bit quantized 33B parameter model will fit on your GPU and you'll be able to use a 2048 token context too. (4-bit quantized larger models are better than smaller 8bit/16bit models)

You can run 4-bit quantized 65B models on your cpu, but it is slow, 1-2 tokens a second instead of 8-15 people typically get with a gpu, but you need two 24gb or an enterprise card with 48gb of ram to load them there.

https://old.reddit.com/r/LocalLLaMA/wiki/models has the information you are irritated about not being listed.

You can just barely fit a 33B GPTQ model in 24GB VRAM. It will be in 4-bit mode, and without maximum context size, but it will be quite fast. Or you can run from RAM+VRAM in GGML format with llama.cpp (or a derivative), which will easily fit 65B models even at 5 or 8 bits, but at much lower speed.

It's fairly simple to estimate ram requirements based on parameter count: https://blog.eleuther.ai/transformer-math/

The first link there gives very specific RAM requirements for each of the models.

"Reasonable speed" is subjective, though.