HN
Hacker News
Top
New
Best
Ask
Show
Jobs
Comment by qumpis | Hacker News Reader
Parent
Full thread
qumpis
·
What's your ram usage for this?
View on HN
JoeMattie
·
I've got the (4-bit quantized) 65B param model running at somewhat acceptable speed on an i9-7900. It uses around 55GB of RAM.
ddren
·
The (quantized) 13B model is 7.6 GB on disk and the program uses around 8 GB to run. It runs without hitting the swap with just 9 GB assigned to WSL2.
Reply on news.ycombinator.com