HN
Hacker News
Top
New
Best
Ask
Show
Jobs
Comment by csomar | Hacker News Reader
Parent
Full thread
csomar
·
Only 21.3GB is required.
View on HN
guerrilla
·
Is this wrong? It says 65.8GB. If it's wrong, what source should I be using instead?
https://llm.extractum.io/model/Qwen%2FQwen2.5-Coder-32B-Inst...
exe34
·
the ollama one is probably quantised.
mistercheph
·
Ollama's default quantization is q4_0 which is quite bad, but you can go to ollama's model page to see all the quantizations they have available, you can do e.g. "ollama run qwen2.5-coder:32b-instruct-q8_0" which will need ~35G + space for context
tmikaeld
·
... with 4-bit quant and 4000 token context.
Reply on news.ycombinator.com