Comment by brimwats | Hacker News Reader

Top New Best Ask Show Jobs

Comment by brimwats | Hacker News Reader

Parent Full thread

I haven't seen any repos or guides to using llama on that level of RAM, which is something I do have. any pointers?

Run text-generation-webui with llama.cpp: https://github.com/oobabooga/text-generation-webui

And here are the benchmarks

https://github.com/ggerganov/llama.cpp/issues/34

Reply on news.ycombinator.com