Hacker News Reader

To add to this, I have a laptop with 32G of RAM and am able to run some 7B models on CPU. But I'd like to work on some larger models. Are there any eGPUs that can aid in this?

View on HN

I have a 12-year-old desktop with 24GB of RAM and no modern GPU and can run the 7B models, it's just no fun :(

I have a similarly old desktop with 24GB RAM and outdated GPU. What sort of token/second were you able to get with the 7B models?

Reply on news.ycombinator.com