Hacker News Reader

Local LLMs and other AI models. Even on my 4090 I run into a lot of limitations, to the point that I've been debating whether I buy a second one or wait on the 5090 to release and move my 4090 to secondary. I've even considered going with moire then 2 cards just to get some larger models running(still cheaper then the high memory server/workstation cards).

View on HN

This comment is assuming the 5090 will come with more VRAM right? I haven't heard if that will be the case for consumer cards, it would be obviously nice if nvidia did, but it doesn't seem like it is in their best interest to do so versus just offering it on their high VRAM on their datacenter/workstation cards, unless e.g. Intel or AMD starts putting pressure on them at this front.

A 4090 runs a lot of these ML models more than fast enough, the problem is just VRAM right now. I think a lot of local LLM people think even a 3090 is plenty fast enough.

Reply on news.ycombinator.com