What's the recommended way to run LLMs these days?
Ollama seems to work with DeepSeek R1 with enough memory using an older CPU but it's around 1 token/second on my desktop.
Ollama seems to work with DeepSeek R1 with enough memory using an older CPU but it's around 1 token/second on my desktop.
Second place is said to be the latest Macs with lots of unified memory, but it's a distant second place.
The recently announced hardware from nvidia is either underpowered, overpriced, or both, so there's not much point waiting for it.
If this pricing turns out to be true: https://www.reddit.com/r/LocalLLaMA/comments/1jgnye9/rtx_pro...
... then this generation of RTX Pro hardware sounds better to me. At the end of the day I don't know anything I didn't see on YouTube or /r/LocalLLaMA, though.