Comment by fgonzag | Hacker News Reader

Comment by fgonzag | Hacker News Reader

Parent Full thread

My first switch was to open code + open router. I used it to try mixing models for different tasks and to try open weights models before committing to the hardware.

Even paying API pricing it was significantly cheaper than the nearly $500 I was paying monthly (I was spending about $100 month combined between Claude pro, chat gpt plus, and open router credits).

Only when I knew exactly the setup I wanted locally did I start looking at hardware. That part has been a PITA since I went with AMD for budget reasons and it looks like I'll be writing my own inference engine soon, but I could have gone with Nvidia and had much less issues (for double the cost, dual Blackwell's vs quad Radeon W7900s for 192GB of VRAM).

If you spend twice what I did and go Nvidia you should have nearly no issues running any models. But using open router is super easy, there are always free models (grok famously was free for a while), and there are very cheap and decent models.

All of this doesn't matter if you aren't paying for your AI usage out of pocket. I was so Anthropics and OpenAIs value proposition vs basically free Gemini + open router or local models is just not there for me.

> but I could have gone with Nvidia and had much less issues (for double the cost, dual Blackwell's vs quad Radeon W7900s for 192GB of VRAM).

> If you spend twice what I did and go Nvidia you should have nearly no issues running any models.

I goodled what a Radeon W7900 costs and the result on Amazon was €2800 a piece. You say "quad" so that's €11200 (and that's just the GPUs).

You also say "spend twice what I did", which would put the total hardware costs at ~€25000 total.

Excuse me, but this is peak HN detachment from the experience of most people. You propose spending the cost of a car on hardware.

The average person will just pay Anthropic €20 or €100 per month and call it a day, for now.

I see a ton of my peers driving around in 80k cars. I drive a 20k used one.

I'm planning a writing a ROCM inference engine anyways, or at least contributing to the rocm vllm or sglang implementations for my cards since I'm interested in the field. Funnily enough, I wouldn't consider myself bullish on AI, I just want to really learn the field so I can evaluate where it's heading.

I spent about 10k on the cards, though the upgrades were piece meal as I found them cheap. I still have to get custom water blocks for them since the original W7900s (which are cheap) are triple slot, so you can't fit 4 of them in any sort of workstation setup (I even looked at rack mount options).

Bought a used thread ripper pro wrx80 motherboard ($600), I bought the cheapest TR Pro CPU for the MB (3945wx, $150), I bought 3 128Gb DDR4-3200 sticks at 230 each before the craze, was planning on populating all 8 channels if prices went down a bit. Each stick is now 900, more than I paid for all 3 combined (730 with S&H and taxes). So the system is staying as is until prices come down a bit.

For AI assisted programming, the best value prop by far is Gemini (free) as the orchestrator + open code using either free models or grok / minimax / glm through their very cheap plans (for minimax or glm) or open router which is very cheap. You can also find some interest providers like Cerebras, who get silly fast token generation, which enables interesting cases.

Reply on news.ycombinator.com