Hacker News Reader

Comment by Almondsetat | Hacker News Reader

The $1500 Intel B70 with 32GB of VRAM can run this model at max context with good performance, btw. If you don't want to drop $5-10k for running DeepSeek this is your best budget option for local refactor/small scale dev help

View on HN

I understand the B70 is a bargain vs AMD and especially nVidia offerings, but to me it feels like I would be buying something that would feel too limited in less than a year. 48G would be much more confortable.

And I know the 96G nVidia cards are selling for over 10k$.

The future can’t arrive fast enough!

You can buy two b60s for $1300 right now (650 each) if you want a total of 48gb. Intel recently raised the price on all of their gpu's except the b60 series, so they are currently the best deal per gb I think.

They have terrible cross connectivity, it's not wise to share a model between them

32GB is perfect for models around 30B parameters. Since qwen has really hit the spot with their 27B dense models, I think it's a good bet. Also, 32GB is enough for other tasks such as image/video generation and loading multiple smaller specialized models

I wonder how this would stack up against 4x RTX 3060, assuming you have the physical room for them.

Since the 3060s would have to communicate constantly using pcie, I think I'd go with intel

Currently software is king when it comes to inference performance though.

Having the entire model in the vram without having to transfer anything is even more king, though

I have a B70, what llama options are you using and what performance are you seeing?

Oh, can it work with the /v1/completions/ auto-complete endpoint?

Sorry, I wrote autocompletion by force of habit. I simply meant it can complete code you have already created a structure for, which personally is very nice

I thought so, but thanks for the clarification. I am a little bit disappointed that local autocompletion models have been left by the wayside in favor of models post-trained for agentic coding. Both Codestral and Qwen-2.5-coder are more than a year old at this point, but local auto-complete seems to me to be such a great usecase.

The latest Qwen models (including 3.8 27B) do still support FIM-style in-editor code auto-completion if that's what you're looking for. I wouldn't want to use a large dense model like 27B for such a task (since FIM-style auto-completion really works best with low-latency responses), but it works.

You don't need $10k to run DeepSeek, I run it on a $1000 system.

Go on… what is this $1000 system?

4x CMP-170HX he purchased with his time machine last month

Could you elaborate please? Genuinely interested

at that price, probably loaded into ddr5 ram, a 3 bit or less quantization, and 3 tps

Reply on news.ycombinator.com