Hacker News Reader

Comment by segmondy | Hacker News Reader

The same way you use a cloud LLM.

I think the point was that for example for programming, people perceive state of the art LLMs as being net positive contributors, at least for mainstream programming languages and tasks, and I guess local LLMs aren't net positive contributors (i.e. an experienced programmer can build the same thing at least as fast when using an LLM).

I know this is false, DeepSeekv3.1, GLM4.5, KimiK2-0905, Qwen-235B are all solid open models. Last night, I vibed rough 1300 lines of C server code in about an hour. 0 compilation error, ran without errors and got the job done. I want to meet this experienced programmer that can knock out 1300 lines of C code in an hour.

Without knowing what you were doing with that 1300 lines of code, there's not much insight that can be had from this.

How much machine do you have to be able to run Qwen-235B locally?

Are 235B models classified as local LLMs? I guess they probably are, but others in this thread are probably looking more toward 20B-30B models and sizes that generally fit on the RAM you'd expect in average or slightly-higher-end hardware.

My beefy 3D gamedev workstation with a 4090 and 128GB RAM can't even run a 235B model unless it's extremely quantized (and even then, only at like single-digit tokens/minute).

Can you run any of those models without $20 000 worth of hardware that uses as much power and makes as much noise as a small factory?

I run them with under $3,000 hardware and inference is about 500-600watts with no noise.

I’m a mediocre C programmer on my best day and I assure you a highly competent programmer could probably use 200 lines of code to do what I achieve in 1300.

Just counting lines is not a good proxy for how much effort it would take a good programmer.

(And I am 100% pro LLM coding, just saying this isn’t a great argument)

Reply on news.ycombinator.com