Hacker News Reader

Comment by mixedCase | Hacker News Reader

Some people need it, some don't. When I upgraded to a 7950X I was unable to find an excuse to buy more than 32GB. "That one time a script compiled clang by default with LTO and OOMed" was not a good enough reason.

If you use a lot of VMs, it comes in handy. If you do a lot of heavyweight compilation in parallel, same thing. But at least on Linux, popping 3 VS Code projects and 4 more browsers doesn't even hit 50% utilization.

View on HN

You can try running tests in parallel with the memory sanitizer (MSAN) enabled. MSAN uses a lot of memory, and on a 64 GB machine I have to reduce the amount of parallelism not to trigger Linux OoM killer.

    MSAN: https://github.com/google/sanitizers/wiki/MemorySanitizer

Funnily enough, RAM I find I can use no matter how much I have, especially now that you have LLMs. I can run a quantised Mixtral 8x7B on 64GB RAM.

But the 7950X is grossly underutilised, I don't know what to do with it.

Shouldn't the quantized LLMs be using the CPU to good effect? I'd imagine your Mixtral is substantially faster than it would be on a weaker CPU.

I have a 4080 with 16GB of VRAM. I experimented with llama.cpp by offloading layers onto GPU and doing the remaining on CPU. I found that it gives me max tokens/sec if I set it to 8 CPU cores as opposed to the 16 available on the 7950X. I guess beyond that, the bookkeeping between the cores might be taking up more time than it is worth.

the answer i found is run gentoo and compile yocto projects.

Yea, very workload dependent. Reposurgeon needed hundreds of gigabytes of ram to convert the GCC repository from SVN to Git, which was a lot of fun, but most of the time I would be fine with 32GB.

Reply on news.ycombinator.com