Hacker News Reader

Comment by denysvitali | Hacker News Reader

Their benchmarks are based on a machine with 72GB of RAM and another with 128GB of RAM - not that it does matter, but I'm really curious if they're outliers or if I should bump my computer specs. That's a lot of Electron web apps running in parallel!

View on HN

I benchmarked Rusts performance on 2, 4, 8 and 16 cores on several widely used crates (https://arewefastyet.pages.dev/). Performance generally improved with the number of cores, but memory made no difference. All of these crates topped out at around 1GB and additional memory didn’t improve performance.

So no, 72GB or 128GB makes no difference.

Yes, maybe I didn't make it clear but I'm pretty sure this is not impacting for Rust. I was just wondering if it's normal to have these specs since my computer has probably "just" 32GB of RAM

I agree that memory size isn't going to matter much, but memory speed should be noticeable, especially on large projects.

Definitely outliers. The Steam hardware survey [0] is obviously not a perfect measurement, but in its February results only ~3% had 64GB and only ~0.3% had more than that.

I recently upgraded to 64GB to run quantized LLM models, but when I'm not using an LLM I rarely use more than 16GB, much less 32GB.

[0] https://store.steampowered.com/hwsurvey

I have 32GB on my Linux laptop (plus 12GB swap) and regularly run out of RAM with just a handful of VSCode's open (it's generally the language servers using all the RAM) and a couple of dozen Firefox tabs (Firefox seems to be much worse than Chrome RAM-wise). If I run my actual work project, which needs a few gigs then it often just completely runs out of memory and hard reboots.

Never have the same problem on Windows even though I only have 16GB there. Feels like Linux is just really bad at memory management. That may be skewing the Steam survey since almost everyone there will be using Windows.

On Linux I wish I had 64GB.

Have you tried zram for instance? There are a lot of things one can do to better utilise RAM on Linux.

I believe Fedora comes ootb with a lot of things to make ram management quite good. Using another distro these days, but Fedora impressed me on that front.

(edit: Try to use zram and zswap, check if your i/o scheduler is correct for your hardware, and increase your swapiness value. These should make your system behave better. For other performance improvements, use ananicy and irqbalance.)

Yes I have zram enabled. First thing I tried.

Even if it is possible to fix this, it's still a horrible bug because you absolutely should not have to blindly tweak settings to stop a desktop computer from hard-rebooting when it runs out of RAM.

I think this is a really fundamental problem on Linux though - it would require enormous changes and unprecedented cooperation between different groups for a proper fix. Much easier to say "you're holding it wrong".

Sounds like you need some swap (?)

Set your swap to double your RAM. Windows has a swap file that grows as much as it needs to.

I'm pretty sure it doesn't. Windows lets you set the swap size limit. The biggest difference is that Windows doesn't over-commit, so you need a big swap file otherwise you'll be running out of memory all the time even before you actually use all your physical RAM.

You might just be misunderstanding Linux's RAM management strategy. Do you actually start getting out of memory errors, or do you see "free" memory rapidly go to zero?

If it's the latter, that's because Linux deliberately uses as much RAM as possible for caches and such, which means "free" should be near zero most of the time. The number that actually matters on Linux is "available" memory, which is the amount of memory that is currently free or used for optional caches and could be reallocated for a new required chunk of RAM should an application need it. Here [0] is a decent in-depth explanation.

If it's the former, there's something very wrong with your installation, because every Linux system I have makes much better usage of memory than Windows, even running a heavyweight DE like KDE. Even my machine that dual-boots Linux and Windows performs noticeably better on all metrics (including memory usage) when in Linux.

[0] https://linuxblog.io/free-vs-available-memory-in-linux/

Some people need it, some don't. When I upgraded to a 7950X I was unable to find an excuse to buy more than 32GB. "That one time a script compiled clang by default with LTO and OOMed" was not a good enough reason.

If you use a lot of VMs, it comes in handy. If you do a lot of heavyweight compilation in parallel, same thing. But at least on Linux, popping 3 VS Code projects and 4 more browsers doesn't even hit 50% utilization.

Funnily enough, RAM I find I can use no matter how much I have, especially now that you have LLMs. I can run a quantised Mixtral 8x7B on 64GB RAM.

But the 7950X is grossly underutilised, I don't know what to do with it.

Shouldn't the quantized LLMs be using the CPU to good effect? I'd imagine your Mixtral is substantially faster than it would be on a weaker CPU.

I have a 4080 with 16GB of VRAM. I experimented with llama.cpp by offloading layers onto GPU and doing the remaining on CPU. I found that it gives me max tokens/sec if I set it to 8 CPU cores as opposed to the 16 available on the 7950X. I guess beyond that, the bookkeeping between the cores might be taking up more time than it is worth.

the answer i found is run gentoo and compile yocto projects.

Yea, very workload dependent. Reposurgeon needed hundreds of gigabytes of ram to convert the GCC repository from SVN to Git, which was a lot of fun, but most of the time I would be fine with 32GB.

You can try running tests in parallel with the memory sanitizer (MSAN) enabled. MSAN uses a lot of memory, and on a 64 GB machine I have to reduce the amount of parallelism not to trigger Linux OoM killer.

    MSAN: https://github.com/google/sanitizers/wiki/MemorySanitizer

If you are like me and opening too many browser tabs (and other applications, of course), you need a lot of RAM. My desktop has only 48 GB of RAM right now and a typical VM usage is around 80--100 GB. I will probably go for 128 GB or possibly 192 GB of RAM for the next desktop.

Rust builds, in comparison, don't actually reach that far, mainly because it is proportional to the maximum number of parallel jobs. In my experience, 48 GB was more than plenty for 8 parallel jobs if I wasn't doing anything else.

My understanding is that processor performance is more important for rust compilation than available memory, but ultimately the atrocious compilation times are why I struggled so much with Rust.

Reply on news.ycombinator.com