Hacker News Reader

Comment by MayeulC | Hacker News Reader

IIRC, swap is actually needed for some memory operations. And when you run out of memory, the behaviour is often worse without swap.

These days I always at least configure an in-memory compressed swap (zram).

View on HN

You recall incorrectly; swap is not needed. It's not just me who runs without it; Google production machines did for many years.

"The behavior is often worse without swap" is more vague / subjective. I prefer that a process die cleanly than everything slow to a crawl semi-permanently. I've previously written about swap causing the latter: https://news.ycombinator.com/item?id=13715917 To some extent the bad behavior can happen even without swap because non-locked file-backed pages get paged in slowly also, but it seems significantly worse with swap.

zram is a decent idea though. I use it on my memory-limited Raspberry Pi machines.

I'm surprised that the swap implementation (or distros swap configuration) is so bad on Linux. I have a fresh Ubuntu 20.10 desktop installation and a few times it has come to slow crawl where you can't do anything, can't escape to tty or ssh, reboot is basically the only option.

I recently learned searching the internets that it was the swapfile configuration. My 16GB RAM machine got a 2GB swapfile from the default installation so I doubled it to 4GB, but today it almost got full again, luckily I was just about to close a big memory hog and that saved me. I have now increased the swapfile to 6GB.

If Ubuntu is giving default installations 2GB of swapfile on 16GB of RAM, shouldn't this happen to lots of users quite frequently? How many users are technical enough to understand this and fix it? How can this be a good solution?

I can't recall that I ever got in trouble on Windows because of the swapfile.

I had zero problems without swap-files for a long time, thinking that 16gb was a lot of ram, and that i could manage. it wasn't and i couldn't.

Now I run two swap partitions the size of physical memory on all machines. Turns out that after some weeks a good third of allocated ram lives in swap proving its usefulness .. and that its quick to turn the aux-swap partition into one for lvm snapshots and i suppose to act as part of the set to emulate'baloon files' too.

tl;dr; bad defaults. go with the size of physical ram and get suspend/hibernate/hybrid working and tested too.

use a 2nd ram-sized partition for snapshots/lvm and emergency space/lvm|zfs.

That explains it, I noticed that if my machine was on for a while a degradation in performance happened, it was probably the swapfile then too, especially when it only has 2GB to work with.

Yes, I probably need to set my swapfile to 16GB, I'm also investigating the swappiness setting, default 60 for desktop.

I often went into similar issues before upgrading my RAM. SysRq+F is your friend :)

https://en.m.wikipedia.org/wiki/Magic_SysRq_key

That manually triggers OOM kill, making your system responsive again at the risk of killing something it shouldn't. At some point, I was probably using it five times a day. It got better when I started using swap (easier to recover when both full). I really recommend zram.

>> It's not just me who runs without it; Google production machines did for many years.

But then you need other mechanisms monitoring "out of memory" situations. For example Kubernetes also require nodes to have swap disabled. But running workloads should have resource request/limits defined and then kubelet config specifies minimums on free disk and RAM memory. If there is not emough RAM, workloads are evicted to another worker node with more free memory automatically.

I agree with your comment if you remove the words "but then" and "other". It's a good idea for orchestration software to detect/avoid unhealthy nodes and enforce the resource limits it uses for bin-packing. It's also a good idea for monitoring systems (by which I mean ones which do alerting and visualization) to track memory usage and pressure stalls. [1] I wouldn't say swap is a substitute for these in any way. A swapping machine is an unhealthy machine.

[1] https://www.kernel.org/doc/html/latest/accounting/psi.html

Well, I'm not inventing it. This phoronix article [1] leads to a LKML post [2]:

> Yeah that's a known problem, made worse SSD's in fact, as they are able > to keep refaulting the last remaining file pages fast enough, so there > is still apparent progress in reclaim and OOM doesn't kick in.

And so on. That's part of the reason there is a working effort towards creating a userspace OOM daemon. I'm grateful for that effort, since I'd rather have some apps crash than having my system unusable. However, there is an issue with the Linux kernel trashing.

I couldn't find mention of it in that thread, but it's mentioned multiple times in passing (as well as in the phoronix comments) that the system behaves worse without swap. It's also noted as one of the reproduction steps.

I agree that it's vague and subjective, but I'm pretty sure I've seen some serious discussion of the topic. There are a few sources that point in that direction [3-4], but I haven't seen anything conclusive.

[1]: https://www.phoronix.com/scan.php?page=news_item&px=Linux-Do...

[2]: https://lkml.org/lkml/2019/8/5/416

[3]: https://unix.stackexchange.com/a/28176

[4]: https://serverfault.com/questions/255661/linux-oom-disk-i-o-...

Depends if you want things to gracefully degrade because you know you don't have enough RAM or if you'd rather things just straight up die. E.g. for the things I work on my laptop with if whatever I do isn't going to work with 128 GB of RAM (80% of which was meant to be cached data not actually used) then it's because it went horribly wrong and needs to be halted not because I needed some swap which is just going to try to hide that things have gone horribly wrong for a minute and then die anyways. Now if I were doing the same things on a machine with 8 GB or 16 GB of RAM then yeah I want to gracefully handle running out of physical memory because things are probably working correctly it's just a heavier load and it can be better to swap pages to disk than drop them from a small amount of cache completely.

Personally, I prefer things that use a huge amount of memory to die instead of having the machine become unresponsive, especially if I have latency-sensitive things at the same time, like a conference call.

Now, my main issue is that disabling swap doesn't cause this. With swap, OOM ends up being detected and corrected by the kernel. On some machines, it doesn't happen without swap, and often neither with swap. A sweet spot seems to be current-to-last CPU gen with 8 GB of memory.

There's probably a lot of factors at play: what makes a task "hung"? How memory pressure is measured. Stuff probably changed with time, SSDs are now pretty quick, yet storage is a lot slower than RAM compared to what it used to be.

Anyway, I use zram and have upgraded my memory, I'll also increase a bit my swap partition size in light of all of this. I'll probably have a look at userspace OOM daemons too.

Reply on news.ycombinator.com