Swap, ZRAM, Zswap and Hibernate on NixOS | Hacker News Reader

Swap, ZRAM, Zswap and Hibernate on NixOS

blog.matthewbrunelle.com

78 pointsspeckx27 comments

I also just recently enabled zswap on my NixOS setup, it's been working fine. I have 32GB of RAM and 64GB of swap, so I really don't need it, but it hasn't hurt.

I was expecting some problems with my switch to NixOS back in June, but it's been just superb. Even things like building deb packages for my Ubuntu 26.04 machines at work have not been a problem at all.

I had dabbled with NixOS for a year or two before making the switch and had some familiarity with the nixlang and various concepts (SOPS, age, flakes). But, what really smoothed out the switch was using Claude Code to develop the NixOS configs. Everything is easy to do that way, and any issues are easy to back out.

Claude code/Git/NixOS is such a nice system to work with.

You can build your entire system from a worktree and merge if it works or boot the old generation if it doesn't. This is great for trialing new kernels, NixOS releases, or alternate Window managers, and rolling back is trivial.

One downside is that there is only one stable NixOS at a time, and no LTS. However in practice I haven't found this to be an issue across multiple desktops, servers and SBCs.

Agreed. One additional thing I do is I have the same setup on my main work workstation and also my old work laptop, so when I do a big swath of package updates (about once a month) I do it on my "scratch monkey" first and make sure nothing obvious breaks. I started doing this after a GNOME update broke my GDM (as you say, until I booted the old generation, but still was a work interruption).

I'm using NixOS "latest" on my workstation, for no particularly good reason.

One success I had was with building up a custom vim setup. I had previously been using Lunar and Astro as an opionated vim distro with LSP+Treesitter and some other "table stakes" things a modern editor should have. But there wasn't a clear way to install Astro in NixOS when I tried it manually. So I just told Claude Code what plugins and settings I wanted in neovim, and now I have a single file that fully describes my vim setup. Before with Astro and Lunar the configs were spread out all over a bunch of files.

I was also able to get Claude Code to bundle up gitbutler-cli, which I wanted to try but so far have not. It builds it from source because there was no existing derivation. It takes a stunningly long time to build, but otherwise it has worked very well in my simple testing of it. Plus, I can build on my powerful laptop and then use that to seed my old, slow ThinkPad T470s and even slower chromebook that I also put my config on.

Yeah my setup is pretty similar, I use a shared repo for my work laptop and home desktop. I test out new setups at home, and typically run nixpkgs-unstable there for a few weeks before migrating the work machine.

I use nixvim for generating my neovim config (and I have a flake output for running that on MacOS where I don't use nix a lot) although I've been pairing back my plugins lately as I find I'm spending less time in the editor.

My MagSafe connector has just stopped working, so I'm going to need to send it in for service, which might mean a factory wipe.

I'm taking the opportunity to try adding my Mac to my nixos setup, pulling in some of my common settings like "ltra" alias for "ls -ltra" equivalent in "lsd" or "exa". Then nix-darwin to do as many of my settings via nix as possible.

Not on nixOS, but I thought I would weigh in on my preferred value for swappiness. 0. For some reason when I have a month of uptime (and I suspend once a day), doing anything with the desktop environment for the first time in a few hours causes a micro stutter for maybe 100ms, as if having to swap out from disk. Turning swap off and on again (meaning no pages are swapped to disk anymore) immediately fixes it. Opening a new tab, pressing the super key even performing certain actions in a video game cause the stutter. I know the internet tells me that 0 isn't a good value, but my computer becomes noticeably worse to use even with a swappiness of 5. I have enough RAM for anything I want to use this computer for, so swap is just for suspend for me.

Agree. I tried using a swapfile last year-ish, as the internet experts tell you to. I have 64GB of RAM. If I fresh boot and start a few apps I'll be using maybe 16GB RAM + 0GB swap. Then a few hours later I see I'm using 8GB of RAM + 8GB of swap... why? Why did Linux waste IO moving data to disk when there's no memory pressure?

That's why I don't use swap.

I suspect the intent is to move stale things out of RAM so it can be immediately used for more productive purposes when needed.

Shuffling out 8GB with a spinning hard drive could take a full minute…

That said, I agree with you. I’d rather not use swap unless absolutely necessary and I don’t want the latter to happen either…

Plus, I don’t want my storage in the “critical path” of memory accesses, even if rare.

> I suspect the intent is to move stale things out of RAM so it can be immediately used for more productive purposes when needed.

These are separate problems, though. You can clean a page by updating its backing store without purging it from memory; and purging a clean page doesn’t require a write. It’s definitely reasonable to take advantage of idle IO resources to clean dirty pages, but pages should only be purged (clean pages first, usually) when there’s an immediate need for them for something else.

Use swap for the important things: the part of your operating system at run time which doesn't get used much, but needs to be around when needed - the contents of /lib and /usr/lib and so on, all the *.so's and the things they allocate - shared libraries among the distro install, these things can be tidily swapped, until you need it.

If you need an application to always be responsive, start it with the swappable flag turned off, i.e. something like this:

    $ systemd-run --user --scope \
    -p MemorySwapMax=0 \
    -- /path/to/my/program arg1 arg2

Anyway, this is a distro-originated knob which every user must tweak for their use-case.

Generally the idea is that regions of memory that aren't being used are swapped out to disk, then other more useful things are brought into memory.

For example: you have a background program that you're not using and in the foreground you're crunching data from disk. In this scenario the most efficient use of your memory is to swap the background program out and pull the disk data into RAM.

But why should it swap things out even when I'm not crunching data and there's no memory pressure? I'd rather everything stay in memory and stay fast, since I have enough RAM.

Maybe not what's happening in your case, but stuff that is in swap will not be moved back to memory unless there's a need for it, so if you did have memory pressure at time 1, which resolved at time 2, then you might still see a lot of swap in use at time 3.

Your distro may vary on the policies around this, but the way the stack generally works, is that whatever shared libraries, and their memory resources, which haven't been touched in a while, are put into swap. So if you've run a few programs, left them open or whatever, switched the UI to other things and whatever over a period of a day, or even a few hours/minutes (its a knob), then bits of those programs will have gone cold. The local libraries and static things that haven't been executed - in a while - will move into that qualification.

You can tweak this, it is indeed a local-system configuration thing that a Linux user should know and think about - and maybe - if your distro has such a thing as a policy about it - this ought to be considered as well.

For example, I use Ubuntu Studio, precisely because it is generally well set up, swap-wise, to absolutely not interfere with audio performance (note: this DAW has been rock solid for decades) .. but every distro has its own 'medium ground' to consider when setting this.

The thesis/antithesis for/against swap is un-useful. Swap is a knob, not a switch.

It's a broken idea, throughput über alles isn't true of an interactive system, latency matters. There's a calculator app I use every now and then. Linux always thinks it's a good idea to swap it out since I'm not using it all the time, which means that every time I want to calculate something there's lag as it's swapped in again.

Another aspect I'd wonder about for relatively simple tasks is just freeing it from memory and being prepared to pay the cost of loading it again, it's probably already on disk. Latency matters, but it seems there will be limits especially considering the transfer speed and latency improvements we've seen to storage commonly installed in most systems over the past 15 years or so.

Then comes the challenge of deciding what processes/data to free or page, whether that's flagged by the system as it's running or at coding/compile time. And that seems similar to the issue of picking a swap strategy for a distro/OS, which will depend on the user's demands or system, setting a good default and exposing how to change the configuration to a better match if needed.

No sorry, its not a broken idea: throughput über alles is true of high-throughput-demanding systems, but latency matters is also true, for latency-demanding systems, and the two use cases represent - in an industrial sense - a very vast array of values. It's not really either/or - you are indeed identifying a knob that needs to be turned, in your particular use-case, on your particular system. There is no one-size fits all: it's a knob.

Is this considering RAM for disk cache too?

I've had good luck using zswap+zstd on Ubuntu 26.04. I have a Framework 13 (AMD) with 16GB, and have a 16GB swap file configured (ext4 root partition).

I had planned to upgrade the machine to 32GB of RAM but I don't really feel it's necessary at this point. Which is obviously great given the current RAM costs!

Thanks for this. So weird though - I literally spent the better part of the weekend reading and learning all about the differences between zram and zswap and ultimately deciding to switch out my previous zram config for zswap on my NixOS system! So seeing this pop up on my hackernews feed felt a bit spooky…

same, I literally just set up my Framework 13 Pro today and was looking for the current recommended zswap settings... looks like a bunch of us are in this thread

I agree with everything else here, but swap files on btrfs should probably be avoided whenever possible [0].

[0]: https://btrfs.readthedocs.io/en/latest/Swapfile.html

Right, you are good to do swap files on btrfs since linux 5.0 or so as long as you follow some rules. Disko thankfully covers all those details for you, but I shouldn't have skipped over that in the piece. I'll make an update when I get a chance.

Swap files work, but I wouldn't quite say "you are good to do swap files on btrfs" because the official documentation (the page I linked earlier) says

> filesystem - must be only single device

> This means swapfiles will have an amplified impact, much larger than the size of the swapfile itself.

> Users should be aware of the impact when using swapfiles alongside other data. This is especially discouraged for root filesystem.

If a swap file is the only thing that will work with your setup, then it's fine to use one, but if you're at all able to use a swap partition, then that's a far better choice. (Swap files are fine on ext4 and xfs; it's only COW filesystems like btrfs and zfs where they should be avoided.)

While everything else is pretty obvious from the nature of an unmovable file, "scrub will skip those block groups which have extents of an active swapfile on them, and those block groups may also contain extents belonging to other files".

Then again, I mostly use XFS, which has no data checksums at all, so YMMV.

> I mostly want to note that swap files can complicate hibernate.

Only in Linux - Windows has no problem with swap in a file. Swap partition is an awful idea because you cannot easily resize it.

Zram is also a bad idea, when it is filled, the system starts swapping to a disk, and old pages nobody needs are kept in RAM forever. Zswap is better in this aspect - old pages are pushed to the disk and newer ones kept in memory. But Linux distributions use zram by default.

Also, it seems that many distributions now place /tmp in RAM (tmpfs). Whoever made such decision is not very bright. When you place a large file in /tmp (for example, viewing a large compressed archive which is automatically unpacked into a temporary location), it fills the memory and the systems locks up completely, including mouse and GUI - because bright people developing Linux cannot think of reserving resources for desktop environment to prevent exactly this situation. Why reserve memory for GUI? Obviously files placed into /tmp are more important. Who needs GUI nowadays when you can easily connect over SSH from another machine and kill the misbehaving program.

If you are making a Linux distribution, do not place /tmp into memory. Or at least check how much RAM user has, and add a limit on tmpfs after which files are pushed onto the disk. Also, do reserve memory for window manager with high priority.

whether to use zram or zswap depends on what your intentions are, hibernation doesn't matter (use priority)

if your intention is to have enough compressed ram such that you never reach the limit (compression ratios are usually around 3 with zstd) you can allocate over 100% of your ram in zram. then have 100% of your ram as a swap partition for hibernation. this works extremely well but should you exhaust your zram swap serious problems may begin.

zswap will be using your SSD and putting wear on it and also imposing latency spikes on old page access, only makes sense if you are certain your swap usage may spike too high in my opinion.

Reply on news.ycombinator.com