Hacker News Reader

Comment by sgtnoodle | Hacker News Reader

I run wireguard on an even lower end Linksys router, and it seems to work just fine. I suppose my internet connection isn't the fastest, though.

It's interesting to see someone using a single board computer as their main computer, and preferring a Linux distro that compiles everything from source.

Why bother with docker for a home server other than for the fun of it?

I've been embracing systemd recently, and using it to manage processes in an embedded system. Once you wrap your head around how services work, it seems pretty convenient compared to init scripts. Notably, systemd works with legacy init scripts, so you could still go that route if you wanted to. Maybe throwing docker into the mix makes it clunkier, though?

View on HN

> Why bother with docker for a home server other than for the fun of it?

I do this. Over time you forget how each service was configured, or simply don't care. Adding more and more stuff to a home server increases the complexity and the attack surface more than linearly in the number of services.

I run nearly all my home services in docker, and I have a cookie-cutter approach for generating SSL certs and nginx config for SSL termination (not dockerized). Provisioning is automated through ansible, so my machines can be cattle not pets, as far as is possible on 3 raspberry pis.

Same, but I haven't started with anything like Ansible yet, only beginning to learn it at work.

Running all my services in Docker keeps it all clean because I'm a very messy person when it comes to Linux. Change, change, change, it works, forget about it, it breaks, find something I did years ago tripping me up now, change, change, change, it works, forget about it.

With docker every service is contained and nearly separated. I can rip something out and replace it like stacking a new network switch in the rack. Delete the container, delete the image(s) and delete the volume if I want to start over with something completely fresh.

I can move everything to a new server by moving bulk hard drives over, restoring docker volumes from backup and cloning docker-compose configs from git. Haven't tried any distributed volume storage yet.

> Haven't tried any distributed volume storage yet.

Having tried Gluster, Ceph/Rook, and Longhorn, I strongly recommend Longhorn. Gluster is kinda clunky to setup but works, albeit with very little built-in observability. Ceph on its own also works but has some fairly intense hardware requirements. Ceph with Rook is a nightmare. Longhorn works great (as long as you use ext4 for the underlying filesystem), has good observability, is easy to install and uninstall, and has lower hardware requirements.

Its main drawback is it only supports replication, not erasure coding, which tbf is a large contributor to its ease of use and lower hardware requirements.

longhorn has no authentication at the moment, so any workload running in your cluster can send API requests to delete any volume. I think they are working on it but it might not be the best solution unless you deploy a security policy to prevent network access to the API pod.

>Why bother with docker for a home server other than for the fun of it?

Because it legitimately makes things easier. Your source of truth for how an app was set up is the compose file, and as a result you only need to care about the storage each app uses for its state. This makes upgrades and back ups almost entirely painless and drama free.

Deploying the average app the old way feels downright medieval.

NixOS uses systemd to manage containers. Works great and is hassle free IMO.

NixOS seems to make needless complexity a point of pride.

NixOS makes purity and declarative configuration a point of pride. With that comes greater complexity.

None of the complexity is needless - it all follows from its core principles.

Maybe you disagree with those principles. Maybe you disagree with software having hard principles.

That's fine. NixOS isn't for you. It's wonderful for me though!

What's the selling point, what's wonderful about it?

Entire OS configured with code, which can be modular with imports and functions. Want to run a Postgres server? One-liner in your config. Want to advertise yourself over avahi? Same deal. Need custom udev rules? One-liner again.

I literally cannot bork my machine. I can always roll back. Useful when messing with low-level stuff like drivers.

I have full control and understanding of updates and the exact versions of all the programs I run. I can always read the source of any program I'm running by looking in the nix store.

Super easy to package new programs and contribute to nixpkgs.

Super easy to tweak packaging other provided without forking via overlays.

I share config across devices in a git repo, so provisioning a new machine is trivial. I got a desktop and I basically imported a bunch of common config along with like 10 lines of config specific to the desktop.

> I literally cannot bork my machine. I can always roll back.

So use a templated VM, or a filesystem with easy snapshotting like ZFS or BTRFS.

The former also allows for easy provisioning of new machines, and doesn't lock you down to a single OS.

I never said NixOS uniquely solves that.

And that doesn't have all the other benefits. Also it's a worse way to handle state and rollbacks. Sledgehammer for a thumbtack.

I already spent the time to learn the hard stuff. So now my life is made easy by NixOS. Same idea behind learning Haskell. Huge personal gains.

I'd say it's more like vim. Steep initial learning curve for a few months then it becomes pretty easy and indespensible. Perhaps Haskell is like this but I never got past the curve.

Are templated VMs contained within a few kilobytes of config files? Can anything in the entire system be included or excluding by tweaking these text files? Can it be used as a desktop on bare metal?

ZFS and BTRFS are orthogonal and often used in conjunction with Nix.

This is what I love about NixOS users - how belligerently defensive y'all get immediately when someone pokes at your project.

> Are templated VMs contained within a few kilobytes of confif [sic] files?

Not kilobytes, but Proxmox supports sparse images so pretty small. More importantly, disk space is cheap as hell, and I value my time way more than a couple of hundred megs of space.

> Can anything in the entire system be included or excluding by tweaking these text files?

Personally I template my images with Ansible, so yes in fact anything can be included or excluded with a text file.

> Can it be used as a desktop on bare metal?

Who cares? The performance hit from a modern Type 1 hypervisor is so small as to only matter if you're also the kind of person who is tweaking obscure CFLAGS for emerge, which is to say it doesn't matter.

> ZFS and BTRFS are orthogonal

Only in that they aren't an OS, obviously, but they perform the same function (rollback) that you mentioned as a positive point for NixOS.

> Who cares? The performance hit from a modern Type 1 hypervisor is so small as to only matter if you're also the kind of person who is tweaking obscure CFLAGS for emerge, which is to say it doesn't matter.

I want to use it as my personal computer...using a VM seems even more fringe and niche than NixOS there lol. And for a home network, VM also seems overkill.

I suppose there are also network effects at play. If you use NixOS for a laptop and desktop, suddenly using it for home infra is actually more economical than using other tools.

This person's main argument is that because Ansible can be used to accomplish some of the benefits (for some definition of accomplish) of NixOS, that NixOS isn't doing anything special or differentiating.

I also get a sense they are for some reason solely focused on cattle server use-cases. I'd say the OP (home infra) is in between your production servers and your PC.

I run my home infra as sort-of-cattle. Nothing that matters is stored solely on local disk. If my Mac were to die, I'd probably spend an hour or so waiting for Homebrew and asdf to install stuff, and I'd have to manually grab some files from GitHub. The worst case would be my Windows desktop dying, since I have nothing in the way of repeatability set up for, but all I use it for is Steam. Again, mostly just dealing with the annoyance of installing Windows and waiting for Steam to re-download a ton of games.

Conversely, I can lose a k8s node and have nothing change. If I lose my NAS (separate node, separate Proxmox cluster), I'd have to boot up the backup (which boots daily to sync, then shuts down) and run an Ansible play to change its IP address so that all the NFS targets still worked. I could make that more automated, I suppose, but it's an unlikely scenario so I'm fine with the small amount of manual labor.

I guess my point is that I don't see the benefit in having a special OS for daily use. If I want to fiddle around and possibly break things, I don't want to be doing that on the device I use daily. I used Gentoo for years in the early 2000s, and no longer have the time or patience for my main computer breaking constantly. If I want to play with something, I spin up a VM. If I want to play with something baremetal, I have an old Dell T310 I can use, and a couple of ancient Macbooks somewhere.

> I literally cannot bork my machine.

I literally cannot be paid to care about borking my machine. It takes ten minutes to reinstall.

For stuff that cannot conveniently be installed locally (hello multiple versions of DaVinci Resolve) or ideally want to be ephemeral (hello basically all the development environments I use), I've got Docker.

The problems solved by NixOS are mostly solved already. NixOS solves them differently, and that's OK, but it's not solving them for the first time.

I've never been able to restore my entire machine completely from config files to the exact same state reliably. What else has solved this?

From top of my head:

A- If you want to be able to restore to any point after installing your system: LVM/ZFS/BTRFS snapshots got you covered.

B1- If you want to install your system the way you want, you have Kickstart (RedHat family) and Preseed (Debian family). You can provide an installation template to the installers, and they install your system the way you want.

You want to get your /etc/ afterwards the installation? Either provide it on a disk, or your local network, or anywhere on the internet. Either integrate it to your Kickstart/Preseed end hooks, or do it after your first boot.

B2- If you want something more fancy? Create a FAI installation media which installs the system you want.

C- You want to transform an existing Debian system to something you want? You get the dpkg package selection state and set to the target system, apply the state so the system is converted to your own selections. Apply the /etc via git or any way you want.

D- Want something more programmatic? Run an Ansible playbook locally (This is how GitLab installs and configures itself during install).

E- For fleet installations there are XCAT and network variants of B1, but they are out of scope for personal systems.

We use E in our system room. I used B1 for personal systems and B2 to deploy an installation country-wide via USB sticks. I used C in the same project for a small subset of servers. In these cases, all systems are operational in the first boot, starting from a known state.

I know a lot of people using A, and nudging me to try/use it. GitLab uses D every time I upgrade it.

Thanks for all the details. "A" can't really be checked into git due to size, and doesn't allow for forward config changes, just rollbacks. Do B1 or B2 allow updates to an existing system? C so far seems the closest to Nix, though it's missing the configuration side. I used D before and frankly Ansible is not great. It is meant to be idempotent but has so many holes that it never really is, and because it's a separate layer from the system itself, the scripts might not work after a system upgrade. With Nix, the config IS the system. E sounds interesting, never heard of it - sounds like you saying it's not suitable for personal use though for some reason.

I never said NixOS is uniquely solving any problems. But - none of the other solutions are really quite the same or equivalent. In total, its unified approach is relatively unique.

I really don't see myself using another Linux distro. It would make more sense to just modify my NixOS config. Don't know what another distro would even buy me tbh.

Does nix use underneath, or something else?

* use PodMan, docker, or?

NixOS's virtualisation module can use either Docker or Podman and if Podman is enabled it has a Docker compatiblity mode (via `virtualisation.podman.dockerCompat = true`) that puts symlinks in all the right places (`docker` binary, `docker.sock`) so that software doesn't even know it's not running vanilla Docker :)

Really not seeing the point of using docker-anything on NixOS here. Shouldn’t Nix already be able to isolate dependency trees from each other? Why would you want to duplicate all those files yet again in a docker image?

Sounds good in theory but there are a couple reasons to use docker. The first is that there are endless packages already prepared for docker that you'd have to manually set up on any OS, let alone Nix. For instance "itzg/minecraft-server". The second is that if you use docker you've got control of where all the stateful volume data sits. I keep it all in one folder for easy backup. The rest of the system is fully managed by nix.

> Why bother with docker for a home server other than for the fun of it?

As others have alluded to, mostly for having everything in code. I went from Docker --> Docker-Compose --> Kubernetes. The latter is 100% overkill, and was mostly done to assist learning. Still, it's very nice to have as close to HA home services as possible (with the exception of a dedicated failover WAN connection, and me needing to manually fire up a generator for power - UPS will last ~15-20 minutes).

Scheduling aside, though, yes you could accomplish all of that with systemd and something like keepalived.

> Maybe throwing docker into the mix makes it clunkier, though?

Correct. It increases the attack surface without adding benefits over light containers like systemd-nspawn or just systemd unit files.

Reply on news.ycombinator.com