Hacker News Reader

Comment by rewtraw | Hacker News Reader

Totally agree that this is impressive, but overkill. I've been running a linux NAS with a few services (transmission, sab, nzbget, the other usual suspects) for over a decade now, starting with on-host services and later migrating to docker+compose (which greatly simplified the bootstrapping process). Recently migrated to TrueNAS to get an even more appliance-like setup, and was very impressed in how quickly I was up and running from scratch. I can have NextCloud deployed in about 10 seconds if I want. Maybe I'm just at the point where I don't want to have to tinker/micromanage my server, but the idea of running kubernetes for a home server gives me anxiety.

View on HN

I run microk8s at home not because it’s easier, but because I want to retain the skills/knowledge if I ever have to work somewhere where k8s is used again. Given how ubiquitous it has become there’s a decent likelihood of that being the case if I go looking for work as a backend software engineer. It also has a couple of benefits in that if I ever decide to move my stuff to a cloud provider like DigitalOcean it’s probably going to be dead simple compared to setting up and reconfiguring something running in a VM.

Yeah, I tried swapping my home server stuff to Kubernetes and regretted it. Not only was it much more complex and indirect, but it required more maintenance. After some months on Kubernetes my home lights, which were driven by a simple daemon, stopped working. Kubernetes was on fire, and it was hours of pasting random incantations to get it back up again. I ripped out Kubernetes, set it up in docker-compose, and it's been fine since.

I’m not sure I follow. Do you mean that you use containers but without Kubernetes? If so, that sounds like a wise choice. AFAICT, Kubernetes is meant for really massive-scale deployments, so it seems quite clear to me that using it for home use is overkill.

Yes, that was my conclusion as well.

A general pattern I see in software is that it scales up much better than it scales down, and Kubernetes is definitely like that. I had hopes it would be useful and educational to run for personal projects, but it's just too heavy.

Interesting observation. I think you’re on to something there.

Kubernetes is actually a set of interfaces. In this case, they are not using default Kubernetes implementation but a simplified one : k3s so the experience could be different. Or not. But it certainly can't be judge on wpietri experience as is.

What was the issue with Kubernetes?

The above question in a way is also my point which more bluntly can be said as, don't run something for the sake of running it and instead run tools you're knowledgeable in.

> Kubernetes was on fire

It might have been something as easy as conflicting IPs on your network that was causing the control plane not get kicked off or the nodes constantly disconnecting.

Something somewhat similar happened to me where my IoT lights caused a thundering hurd kind of problem on my network ip space related to some internet disconnect and I guess a bug in the code. They just kept conflicting IPs with each other and would just loop over the ip space they had. Never dealt with this before and just unplugging the whole switch for 5 mins resolved whatever infinite bug they had.

Honestly, Proxmox + TurnKeyLinux LXC for Docker + Caprover does everything I could ever want. If I need a VM, I'll just spin it up in Proxmox.

Also on Proxmox, but run separate LXCs for each docker container. Lets me do per-“application” QoS/monitoring as Proxmox is also running my firewall.

Am I missing out?

Offtopic question now that you mentioned running a Linux NAS... Which hardware are you using? Because I'm interested in building a home NAS but I don't seem to find any small box ready to hold several disk that isn't ridiculously expensive.

I have had Fractal Design's Node 304 as a home server / NAS for the last 3,5 years. It's quite reasonably priced, the provided fans are quiet enough and can be replaced later on with Noctuas. It doesn't offer disk hotswap but otherwise it's quiet and I can suffer the downtime ;-) + can handle 6 3,5 HDDs :)

https://www.fractal-design.com/products/cases/node/node-304/...

I'm running a Dell T140.

It's not the cheapest, but has idrac "remote control bios". So I never have to sit next to the box. It houses 4 disks + you can add SSDs via PC and it has ECC RAM. Plus I added a couple of years "Next business day on site" warranty, so I'm pretty safe against long downtimes.

I recently bought a Chenbro SR30169 for my backup storage. It’s super compact but also quite noisy.

Instead, I’d recommend getting a µATX tower with sound dampening preinstalled. The extra interior space also makes installation and maintenance a lot easier.

I used Helios 4 (32TB) and currently building a Helios 64 (80TB) NAS. It works great with armbian, which was the key for me.

For the past 7 years I run Ubuntu + ZFS with a Xeon E3 + ECC RAM + SuperMicro MoBo inside a NORCO 20-bay 4U chassis. This was a great setup for years since I needed 40TB+ and at the time of build 2TB drives were the best price per gb. Now, I’m moving back from a house to an apartment, so I need something much more compact and quiet and went with a TrueNAS Mini XL, since it’s one of the few tower-like form factors with enough hot-swap bays for me. It runs nice and quiet, and comes with 10GigE. TrueNAS (the OS) has been a pleasure to use too, but it’s obviously a bit more limited than a full Linux install. I now have it loaded with 10TB drives.

ZFS is the hero here. I tried many solutions in the past (mdadm on ext, btrfs, XFS, hardware RAID) and got burned by rotten bits and confusing UX leading to user error data loss. But in the past 7 years running ZFS I haven’t lost a single bit, and been continuously impressed with the incredibly easy to use CLI and “intelligence” of the file system itself. Snapshots, send/receive, datasets, everything is just so well refined. It felt ahead of its time a decade ago, and still does today. Moving the file system from Linux to the FreeBSB based TrueNAS was effortless.

I run FreeNAS (FreeBSD based) on HP Microservers. ECC RAM, 4 drive bays, ...

The older Gen8 boxes Just Work.

With FreeBSD, the newer Gen10 boxes would hang the first time you boot, you need to pause it and set the hw.pci.realloc_bars loader variable to 1. Haven't tried a new install recently. Details here: https://www.virten.net/2017/10/fix-for-freenas-on-hpe-micros...

Not “small”, but I vertically mounted a 4u chassis (front-side down so cool air comes in from the bottom and hot air is exhausted out the top) and it actually feels not much bigger than a standard ATX tower, except I have 9 HD bays.

The case was $120CAD, and the mobo/cpu/ram (i3) were another maybe 300.

Caveat: I’m not running ZFS, and just do classic “data lives in two places on-site and one off-site” backup

Reply on news.ycombinator.com