Running an Open Source Home Area Network
xn--gckvb8fzb.com
xn--gckvb8fzb.com
It's interesting to see someone using a single board computer as their main computer, and preferring a Linux distro that compiles everything from source.
Why bother with docker for a home server other than for the fun of it?
I've been embracing systemd recently, and using it to manage processes in an embedded system. Once you wrap your head around how services work, it seems pretty convenient compared to init scripts. Notably, systemd works with legacy init scripts, so you could still go that route if you wanted to. Maybe throwing docker into the mix makes it clunkier, though?
Maybe you disagree with those principles. Maybe you disagree with software having hard principles.
That's fine. NixOS isn't for you. It's wonderful for me though!
I literally cannot bork my machine. I can always roll back. Useful when messing with low-level stuff like drivers.
I have full control and understanding of updates and the exact versions of all the programs I run. I can always read the source of any program I'm running by looking in the nix store.
Super easy to package new programs and contribute to nixpkgs.
Super easy to tweak packaging other provided without forking via overlays.
I share config across devices in a git repo, so provisioning a new machine is trivial. I got a desktop and I basically imported a bunch of common config along with like 10 lines of config specific to the desktop.
So use a templated VM, or a filesystem with easy snapshotting like ZFS or BTRFS.
The former also allows for easy provisioning of new machines, and doesn't lock you down to a single OS.
And that doesn't have all the other benefits. Also it's a worse way to handle state and rollbacks. Sledgehammer for a thumbtack.
I already spent the time to learn the hard stuff. So now my life is made easy by NixOS. Same idea behind learning Haskell. Huge personal gains.
ZFS and BTRFS are orthogonal and often used in conjunction with Nix.
> Are templated VMs contained within a few kilobytes of confif [sic] files?
Not kilobytes, but Proxmox supports sparse images so pretty small. More importantly, disk space is cheap as hell, and I value my time way more than a couple of hundred megs of space.
> Can anything in the entire system be included or excluding by tweaking these text files?
Personally I template my images with Ansible, so yes in fact anything can be included or excluded with a text file.
> Can it be used as a desktop on bare metal?
Who cares? The performance hit from a modern Type 1 hypervisor is so small as to only matter if you're also the kind of person who is tweaking obscure CFLAGS for emerge, which is to say it doesn't matter.
> ZFS and BTRFS are orthogonal
Only in that they aren't an OS, obviously, but they perform the same function (rollback) that you mentioned as a positive point for NixOS.
I want to use it as my personal computer...using a VM seems even more fringe and niche than NixOS there lol. And for a home network, VM also seems overkill.
I suppose there are also network effects at play. If you use NixOS for a laptop and desktop, suddenly using it for home infra is actually more economical than using other tools.
I also get a sense they are for some reason solely focused on cattle server use-cases. I'd say the OP (home infra) is in between your production servers and your PC.
Conversely, I can lose a k8s node and have nothing change. If I lose my NAS (separate node, separate Proxmox cluster), I'd have to boot up the backup (which boots daily to sync, then shuts down) and run an Ansible play to change its IP address so that all the NFS targets still worked. I could make that more automated, I suppose, but it's an unlikely scenario so I'm fine with the small amount of manual labor.
I guess my point is that I don't see the benefit in having a special OS for daily use. If I want to fiddle around and possibly break things, I don't want to be doing that on the device I use daily. I used Gentoo for years in the early 2000s, and no longer have the time or patience for my main computer breaking constantly. If I want to play with something, I spin up a VM. If I want to play with something baremetal, I have an old Dell T310 I can use, and a couple of ancient Macbooks somewhere.
I literally cannot be paid to care about borking my machine. It takes ten minutes to reinstall.
For stuff that cannot conveniently be installed locally (hello multiple versions of DaVinci Resolve) or ideally want to be ephemeral (hello basically all the development environments I use), I've got Docker.
A- If you want to be able to restore to any point after installing your system: LVM/ZFS/BTRFS snapshots got you covered.
B1- If you want to install your system the way you want, you have Kickstart (RedHat family) and Preseed (Debian family). You can provide an installation template to the installers, and they install your system the way you want.
You want to get your /etc/ afterwards the installation? Either provide it on a disk, or your local network, or anywhere on the internet. Either integrate it to your Kickstart/Preseed end hooks, or do it after your first boot.
B2- If you want something more fancy? Create a FAI installation media which installs the system you want.
C- You want to transform an existing Debian system to something you want? You get the dpkg package selection state and set to the target system, apply the state so the system is converted to your own selections. Apply the /etc via git or any way you want.
D- Want something more programmatic? Run an Ansible playbook locally (This is how GitLab installs and configures itself during install).
E- For fleet installations there are XCAT and network variants of B1, but they are out of scope for personal systems.
We use E in our system room. I used B1 for personal systems and B2 to deploy an installation country-wide via USB sticks. I used C in the same project for a small subset of servers. In these cases, all systems are operational in the first boot, starting from a known state.
I know a lot of people using A, and nudging me to try/use it. GitLab uses D every time I upgrade it.
I really don't see myself using another Linux distro. It would make more sense to just modify my NixOS config. Don't know what another distro would even buy me tbh.
I do this. Over time you forget how each service was configured, or simply don't care. Adding more and more stuff to a home server increases the complexity and the attack surface more than linearly in the number of services.
I run nearly all my home services in docker, and I have a cookie-cutter approach for generating SSL certs and nginx config for SSL termination (not dockerized). Provisioning is automated through ansible, so my machines can be cattle not pets, as far as is possible on 3 raspberry pis.
Running all my services in Docker keeps it all clean because I'm a very messy person when it comes to Linux. Change, change, change, it works, forget about it, it breaks, find something I did years ago tripping me up now, change, change, change, it works, forget about it.
With docker every service is contained and nearly separated. I can rip something out and replace it like stacking a new network switch in the rack. Delete the container, delete the image(s) and delete the volume if I want to start over with something completely fresh.
I can move everything to a new server by moving bulk hard drives over, restoring docker volumes from backup and cloning docker-compose configs from git. Haven't tried any distributed volume storage yet.
Having tried Gluster, Ceph/Rook, and Longhorn, I strongly recommend Longhorn. Gluster is kinda clunky to setup but works, albeit with very little built-in observability. Ceph on its own also works but has some fairly intense hardware requirements. Ceph with Rook is a nightmare. Longhorn works great (as long as you use ext4 for the underlying filesystem), has good observability, is easy to install and uninstall, and has lower hardware requirements.
Its main drawback is it only supports replication, not erasure coding, which tbf is a large contributor to its ease of use and lower hardware requirements.
Because it legitimately makes things easier. Your source of truth for how an app was set up is the compose file, and as a result you only need to care about the storage each app uses for its state. This makes upgrades and back ups almost entirely painless and drama free.
Deploying the average app the old way feels downright medieval.
Correct. It increases the attack surface without adding benefits over light containers like systemd-nspawn or just systemd unit files.
As others have alluded to, mostly for having everything in code. I went from Docker --> Docker-Compose --> Kubernetes. The latter is 100% overkill, and was mostly done to assist learning. Still, it's very nice to have as close to HA home services as possible (with the exception of a dedicated failover WAN connection, and me needing to manually fire up a generator for power - UPS will last ~15-20 minutes).
Scheduling aside, though, yes you could accomplish all of that with systemd and something like keepalived.
Signal's creator has reasonable concerns about Web3 and NFTs? Better drop it, since "[Moxie has] very disturbing opinions on decentralization, [and] cryptocurrency..."
6+ paragraphs written about the print quality on various deskmats.
Tables of every material possession they own, seemingly lacking only the UPC.
The site never claimed to be otherwise, so there's that.
Sadly I was disappointed. It's just another of these "digital gardens" or whatever the term for the type of sites is, that one can find across various platforms like neocities these days.
Not sure what the described homelab has to do with crypto or decentralization, though.
The posted website is the second time I've seen this, but I have to imagine it is a specific style or practice.
I was confused and starting navigating away from the page as the start of the article was below my viewport on my macbook with default resolution.
(this is not supposed to be an advertisement, just a question as those are on my radar if I ever buy a router to plug on my ISP cable modem)
I switched to a Ubiquiti Edge Router and went the "discrete devices" route, one device for one purpose.
The edge router lasted 5 years or so before some power component blew, and with the sorry state of updates from Ubiquiti these days, I decided to move on. When I was looking around Turris came up a lot but it goes against my "discrete devices" approach so I bought a Protectli i5 box and run pfSense.
Ideally I'd like to get to the point where my Ubiquiti Unifi APs can be replaced with generic but capable hardware running Linux or BSD too for WiFi 6 speeds (not sure that's currently possible/available) and Switches are likely a way off.
When I open the link in iOS Safari I see the unicode characters.
I do kind of like that HN retains the xn-whatever as it's actually formatted in an authoritative DNS zonefile, because this is hacker news. As I do not read the language in question the unicode original and the xn-whatever are equally incomprehensible to me.
What software libraries (or algorithm) exist to tell that 2 UTF-8 strings don't have homoglyphs?
>Since the Panda PAU09 N600 works out of the box, the only thing needed to enable the WiFi access point is a working hostapd configuration: [75+ lines of config for something that "works out of the box"]
>One pitfall here is that Docker likes to mess around with iptables, leading to WiFi clients being unable to communicate with the internal network. In order to fix this, another drop-in configuration is needed:
>Update: Because I simply couldn’t get it working with this rule, that was suggested in the Docker docs in first place, I eventually gave up and used a jackhammer to fix it: [...]
I see wifi is still a shitshow in the libre world. Running entirely free software is a laudable goal but the above sums up why it's never going to be mainstream. All that fuckery for previous generation .11ac?
No thanks, I have a zillion other more important things to do with my time.
Meanwhile, I run my own Linux router on a Raspberry Pi. It covers all the wacky edge case needs I have such as Wireguard, VRFs, VLAN, etc.
Out of bos without tuning, it's more like 750mbps maximum throughput. Most of the optimizations are pinning IRQs and RX buffers to particular CPU cores, otherwise it'll max out one core and limit the throughput.
I'm routing and forwarding, as well as stateful firewalling with nftables.
Do you have any suggestions / personal experience with systems like this? I'm really concerned about getting something that isn't powerful enough to process stuff at gigabit+ rates, and most of the "routers" I've seen get pricey to do that.
As the other commenter said though - max speed is 1gbps. And approx 200mbit/s over wifi with 5ghz and the same network card I used.
This is an annoyingly unfilled product space. I keep on wanting and the market keeps failing to provide something small, cheap, and with 2 gige ports. Heck, I don't even need x86_64, as long as there's a well supported Debian port.
Anyone have suggestions for even two gigabit ports and the lowest possible price?
Also, I'm curious if anyone has concrete examples of advantages of anything other than Debian for my router. I want to know if I'm missing out on cool features or some such :)
ARM-based devices are not supported by pfSense / opnSense so you should stay with x86_64 based devices.
I was considering a QOTOM Q355G4 until I decided I didn't need 4 NICs. They're $300-400ish, depending on configuration.
So the hardware is there, it's just from companies I'd never heard of.
The disadvantage is that everything is built from source, so it takes forever to install anything and it's very fragile, and it's a rolling release distro so you're installing stuff every single day and it's very fragile.
I stopped using Gentoo and Arch because it was just easier to be on the same OS as other devs and it didn't feel like I lost much switching to Ubuntu. Every now and again I'm tempted to go back to Arch, but honestly, there are other things I'd like to play with rather than my OS
I have seen similar sites (like https://vuxu.org/vuxi/boxes by Leah Neukirchen) years ago, but none was that well documented and verbose.
Great stuff.