- I have to manually check the zpool periodically to see if there are faults. I went about 30d with a DEGRADED disk before I realized it, turns out it was a bad cable but if another disk failed I would have been hosed.
- I have to manually run zpool scrub (yeah I know about cron but chances are if I just add the cron entry it will fail, probably some permission thing or a 1-character bug in the cmd line. Nothing like this is ever easy, it's a 1-hour rodeo, at minimum. And yeah, I'm lazy too, not gonna deny it)
- I can only get NFS v3 working, not v4, no idea why. IIRC v4 isn't technically supported by either FreeBSD yet. But rather than just say "not supported" it gives you some cryptic bs that you waste minimum 20m on googling and trying things. Everything you try, you have to triple check: restart rpciod? Or was it nfsd? exportfs -ra? Or was it some encryption version in /etc/exports?
- Samba kinda works, no idea how the permissions are supposed to work, but turns out SMB is more resilient to network interrupts than NFS which just hangs the client indefinitely. So for listening to music or reading a book its fine.
- I wanted to set up VPN on a 2nd network (not VLAN, just a 2nd subnet that routes out the tun device. I thought since I control DHCP, I could force certain devices to only ever have access via VPN), turns out this isn't possible or "very hard" doing some hack to the NAT pf rules, way above my skill level. I wasted hours on this, got the VPN and tun to authenticate, but got stuck at the routing layer. Oh also I think dhcpd doesn't like routing to different adapters? Can't remember.
- PXE was a disaster, as it usually was. I got it barely working enough to boot my tape backup computer off it and run backup scripts.
- I want simple logfile aggregation. I dont want to have to ssh into the NAS and check the logs/dmesg. I admit I haven't done a lot of research (10 years ago there was nagios, thats all I know), but it bugs me profusely that I cannot easily monitor the logs of all the machines on my network, and I rarely ever hear about anyone else doing it.
so probably 40% success, 60% (frustration|half-assery|disappointment). I guess it's just masochism, like the weightlifters who love the pain of adding 10 extra pounds to their set. At least they get some results eventually. :/
Overall, I don't regret doing it the hard way, but I am often times envious of the Proxmox users who (presumably) just plug and play.