Show HN: HomelabOS – Ansible scripts to deploy self hosted cloud services
gitlab.com
gitlab.com
For me personally, I’m happy to cede some small degree of control and whatever else to a good quality third party provider who has a team that is paid to actively maintain and secure the product, leaving me to spend more of my limited free time with my family rather than debug why the letsencrypt certificate for my self hosted mail server/VPN/cloud store/whatever hasn’t auto-renewed correctly etc.
I’m sure there will be plenty of “i’ve ran my own mailserver since the 90s and nothing has ever broken” wizards here, but everything works until such time as it doesn’t. I’d be curious to hear from people who have had trouble.
The biggest pain point is installation. Most selfhosted software has a ton of dependencies to install first. And after that there's usually some configuration that has to be done before it will work. A complicated installation is enough to drive even tech savy users away.
I've had some luck with my own software by targeting Windows users as well. Most people don't want to setup a linux box just to selfhost a single piece of software.
The real wins from Docker (for this use case) are:
1. Docker is a better process supervisor than systemd and friends
2. Simple, fast deployment (no manage ansible scripts or rebuild/reboot a machine image)
3. Built-in, standard logging
1. Spend maybe 2-3 full days install kubernetes, helm
2. Spend maybe 3-4 hours installing your app through helm because you're new to installing things in kubernetes.
3. The next app that you want to install on your server is only 20 minutes away, now that you understand how kube and helm work.
Scenario #2:
1. Install app directly on server, hunt down dependencies and other weird things, probably takes 1 day at least, to do the whole installation.
2. The next app that you want to install will take the same amount of time again.
I'd go with Scenario #1 as it is more scalable if I want to install more apps on my server.
I've found using docker-compose is a nice way to do basic orchestration for "self-hosted" type apps.
What distro are you running? Either I am very spoilt with Arch (+AUR) or this is way off the mark.
[1] https://kubernetes.io/docs/setup/independent/create-cluster-...
Initial config surely varies from wall of text config to a simple web based config but popular apps usually have decent doc or googleability to keep you out of maze easily.
Invariably you end up having to add GPG keys for a bunch of weird and wonderful 3rd party package sites, then fight with your package manager to get it to use them, spend hours scouring the web for source code tarballs, spend an age fiddling with compiler flags to get things to build, and... ach! Eventually you're bound to wish you'd never started in the first place!
Also, compiling software is quite normal on Linux. Often it's just `make && sudo make install`, and everything works - but sometimes you've got to go on a goose chase to fulfill dependencies, tinker with compiler flags etc.
It's very normal on Arch Linux at least to pull build files from the Arch User Repository and build the software locally.
I use an aws lambda running flask with a password to expose a website where I can IP whitelist any other servers to the requesting IP, it sends me an email when an IP is added and I periodically clear them all (try to do when leaving if on public wifi).
It's a kind of very low-tech but I think very decent extra first layer of security, none of my servers are visible to anyone most of the time but I can hit them whenever.
some things that can trip you: * for the ip there's a stupid mistake I made, when testing locally you can request.get(myip) to get which ip to use, remotely you pull it off of request.remote_addr * your lambda will need a specialised IAM role to have rights to do all this, it's configurable in zappa config and I just clicked creating one through aws management console * this is in the zappa docs but I didn't rtfm so I buil everything using conda which doesn't work, then I built it with a virtualenv with the same name as the project which doesn't work.
Also, I don't expose any of it directly to the internet.
IMHO Docker has been a godsend for deploying self-hosted software.
I haven't tried it yet, but watchtower (https://github.com/v2tec/watchtower) can be used to automatically update running Docker containers.
This is also a set of Ansible playbooks and roles that use a Debian Stable net install as a base and let you pick and choose the services you want to configure. It supports multi-host deployments, provides an internal CA by default and is designed to be used as a base for your own applications - you are encouraged to use DebOps roles in your own playbooks to integrate with existing service configuration like firewall, web server, etc.
You can check out the list of Ansible roles available in the project [1], there's also a Getting Started guide [2]. Project has its own IRC channel, #debops on FreeNode, as well as a mailing list [3], if you need support.
[1]: https://docs.debops.org/en/master/ansible/role-index.html
[2]: https://docs.debops.org/en/master/debops-playbooks/guides/ge...
I've backported packages for myself a few times, and in the other cases I'll install binaries beneath `/opt`, where they'll be out of the way.
Given that the name is "HomelabOS", I'm sure that this is aimed at people who are doing this because they find it fun and/or to learn new technologies, not people who firmly believe this is the most efficient way to run production software.
Including these fabled "cloud services" that everyone believes have no down time..... yet I get alerts monthly from these various services about unplanned outages of this service or that service for the various "cloud" products my company uses...
Services from major companies like Microsoft who has all kinds of issues with their Office 365 Product, and in addition to the endless Windows 10 and office Activation server issues...
While it looks good on the marketing brochure to have "good quality third party provider who has a team that is paid to actively maintain and secure the product," reality is often very very different.
I prefer the control, and responsibility. Others prefer to loose control and have the ability to "blame the vendor".
The key to avoid the burden is to restrict the scope of your services to what you actually need.
Let me elaborate.
I've been hosting my own email (plus something else) at home for the last five years or so. So far, so good: various services signup, job-related email exchanges, no problems.
I rarely touch my home server and I mostly do software upgrades. I run postfix, and when configuring virtual users on virtual domains, I invested an afternoon to actually go trough the documentation and came up with a reasonably simple configuration where all the data about users and domains is stored on simple text files: no SQL database to manage (not even SQLite).
Such a solution is easy to back up and easy to interact with.
For more adventurous testing, I just have another machine.
That's true. I was running email, personal web server, and VPN with soverign and that worked really weell for years until it didn't.
Still, that is a long time of pretty functional work.
At this point, I know enough professionally to run my stuff just fine, so I do. But seriously, although I will never get the 3-5 hours of time that it took me to debug my mailserver setup back, that was spent almost 5 years ago and I haven't had to touch it since, really.
A lot of this stuff really does run just fine once you have it setup. I didn't find the maintenance burden all that high even when I knew less about these systems than I do now.
I am often surprised about how often docker images are updated and auto-spun up in my home environment.
Also, this assumes that the container run arguments for the container don’t change over time. Good container maintainers are sensible about this, but you are still leaving your uptime to the chance some stranger cocks something up.
for certain homelab style applications this might even be an annoyance - if I’m in the middle of a film I probably don’t want the Plex container updating itself.
It did take me quite a bit of time to set up though, and understand what I was doing. I would be incredibly hesitant to run what was in the link, as I have no understanding of how it works, security issues, etc.
FreeNAS also allows for VMs, so I can install VMs on my freeNAS and allow then the same thing (I use regular ext4 for the VMs, as they are mounted on a zblock and get snapshotting). Right now they are all encrypted, so if something does restarts, I have to physically be there to decrypt. I think i can use dropbear SSH to prevent this, but I am okay with this limitation for now.
If you have specific questions please let me know.
I run a similar project called Ansible-NAS, which was borne out of FreeNAS being a pain in the ass to manage and upgrade. https://github.com/DaveStephens/ansible-nas
That said, for transient development purposes it's perfect. As an on-prem replica of object storage/S3, it's tantalizingly close but not quite there. Maybe it's improved in the last year or so since I last used it...
I'm going to try Zenko next.
A couple of questions: - Can I deploy this to a Digital Ocean droplet or similar? (I am assuming it's the case, but just checking). - There's openvpn, and there is pi-hole. Can I assume that if I connect a device to the VPN, I'll also get ad blocking via pi-hole as a bonus, or do I have to edit my DNS servers on the device separately.
A couple of software suggestions: - I'd love to see Wireguard instead of openvpn. The setup/speed is just amazing. - I'd love to have Matrix (https://matrix.org/blog/home/) as a messaging option
Pi-hole out of the box support is a bit wonky at the moment, I've been working on it, but it's not quite to the point you described just yet. Contributions encouraged!
Those both sound great to me, and again, Merge Requests are highly encouraged. :)
In general though it looks like they are taking a very different approach to deployments as a whole. They describe some of those differences here: https://sandstorm.io/news/2014-08-19-why-not-run-docker-apps
While I won't get into the specifics of the pros and cons of each of their bullet points, I will say HomelabOS arose (as some of the other commenters have pointed out) as a way for people interested in this sort of thing to experiment with it. Sandstorm looks more geared toward being usable by 'anyone', which is an admirable goal, if perhaps a bit ambitious in my mind.
Additionally the plan is to move to Kubernetes soon for multiple node deployment, and that wouldn’t really be possible without Docker.
And to be clear, some software is installed directly by Ansible, where it makes sense to do so.
Regarding multiple users, it really just sets up an admin user for most services. Any multiple user support is then up to each service individually. But if you're asking if it does automated separate instances of services for different users, then no, it does not do that.
and/or implement/deploy simple personal web services (rss reader, wiki, maybe even webmail?) on top of apigw/lambda+other services as necessary.