Hacker News Reader

I think the whole "self hosting isn't easy" meme gets repeated so much that people just take it as given now and default to managed software. Or, someone might argue "Well, my grandmother who knows nothing about tech cannot self-host, so it's not viable!" ignoring there is a huge spectrum of competence between grandma and a seasoned Linux sysadmin. People aren't morons, and there's enough info out there on how to do it. I agree it's not organized very well, but it's not like setting up a web server is dark wizardry.

With all the tools out there and easy access to VPS services and even bare metal for your basement, there's never been a better time to self host. And not just web servers, but E-mail, git, photos and media, and so on, it's very accessible.

I agree it's overblown. It's amazing how robust of a setup (more than sufficient for residential use!) you can get with little effort given how easy things are nowadays.

I've been self-hosting a lot of load-bearing household stuff (I have stuff on the "wife-critical" path: if it goes down, "the internet goes down" and I get a text from her) for almost 10 years and I've only had 2 incidents of particular reputational-risk note:

1) a routine reboot of the main server triggered a BTRFS bug that blocked mounting it again. This took an evening and a reboot into an arch linux ISO to fix (arch had a new-enough version of the btrfs tools that had the ability to fsck/repair the fs).

2) my proxmox setup was initially installed with zfs and zfs-on-root. This exploded and the "on root" part stopped working one day. This was the most annoying thing to fix so far because I ended up dumping any interesting data to an external HDD and just re-paving the server, this time reinstalling with just ext4 and lvm (which is admittedly a setup I'm much more comfortable debugging). No issues since then.

Both these events are from over 3 years ago, so it's been smooth sailing in recent times.

The complaint is fair though. Trying to find a complete or the "correct" guide to something is very difficult even when you already know roughly what you are doing.

I took me ages to work out how to setup postfix properly from about 10 slightly different "guides". The Postfix book wasn't even that helpful. There are also lots of very out-of-date guides that might have been OK for 2015 but not anymore. They don't get deleted because "link juice"

It is sad but true but you get one little bit wrong and you potentially leave a door wide-open.

Postfix is a special kind of hell though, in that getting a good setup requires wading though decades of legacy stuff and patching together a bunch of non-default stuff to get, for instance, dkim signing and stuff working right. I've done this before myself, and agree it was super annoying and not fun, but I also think it is potentially the biggest outlier in self-hosting difficulty I've encountered.

Lots of services are barely more than - apt install, systemctl enable --now, ufw allow 8080 (if you even firewall within your network).

I actually found Postfix fairly easy to configure once you have a solid understanding of Email (which took me a good while at first). Dovecot on the other hand...

Majority of the documentations I came across usually have the mantra of "Do this and you are golden". I know it is not dark wizardary, it just the documentations are aiming for someone who have the experience and the technical knowledge of this. Whereas there are people who are pushing "self-hosting is the answer! Even your tech-inept grandma can do it!" without providing documentations for inexperienced people like me. Annoyingly that some guides have parts that have a links to other guides that barely provide information about this. It is like "I know how to set it up but I am not gotta tell you how to do it, so here the link that might help" and it didn't help at all.

When I have begun to install and manage servers, more than 20 years ago, I did not have any kind of prior experience and I did not have anyone whom I could ask.

So I have just read the handbook, but I have read it completely, which needs more than a day.

It is likely that there are also other operating systems and Linux distributions that have good documentation, but I can testify only about those that I have used in the beginning, the FreeBSD handbook and then the Gentoo Linux handbook.

Both handbooks were good enough to convert anyone into a system administrator.

Unfortunately, both handbooks are not as good in 2022 as they were e.g. in 2002, because they have not always been updated after every change, or the updates have not been as detailed as the original parts of the handbooks.

Even so, both handbooks remain reasonably good today.

Especially the FreeBSD handbook is good for someone who lacks experience, because FreeBSD is much more self-contained, i.e. there are a lot of choices that have already been made for you and you do not have to worry about them.

So for someone who is inexperienced, I believe that the fastest way to managing a server remains to read the complete FreeBSD handbook and install and configure a server based on that.

There are programs which are available only on Linux, but the administration of a Linux server requires much more work than for a FreeBSD server (even if much less than for a Windows server), so for a beginner I think that FreeBSD with its more complete documentation and less possible choices is easier to try.

I'm skeptical that your layperson would be able to keep self-hosted applications secure constantly. Hell, huge corporations have a difficult time with it.

I have this issue too. When I tried to set up self-hosting, I assumed that there are steps that requires me to expose it to the internet. Turn out that it already exposed and didn't (or barely) provided the information of how to close it off securely and keep it private network only. When I tried to find information about it, there was always guides that are not consistent with it. Some will say I have to go in php.ini to do this, then go to SQlite to do that, then go to other files do there, then adding 20 steps to keep it secured. I'm just wondering why there are not any centralized options to do this. I just want a option that I can tick in the software and left it off as that.

I understand those documentations are not for laypeople for me. However it is annoying when people out there kept pushing the self-hosting for beginners narrative without providing the necessary tools for laypeople to keep themselves secured and reliable.

>I understand those documentations are not for laypeople for me. However it is annoying when people out there kept pushing the self-hosting for beginners narrative without providing the necessary tools for laypeople to keep themselves secured and reliable.

And that, in a nutshell, is the problem.

A few clicks, a configuration form and integrated tools to set up external dependencies (i.e., LetsEncrypt certs), et voila! You're running a self-hosted application.

AFAICT, this is more about developers not creating the packaging/configuration/management tools necessary for effective use by non-technical users.

Sure, I can write a sql query to modify the schema of an applications' database, but my highly educated and intelligent physician brother would just throw up his hands in disgust.

Make self hosting easy and people will use it. And Docker-compose isn't "easy" for a lay person.

It sounds like part of the difficulty has to do with the general poor quality of online tutorials. There is a need for properly written guide books and magazines, but unfortunately, it seems like there is no way to pay for people to write them.

> My biggest beef with self-hosting is that they expect us to set up the SSL/TLS certificate without explaining the step to set it up. Some guides does have section about it but never provide the details about creating CA for my self-hosting needs. I turn to Google/DDG to find information about it and they are all over the place or leading into dead-end.

If you have your own domain pointed at your server, the Let's Encrypt certbot can automatically pull in a certificate and configure your apache/nginx webserver (alternative webserver caddy has this feature built in as far as I know).

If you don't have your own domain, don't go with self-signed certificates. Get a free https://desec.io/ subdomain, and they have their own certbot plugin to generate automatic certificates.

> If you have your own domain pointed at your server, the Let's Encrypt certbot can automatically pull in a certificate

Yeah, but don't have a mistake too many times, or Let's Encrypt will block you for a week until your rate limit times out.

I hit this. I understand why Let's Encrypt has to do this, but it's very annoying and you have no choice but to do nothing for a week.

There needs to be something in between Let's Encrypt (free) and a couple thousand a year (other CAs).

If you use Caddy, you'll almost never run into rate limits from Let's Encrypt, because Caddy rate limits itself, and will fallback to ZeroSSL instead of Let's Encrypt, and even fallback to LE's staging for additional retries against LE before trying the live one again if it works with staging. See https://caddyserver.com/docs/automatic-https#errors

Use the LetsEncrypt staging server for testing. When you have a process that works, switch to prod.

That's a tautology saying "Don't make mistakes."

A DNS misconfiguration can cause your Let's Encrypt to do weird things on a configuration that was (and still is) perfectly correct.

That was how I hit it. I eventually figured out what people screwed up in DNS. But certificates still didn't clear. So I spent an extra couple hours staring at DNS trying to figure out what I missed when the issue was that we bumped into the rate limit at Let's Encrypt (which is REALLY low--I think 5 failures is enough to trip it) while the DNS was bad and the only thing we could do was sit around for a week with dead certificates.

Not fun.

Sorry, quick comment, didn't mean to be glib.

I've hit the problem you describe, and I feel your pain. I also respect LetsEncrypt's choice to rate limit failures. I renew a couple dozen domains at a time, so one error can quickly cascade into being blocked. IIRC the block timeout starts at 24 hrs and goes up from there if you keep trying -- this is easy to do if you don't see the raw response error message!

After being bitten by this a couple times, I added a dry-run step to my autorenewal script. If the dry-run exits with success and generates a good new cert for the domain, I repeat by pointing to the LE prod server. This works every time (so far, but for years now).

I'm suggesting that any LetsEncrypt certificate automation system (or docs) targeted at relatively low-sophistication users (i.e. not you or me) should include this sort of dry-run check so that the user doesn't paint themselves into a corner with a somewhat persnickety, but essential, service.

Also of course, it should attempt to renew after 60 days, so that if things go badly wrong, there are a few block-timeout retries available before the 90 day expiration.

To do this right you should also think of backups, updates, and monitoring. Self-hosting is true freedom but doing it right for things like email is akin to running a small business. On the positive side docker makes many things a breeze.

I tried with Docker before and it is not a breeze as you think it is. I tried to use Docker for Calibre-Web and it is a pain to make it work. Because Calibre-Web required to access their database in the filesystem outside of Docker. Docker provided minimal (more of lacking) information of how to expose the filesystem for Calibre-Web to use their database. Calibre-Web cannot create their own database, it relies on Calibre, standalone app, to generate the library that it need to have access to. It took me ages to finally to find a way to expose the filesystem and only provide permission to access that particular library.

I am surprised by this shortcoming of Calibre image. I guess the trade is learning how to install calibre vs. learning how to deal with docker. I'd also agree that even if you use docker and installation is easy - for any self hosted apps you are using for a long enough time you end up learning enough about them to be able to install without docker (and avoid managing docker in addition to everything else).

Calibre does have their "book sharing" solution that are built in the software. However it is more like content server. Calibre-Web is a third-party solution that are not affiliated with Kovid Goyal (creator/main developer of calibre). And Calibre-Web is basically browser version of Calibre without requiring other people to use Calibre to access Calibre-Web. So Kovid did not create this calibre-web image from what I understand.

This was my single biggest hurdle when I was trying to set up a personal VPN to remote manage things- I didn't really understand what I was supposed to do, or why things just never quite worked.

If you find self hosting too annoying you could always try Yunohost to have one click deploys for the most common services.

https://yunohost.org