So by the time your browser requests an API call to a Hetzner server, there is a 100~170ms overhead. Compare that to a hosting provider that has datacenters in US West and Asia
The hardware used is consumer hardware too.
This is literally the opposite of what I would expect for a SaaS with uptime requirement... I feel like OP/the developers behind this might have made a really terrible choice and could be paying more in manpower than the savings they have by using a budget server discounter
Don't get me wrong, I have nothing against Hetzner, but this seems like an unfit usecase. This is literally what you use cloud platforms for..
really puzzling that a healthcheck provider itself is not accounting for this potential issue.
Is there some sort of edgy reverse proxy here? Doesn't seem like it. Even then there would be a big latency.
We haven't been using them for anything in production, so I am not 100% sure - they have a backbone outage right now however..
There are no edge / reverse proxies being described in the OP article, just a HAProxy configuration within the same DC.
They also do not have adequate protection systems in place
This caused major problems for Healthchecks, as I'm using Wireguard for private networking, and Wireguard uses UDP. Here's a postmortem about this incident: https://status.healthchecks.io/en/incidents/dcbcDNd89LptHoWi...
Hetzner also offers server-grade hardware btw. But if your setup is fully redundant anyway it doesn't really matter anymore if it's consumer grade or not (which the setup described in the article is).
1) runs single-DC 2) uses consumer hardware
I agree on the notion that with redundancy it is not as important what kind of hardware is used, but it still is something to consider (seeing how this is a one-person venture and server outages are still a huge time-sink), and I wouldn't even necessarily speak about true redundancy in this case
[0] https://docs.hetzner.com/general/others/data-centers-and-con...
Not going to argue about the shared fate, as I've seen more outages than not affecting whole locations..
even better if they offer Singapore & Tokyo.
I run dedicated servers at Hetzner and the consumer parts are CPU and motherboard. Memory is ECC, NVMe SSDs are data center grade. How often do you see CPU failure?
Over the same period, I think there's been two outages of AWS us-east-1 and a multi-region GCP outage. Plus weekly outages of a grownup unicorn SaaS like github or slack.
It's a tradeoff: the simplicity of such a setup reduces outage risk, and the costs saved can be spent on monitoring tools or extra redundancy.
His stack already uses multiple appservers (so the loss of one wouldn't even be noticeable) and has a manual way of failing over to a DB replica.
It's often cheaper to work around node failures (which you often need to do for other reasons anyway - DC-wide outage, etc) than have more reliable nodes.