Hacker News Reader

Comment by sparrc | Hacker News Reader

Decent summary but a little out-dated on DNS load balancing.

Major cloud services like AWS support health/status checks through DNS these days: https://docs.aws.amazon.com/Route53/latest/DeveloperGuide/re...

It's also trivial to get around the client caching issue, just set a low TTL. Perhaps in the olden days providers had stricter limits on the minimum TTL you can set, but these days you can set it practically as low as you want.

EDIT: as a few commenters have fairly pointed out, TTL can easily be ignored by poorly-behaved ISPs and clients, so I'll admit calling it "trivial" to get around is not exactly accurate.

View on HN

Having executed several "no-downtime" cutovers between systems via DNS updates, I will warn you that a surprising number of clients never re-resolve DNS, so the TTL is effectively "forever" from their point of view.

For the rare case of lift-and-shift-ing for a system upgrade I felt morally okay about eventually pulling the plug on them, but I'd hesitate to design a system that relied on well-behaved DNS clients if I had a reasonable alternative.

Another gotcha would be UDP based services. Since it is packet oriented and not connection oriented, when should it re-resolve? Most will not until the application is restarted.

When I last updated a domain most clients saw the change within the TTL (1 hour)... except for my cable ISP at home. It took them the better part of a week.

Moving by DNS change isn't usually that bad. The old system (load balancer) can proxy requests to the new system. Most clients will follow DNS but the laggers won't have too much trouble. Assuming the service already works behind a load balancer of course, that is usually not something than can be fork-lifted in.

Except it’s not trivial at all because isp resolver will just disregard your low ttl

TTL is difficult in practice due to client implementations and other issues like that. Be careful using DNS anything. DNS was not designed to immediately resolve anything. That's why IPs are mostly used.

Many applications do not refresh their DNS with every connection either. Take for example an Apache reverse proxy that's reusing long lived connections. So updating DNS may still require restarting/reloading many upstream services.

https://stackoverflow.com/questions/52032150/apache-force-dn...

I know about the caching issue is a little trivial but it was worth mentioning. Though I should have mentioned the low TTL piece. I will add that to the post. And also will add the health check part too. Reading up a bit about it. Thanks for the information!

Reply on news.ycombinator.com