Hacker News Reader

Comment by zenlikethat | Hacker News Reader

I think this is why some techniques for doing rolling deploys set up a health check for the node which is configurable by the deploy process. That way, otherwise healthy nodes report their status as "down" while still accepting and finishing requests, but then get taken out of the HAProxy rotation without having to re-load the configuration. The new code is rolled out on that node, and the toggle is set to report "healthy" again, so HAProxy brings the node back into rotation.

View on HN

This is what I've done in the past, not sure why we need to "hack TCP" to get zero downtime.

As an additional thing I do rolling deploys where I just create an entirely new VM and add it before removing the old one. This just means that I don't have to recycle a node that might potentially have some state on it.

The article is about restarting HAProxy without any downtime. HAProxy restarts are needed when adding new service instances or adjusting configuration options. This is a different and much harder problem than gracefully restarting load balanced service instances.

As other posters have mentioned the solution in this link is actually suboptimal. I showed how the iptables solution leads to 1-3s latency for connections that establish during the restart but you can avoid it by getting a little more hardcore.

See http://engineeringblog.yelp.com/2015/04/true-zero-downtime-h... or http://inside.unbounce.com/product-dev/haproxy-reloads/ for solutions that are even better than this one.

Why shouldn't HAProxy have an API for updating its configuration while running?

It should, and it allows you to do some changes over a socket, but allowing complete updates with the current haproxy architecture is a lot of work and isn't there yet.

That's the approach we take, documented at http://www.boxever.com/gracefully-shutting-down-haproxy-with...

The challenge is that whether HAProxy's own support for graceful restarts works depends on which OS and kernel version you're running and what network socket features are supported, so ensuring that's all kosher is a bit of a challenge. I took the approach that it's simpler to do something that takes a bit longer without having to worry about all that.

Reply on news.ycombinator.com