Has anyone actually done this at scale and seen how it works? We've talked about it a few times because Consul looks really cool but it doesn't appear to have the full capabilities of SmartStack yet, in particular Synapse (https://github.com/airbnb/synapse) for intelligently doing stats socket updates, intelligent reloads, or controlling how HAProxy reloads over time.

Also consul only has two levels of services right, "local" and "remote"? I'm not sure that is sufficiently powerful to describe SOA's which need to be able to differentiate services on the machine, rack, colo, datacenter, region locality? I suppose you could do what we do in SmartStack and just register the same service instance at a few different places since I think that consul can register arbitrary services.

Maybe all we need is a consul watcher in Synapse, and a consul-template for the Synapse configs, and we can get the best of all worlds. With Linux 4.4 SYN handling in Linux might even be fixed by Eric Dumazet being super hardcore (e.g. https://lwn.net/Articles/659199/) and we won't even have to worry about reloading HAProxy.

https://github.com/eBay/fabio might also be an option for you.

Disclosure: I'm the author.

As long as you don't have too many microservices... the polling and templating doesn't scale too well.

This is related to https://github.com/hashicorp/consul-template/issues/165#issu... although that is even more extreme.