Simulating Service Discovery with Docker and Etcd | Hacker News Reader

Simulating Service Discovery with Docker and Etcd

talwai.github.io

53 pointstalwai12 comments

I like consul over etcd, as it implements health checks for your services, and provides DNS services.

In combination with consul-template you can pretty easily create automated service discovery which will update a configuration, such as HAProxy or nginx LB config, and reload the service.

Indeed, I recommend giving Consul a try if your looking into deploying a microservices architecture. I even go as far as saying that Consul provides some of the stuff an ESB is suppose to provide in a traditional SOA environment, most prominently: location transparency.

I'm using the Deis publisher from the Deis project to accomplish service discovery.

It's a neat go daemon that runs as a global unit(every node) listening to the docker event bus and updating etcd for you. From there it's easy to use confd to manage updating nginx or haproxy (which is what the deis router does).

You can use skydns[1] to implement service discovery based on DNS. It has etcd as a backend and creating/updating a DNS entry is done by creating an entry in etcd with a TTL. Then, you can use DNS the same way you are already doing without service discovery.

[1] https://github.com/skynetservices/skydns

Kubernetes seems like it is the future way to do this. It auto spins up, wires and load balances docker containers implementing named services within a cluster of vms.

https://github.com/GoogleCloudPlatform/kubernetes/blob/maste...

Given how much effort you have to go through with etcd and discovery, why not use an MQ like RabbitMQ, that will take care of replication, etc as well?

First of all, because one of their explicit choices was to use HTTP. Using a persistent message bus like RabbitMQ has great advantages, but in terms of failure modes acts very differently to HTTP. For example, an upstream service failure would result in a timeout on getting a response (or ideally, a different instance of that service would respond to your message instead), rather than a more traditional TCP socket error or close.

Again, not necessarily worse, just different and unfamiliar.

The other reason is that this really only pushes the service discovery down a layer - how do you know where RabbitMQ is running? Of course, the same could be said for etcd - how do you know where etcd is running in order to query for more services.

Thanks for the explanation. I was looking for any dealbreakers with the MQ approach, but the concern you mentioned is valid. About pushing discovery down a layer, sure, but you'd do it the same way you'd query etcd, as you say.

ekimekim summarized the reasoning quite well - primarily it was because I wed myself to HTTP early on in the project. To the question of knowing where the service registry itself lives, that is the one dependency which cannot (to my knowledge) be resolved dynamically. A persistent DNS record, which you update whenever the registry location changes, is the best I can think of atm.

Thank you for the response. You're right, I think discovery needs to be resolved statically, but a persistent DNS record hosting a server that can give you the rest of the info seems reasonable.

RabbitMQ has decided to punt on any robust scheme for surviving network partitions. If you're building a distributed architecture and you care about availability and correctness you should just avoid it.

https://www.rabbitmq.com/partitions.html

Interesting read, have you ever used federation? It looks like it's built for exactly this purpose.

Reply on news.ycombinator.com