Comment by duggan | Hacker News Reader

Comment by duggan | Hacker News Reader

People talk about Zookeeper as a negative, but in a quorum it's been one of the most stable and reliable pieces of software I've deployed, despite being a bit frustrating to set up / configure. Netflix's Exhibitor[1] is an indispensable addition to it.

Also, once you're on the Big Data™ train, a lot of things like to plug into Zookeeper, so it becomes more of a convenience.

Kafka, and presumably DL, are at their most useful when you're pushing the limits of NIC and/or HDD performance for throughput. Zookeeper's configuration is a footnote in the complexity of managing one of these systems, and lets them avoid implementing their own byzantine coordination system. Also, folks seem to appreciate Aphyr's opinion, and he states it pretty plainly: Use Zookeeper. It’s mature, well-designed, and battle-tested. [2]

[1] https://github.com/Netflix/exhibitor

[2] https://aphyr.com/posts/291-jepsen-zookeeper

I found Zookeeper to be pretty easy to learn about and deploy. I also found it got a semi-bad reputation with others because they did not want to learn anything. Two major examples being:

* attempting thousands (tens of thousands?) of simultaneous writes across data centres/continents at the same instant and then saying it was slow. Subscribing all clients to updates on all parts of the tree. The architecture had been grown by people who didn't understand the guarantees and constraints.

* calling it unreliable after arbitrarily moving nodes around without changing connect strings and generally mis-configuring it. Essentially, pointing clients at machines that no longer contained nodes and blaming ZK for this not working.

It could be easier to setup. People often don't want to think about such things at all. But it's also not the hardest, and I found it to be very resilient to node and network failures when deployed correctly.

Agreed, some of the problems I encountered were because I did not carefully read the documentation. Some are because a lot of tooling has become easier to deploy in the years since Zookeeper was released, so the bar has been raised.

Which is just to say there's room for improvement. A dependency on Zookeeper is fine if you've already got a configured cluster, and a cognitive speed bump if not.

Reply on news.ycombinator.com