Zookeeper resilience at Pinterest
engineering.pinterest.com
engineering.pinterest.com
zookeeper is nice.. and well known, battle tested, and a pita on devops (no dynamic reconfig till 3.5), and devs. a pain to interact with at reliable scale outside of native java clients, oh.. and the zk src.. cyclomatic complexity and npath complexity on large scales.
i keep hoping for something like etcd (raft + rest) can bring simplicity and ease of cross-language use for distributed coordination to the many.