Etcd 3.3 – Improvements to stability, performance, and more | Hacker News Reader

Etcd 3.3 – Improvements to stability, performance, and more

coreos.com

59 pointsphilips9 comments

Not an etcd user but my first question was why they chose some new (and now unmaintained) golang key value store as their database instead of something more widely used (sqlite or bdb seem like the obvious alternatives). I was able to find this analysis they did in 2015: https://github.com/coreos/etcd/issues/2646 .

On the face of it, it feels like using a more established an well tested database would have been a better choice, even with the facts available to them at the time. Personally I would have been really reluctant to adopt a database whose first commit was in the previous year with only one major contributor. Three years later, they've had to adopt the database code after it was abandoned by the original maintainer.

The previous maintainer didn’t abandon it, he finalized the API and features. CoreOS wanted additional features. The original maintainer wanted to keep the scope minimal.

The reason for adopting a pure Go key value store is to utilize all of the Go tooling, like profiling, to aid with discovering performance bottlenecks. Crossing the language boundary loses visibility into problems.

As an etcd maintainer the reasoning for using a Go key value is exactly this: we can use all pure Go tooling easing debugging and profiling.

Plus, by having a simple Go only build system we reduce the burden for on-boarding developers which I believe is part of the reason for etcd's solid and growing developer base: it is easy to get into. Well, easy relative to other databases, ha!

Etcd has been phenomenal. In terms of solving the simple KV HA story, I can't think of a single technology that I have ever used that is as simple to set up and operate. We run etcd clusters with millions of keys and 500+/sec write rates and I can count on one hand the number of problems / tunings / issues we've had over the last two years.

There are lots of reasons why you might not want to use etcd, but if you can live with simple K/V, need HA, and need something that is simple and just works, I can't think of anything else that is comparable except for Consul (which I haven't used much, but has similar design principles).

Not a user but I think Zookeeper is worth a mention.

The article neglected to directly mention the new transaction features, but etcd 3.3's txn semantics are much more powerful than 3.2. It's what makes the new key leasing feature work and can be used to support key range queries over STM as well.

the benchmarks are not very convincing since both the etcd and Go versions have been changed. Go 1.9 has received another round of optimizations that probably also help with performance.

They should have capitalised on the name pun using "etc." instead of "and more" in the title... Great job nonetheless!

There is yet another key value database written in Go: https://github.com/dgraph-io/badger

Badger looks promising, though I'm not database-expert enough to know when to choose LSM tree database. (BoltDB is B tree, and Badger is LSM tree with value log.)

Reply on news.ycombinator.com