> Vault's documentation clearly states to not use ETCD as the HA backend.
Yes, when I reported bugs against the etcd HA backend they said they were going to add more prominent warnings to the docs, if I remember correctly. :-) I think the underlying bug comes from overly optimistic multithreading code using CSP, but I stared at the code for several hours and couldn't see how to make it unambiguously correct.
We now have our own in-house Redis "HA" backend that I wrote which appears to be rock solid in production, but which temporarily disables Vault during Redis restarts. So it's really "fake HA" but it works nicely for our use case and never requires manual intervention or embarrasses us during business hours. I refuse to set up and administer an entire Consul cluster for a single leader-election lock. Managing Consul has its own complications: https://www.consul.io/docs/guides/outage.html
> Instead they started with an architecture where everything goes over an HTTP socket through a daemon with high probability for contention, deadlock, and failure. None of that has anything to do with Go.
The Docker daemon's wire protocol suffers from a kind a of sloppy thinking about data types, in my opinion. Consider this line in a Rust Docker client: https://github.com/faradayio/boondock/blob/bf29afb0c78f8d5d6...
pub Ports: Option<HashMap<String, Option<Vec<PortMapping>>>>,
Here, the `Option` types say, "This might be a JSON `null` value on the wire." So the `Ports` member might be `null`, or it might contain an empty hashmap, which in turns contains values that might be `null`, or might be `PortMapping` value. The corresponding Go code is extremely unclear (at least in most similar situations) as to which values can be `null` on the wire.
There's a lot of really foggy specifications going on in the network layer of the Docker server. It's not bad in the way that certain old Unix servers like `csvd` were bad, but I feel like Docker could be better if Go demanded more clarity about data types.
On the other hand, Docker is clearly a raging commercial success, and it actually works quite nicely (at least if you use Amazon's ECS to manage your clusters). So maybe I'm just being overly fastidious. Clearly all this new Go-based devops infrastructure is hugely successful, even if many of us get frustrated with it now and then.
Still, I really wish that more programming languages forced you to be clear about which values can be `null` and which can't. Hoare called it his "billion dollar mistake": http://lambda-the-ultimate.org/node/3186