krenoten

epaxos is not general purpose replication. It leaves the problem of reconciling multiple replication streams to another layer. I've heard that they do 20+WA (distributed, before you consider on-disk WA) for much of it. shrug

A useful perspective for me has been that C, A, and P are spectra rather than all-or-nothing. A system may bend a bit on availability when some of its replicas are partitioned, because there are fewer servers that can respond at all, smaller aggregate queue depth etc... It has been said that consistency was specified rather poorly in Brewer's original paper, and we've since seen work that dives into the consistency spectrum. Peter Balis's "HAT not CAP" paper is an often-cited example of such work. Aphyr again comes through with a summary of some of these ideas in https://aphyr.com/posts/313-strong-consistency-models .

We know that we can't have 100% harvest and 100% yield in the presence of partitions. We can play with different things though to shape the curve of these things as conditions worsen, however

MTBF of 2PC-strapped-to-quorum is no different from MTBF of a 2PC-strapped-to-spof replicas.

MTTR is bounded by reelection latency, rather than replica recovery, although you still may eat a write amplification cost for rereplication.

write amplification is 3-5x of non-quorum-backed 2PC system, depending on replication ensemble size.

google further multiplies write amplification with geo-redundancy, so bump that WA by another 3x+.

it's an insanely high cost to pay for availability, but for an advertising company it's important to count the beans accurately.

Aphyr covers the impossibility of practical CA systems here: https://aphyr.com/posts/325-comments-on-you-do-it-too

Sometimes it's helpful to consider a distributed system through the lens of "harvest vs yield" where harvest is the proportion of information in a system reflected by a response, and yield which is the probability of receiving a response at all.

https://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.24...

The system you describe would typically be referred to as AP, because it maintains availability in the presence of partitions.

to the extent that the left is associated with the maintenance of our rights

This needs to be the first thing anyone who works with stateful systems learns. NEVER rm. mv is insufficient. mv dir dir.bak.`date +%s` has prevented data loss for me several times.

everything is broken, and we are usually late to notice due to the infrequency of happy-path divergence.

This is a great one to remind us that we help to shape the world that politics have to operate within. We are not just subject to politicians.

Yo, make node-local a first class citizen already... This is an area where mesos is a far more suitable choice right now for stateful systems.

Joining one of the FANGs as your first job is a great way to kill your chances of developing a highly diverse skill set early on. Good luck doing anything interesting when there are 120 superior engineers who are bored out of their minds all vying for the opportunity. Working in a smaller company that has real problems to solve tends to be much more rewarding. There's a reason the technical leadership at the big 4 tends to be comprised of engineers who came from elsewhere.

I've been far happier as an engineer in NYC and Berlin than SF. Gentrification has made SF a pretty boring place to be, and I felt that my overall sense of creativity plummeted there when I was surrounded by so few people from different backgrounds.

There's a great documentary about this by Adam Curtis called [HyperNormalisation](https://www.youtube.com/watch?v=-fny99f8amM), discussing how truth is seriously under fire right now.

in the past they've spread servers into different rooms that had interconnects that challenged a team I was on to spend extra time writing compression code. is this better now?

That's a great example of how even some of the most influential minds are fallible. Lamport is sort of the god of distributed systems, having invented vector clocks, paxos which is how there is any sanity in modern data centers (chubby, ZK, etcd, etc... are based on the core ideas of this), TLA+ for reasoning about correctness, so much great stuff. Check out some of these! http://research.microsoft.com/en-us/um/people/lamport/pubs/t... http://research.microsoft.com/en-us/um/people/lamport/pubs/p...

They had an incentive to doggedly seek IP so that they could remain as entrenched as possible. They didn't plan on actually profiting from it in most cases, they just wanted to be the first ones there so that nobody else could be.

It's totally unreasonable to mandate a particular FS for modern general purpose software.

Multiple files does mean multiple threads. Unless you manage to run this without a modern operating system.

This feels like the kind of bug I catch early in implementation using quickcheck x_x Is property based testing used in the development of these kinds of systems?

It's dangerous and irresponsible to build services advertised as such to those you cannot provide the claimed protections to. When we build infrastructure with rotten foundations, people can die.

I occasionally find the client from the arch linux aur repos pegging a CPU core as well. bad syseng :(

America today is not the America of WWII. We are not saving the world from Hitler today, we are blowing up weddings and causing Europe to be flooded by refugees running from the chaos left in the power vacuum we created. The reaction to this caused Brexit, and nationalism across Europe is again on the rise thanks in large part to our foreign policy.

The preview contains old working exploits. They should release some post-snowden compilable source if they want a little more legitimacy. Python files the russian government picked off a compromised server a few years ago are not interesting. Code shared by snowden with someone who had bad opsec is interesting but not as interesting as what they claim this is.

Military misadventure happens all the time. When I was growing up near a military base in Norfolk, Virginia, a family who was friends with ours had a blast of machine gun fire rip through their car while traveling with the mother, father, and baby inside. Luckily nobody was injured.

Pitchfork is not a publication that is targeted at software engineers.

No, as long as it's 3.10+ (minimum supported version for DCOS generally) it should be good to go.

That's exactly the point. It triggers a reaction. Way better than "blahdb" IMO :)

This is one of the most exciting DB's coming into the world, and I've had a lot of fun watching progress from the sidelines! Keep kicking ass!

this is rarely a concern for things that are ephemeral in nature to begin with. it's also a 20 line code change to add a similar ttl buffer into memcached.

redis is single threaded, so 32 cores doesn't mean much without sharding.

I generally prefer memcache unless you have a super locked-down infrastructure (no engineers to deploy a KEYS operation that destroys a shard and all the systems that rely on the data inside until it's finished). Multithreaded + simpler API is great for multitenancy when you have to provide infrastructure to engineers who don't want to learn about infrastructure.