Hacker News Reader

Comment by esskay | Hacker News Reader

I think a lot of its probably technical debt. So much internally still relies on legacy systems in US-East-1, and every time this happens I'm sure theres a discussion internally about decoupling that reliance which then turns into a massive diagram that looks like a family tree dating back a thousand years of all the things that need to be changed to stop it happening.

View on HN

There's also the issue of sometimes needing actual strong consistency. Things like auth or billing for example where you absolutely can't tolerate eventual consistency or split-brain situations, in which case you need one region to serve as the ultimate source of truth.

> billing […] can't tolerate eventual consistency

Interesting point that banks actually tolerate a lot more eventual consistency than most software that just use a billing backend ever do.

Stuff like 503-ing a SaaS request because the billing system was down and you couldn’t check for limits, could absolutely be locally cached and eventual consistency would hurt very little. Unless your cost is quite high, I would much rather prefer to keep the API up and deal with the over-usage later.

Banking/transactions is full of split-brains where everyone involved prays for eventual consistency.

If you check out with a credit card, even if everything looked good then, the seller might not see the money for days or might never receive it at all.

Interestingly, TigerBeetle manages to have distributed strict consistency over 6 machines.

Banking is full of examples of eventually consistent systems. ACH, credit card transactions, blockchain...

Sounds plausible. It's also a "fat and happy" symptom not to be able to fix deep underlying issues despite an ever growing pile of cash in the company.

Fixing deep underlying issues tends to fare poorly on performance reviews because success is not an easily traceable victory event. It is the prolonged absence of events like this, and it's hard to prove a negative.

Reply on news.ycombinator.com