Comment by vitus | Hacker News Reader

Comment by vitus | Hacker News Reader

An interesting exercise I went through while reading this article was taking these observations about centrally-managed top-down organizations and considering them in the context of centrally-controlled distributed systems. A couple of snippets:

> One of the core principles of the free market is that central planning committees can never be as efficient or as effective as the people doing the work. By definition a free market economy lacks a decision-making hierarchy; the ‘free’ means every agent (individual or corporation) can decide for themselves, without needing permission from a manager above.

> ...

> It should be immediately obvious why this fails miserably: The distance between the central planning committee and the farmer is so great that good decisions are nearly impossible. It’s nearly impossible for critical feedback to make it from the edge, where the farmers are working, to the central planning committee in time to affect decisions, and then for those decisions to make it back to the edge in time to be useful.

One of the core arguments for using a centralized control plane (versus a distributed one) is the ability to perform a single global optimization, which can yield better results compared to local optimizations. I find it interesting that the free market principles dictate the opposing conclusion that this global optimization is infeasible, although it may come down to my next point:

If you're not careful, and your centralized system injects dozens of minutes of delay due to the additional complexity of performing the global optimization, you end up with the globally optimal decision from 20 minutes ago, which in the context of, say, load balancing is an eternity. Or in the bureaucracy of government (or hardware planning cycles, etc), you may see months to years of lag between the inputs your model accepts, and the outputs your model produces. And all the while, you have a full pipeline of decisions coming down the line, based on stale data.

(There are other reasons for centralized control, such as orchestration of distributed computation being difficult to do correctly, and it being easier to reason about a single node rather than the complex system-wide interactions.)

While I see where you are coming from, I don't think the free market has much to do with it, as "free market" to the extend that such thing exists outside the fantasies of ancap weirdos, is just one of many types of distributed decision making systems, and as such shares a lot of traits with them.

So it's a question of organising big systems and how best to do that. Nothing says that we can't have organisations structured around this decentralised decision making, and quite the opposite - we both have the evidence this works, and it has been so thoroughly studied there are tons upon tons of pages written going back all the way to the 20s ...

Another factor which may come into play is inaccuracy. In a distributed system where there is a known algorithm for making local and global decisions that are as-close-to-optimal as the algorithm can determine, then everyone gets to benefit from the increased knowledge present at a global level rather than at a local level. In the human world, some people simply make bad decisions, since there isn’t an algorithm for a lot of leadership decisions. If that person is the global decision maker, then rather than everyone benefitting, everyone suffers; whereas with a local decision-maker, only a small group suffers if the leader makes a bad decision (and hopefully other leaders pick up the slack).

It’s also partly because a distributed system can factor into its decision-making a vast amount of information, as where humans may not have sufficient understanding of all areas (and again, decisions are not algorithmically optimizable) to make the proper decision. As where more knowledge can help an algorithmic decision-making process, it can overwhelm a human one.

My experience is that more knowledge can help an algorithmic decision process make a better decision, but it can also take longer to make that decision when you're dealing with combinatorial explosion. When you inject delay into a feedback system, you almost always will introduce oscillations. (You can do some things to partially correct for it, but that works best when you have a predictable and consistent amount of delay.) Intuitively, you can think of this as taking too long to respond, and then overcorrecting time after time, since you keep overshooting your target and not realizing that you've overshot for some time.

That said, computer systems are similarly prone to inaccuracy. Perhaps not to the same extent as human organizations, but they have similar propensity to fail spectacularly. If you fail to model certain characteristics of the system (or model them poorly), then you can end up in a situation where your load balancer is causing significant congestion and not thinking anything is wrong. Or your hardware planning pipeline thinks that it has lots of spare capacity even though you're overflowing every day at peak. Or your central planning committee for the government thinks you have lots of excess grain production when that's simply not the case in reality.

(Algorithm implementations can also be outright buggy.)

Another way your model can fail is if you treat resources as infinitely fungible. Suppose you have enough food in Shanghai to feed two billion people, but no food anywhere else in China. Without an adequate transportation / distribution network, everyone outside Shanghai is going to starve, and all the excess food in Shanghai is going to rot. Similarly, your servers in California aren't going to do very much for users in India if you don't have the infrastructure to actually serve that long-haul traffic.

Another point that I didn't bring up, but related to your point about global vs local decision makers, is that centralized systems have much larger blast radius. In the case of China during the Great Leap Forward (one of the examples from the article), bad decisions led to famine across the entire country. In the case of a global load balancer, you can break your service globally.

>If you're not careful, and your centralized system injects dozens of minutes of delay due to the additional complexity of performing the global optimization, you end up with the globally optimal decision from 20 minutes ago, which in the context of, say, load balancing is an eternity.

That actually doesn't seem avoidable, because some of this time will need to be spent identifying risk and malice/corruption.

Reply on news.ycombinator.com