In Soviet Union, Optimization Problem Solves You (2012)

Speaking of that I recommend watching Pandora's Box documentary. It is fun and pretty well made (although the quality on youtube is rather low).

Here is where they start talking about Gosplan -- the central economic planning department:

https://www.youtube.com/watch?feature=player_detailpage&v=h3...

It is funny to hear how they tried to control things. At some point during Stalin's time they planned how many people to arrest and where. No matter if they were guilty or not. The it talks how they tried to modulate various controls -- they measured success by amount of raw material consumed, so all of the sudden they ended up with oversized couches, and trains were being run for thousands of miles empty just to burn the fuel so everyone can get a bonus during the years' end. Then they started to fix prices for everything. That ended in disaster of course and so on.

Then have to like the taxi driver driving past Gosplan and saying "how they hell do they come with such ridiculous plans".

EDIT: They also mention Victor Glushkov, the father of Soviet Cybernetics. Here is a documentary about him as well. It is putting it in good light as if Soviet Cybernetics used in planning would be successful. It is a propaganda film. But it is fun to watch:

https://www.youtube.com/watch?v=lMS1hBhV2-4

He talked about paper-less office and economy back in the 60s.

Cosma Shalizi's article is correct, as far as it goes (as far as I know). But some of the commenters here did not read the whole thing. He adds, capitalist free markets can't optimize either: 'If it’s any consolation, allowing non-convexity messes up the markets-are-always-optimal theorems of neo-classical/bourgeois economics, too. (This illustrates Stiglitz’s contention that if the neo-classicals were right about how capitalism works, Kantorovich-style socialism would have been perfectly viable.) Markets with non-convex production are apt to see things like monopolies, or at least monopolistic competition, path dependence, and, actual profits and power. (My university owes its existence to Mr. Carnegie’s luck, skill, and ruthlessness in exploiting the non-convexities of making steel.) Somehow, I do not think that this will be much consolation). [...] 'Both neo-classical and Austrian economists make a fetish (in several senses) of markets and market prices. That this is crazy is reflected in the fact that even under capitalism, immense areas of the economy are not coordinated through the market. [...] ' The conditions under which equilibrium prices really are all a decision-maker needsto know, and really are sufficient for coordination, are so extreme as to be absurd.(Stiglitz is good on some of the failure modes.) Even if they hold, the market only lets people “serve notice of their needs and of their relative strength” up to a limit set by how much money they have. This is why careful economists talk about balancing supply and “effective” demand, demand backed by money.

'This is just as much an implicit choice of values as handing the planners an objective function and letting them fire up their optimization algorithm. Those values are not pretty. They are that the whims of the rich matter more than the needs of the poor; that it is more important to keep bond traders in strippers and cocaine than feed hungry children. At the extreme, the market literally starves people to death, because feeding them is a less”efficient” use of food than helping rich people eat more.'

The article has another weakness, which is that Shalizi does not seem to understand how capitalist corporate planning works in real life. Increasingly, whole supply chains are balanced to point of sale events and other signals of incipient demands. And those whole supply chains do not do open market exchanges, the members are contractually bound. It's not optimal, but it's good enough. And all of that could be done without prices or money.

Red Plenty (the book mentioned at the top of this) is truly great and well worth tracking down.

Am I the only one suspicious of his claim "It will not do to say that it’s enough for the planners to approximate the optimal plan...This route is blocked."? It seems to me that in all my experience, computational complexity even for approximations rarely matches up with practical computation required. Computational complexity is worst case, doesn't understand the concept of "good enough for our application" when it's not mathematically defined.

For example, what's the computational complexity of training a deep neural network? Probably something pretty horrendous, even if you say it only has to be approximate within a factor of the optimal weights, and you allow it to fail some percent of the time, etc, etc. You could probably write an article about how training neural networks in the lifetime of the universe is impossible even in theory, if you defined "training a neural network" by starting out with the problem of finding the optimal weights, and then relaxing the requirements. But that entirely misses the point that training a neural network is not fundamentally about finding the optimal weights, but finding some weights which are good enough, which is measured in terms of real world performance in comparison to the alternatives.

Likewise, it seems like the discovery of the mathematics of linear optimization was mixed too strongly with the real-world problem it was trying to solve. The question is whether it is computationally feasible to outperform market based economies using this technique, and that's the only question that's really make-or-break it for the math side of things.

(Of course I may have misunderstood something, feel free to correct me)

I haven't read the book, but I believe one of the largest problems that centrally-planned institutions (like large corporations) face is that the planners tend to remove redundancy too much (in the pursuit of minimal cost).

For example, let say you have chain of 50 products that need to be made into a product. If each of these is produced by one factory and supplied into another, and each factory is late for a week, your new thing is one year late!

This was a frequent problem, that the planners tended to designate one producer of some product, in order to save costs. But the product wasn't good enough or late, and then you had a cascade of failure ending up in shortages of goods.

So there seems to be a tradeoff between redundancy and cost. What is optimal? Would you think it's optimal to say, send the cheapest possible rocket to the orbit, with no backup systems?

However, in the free market, you cannot control redundancy. It just happens through freedom. That pushes it out of the cost optimum but makes it lot more robust, and winning in the real world. A good example of "worse is better" indeed.

I also don't think the actual planning problem is very difficult. There are multinationals that are larger than some state economies and still can do that. So I think traditional economic textbook explanations of central planning failures are wrong, because they ignore the tradeoff.