Right, and this is my fundamental issue with the software industry. If we were paid 2 or 3 times as much, we'd be assigned to do higher-impact work. The reason 90% of software engineers are assigned to bullshit is that our compensation is so low that it can make economic sense to put us on the sort of low-impact stuff that companies never have the sense to drop in favor of more ambitious, more interesting projects.

Since the 1960s, we've been approaching (but waveringly, with backward motion from 1973 to 2008) what some call the "Singularity", but it's actually a more prosaic shift: a movement from the late industrial era (3-6%/year economic growth) to a technological one where 10%/year (or better) growth is not out of the question. The first sign of it is what I call the convexity switch. The input-output curve of a worker used to be concave, with the A-players producing 120, your B's producing 110, your C's at 100, your D's at 60 and your F's (slackers) at 0. Intimidation-based management and closed allocation (which are performance-middling, bringing down the best and up the worst) made sense in a time where one slacker cancelled out 2-5 A players (relative to the C standard). But we're moving into a different era where that curve is convex: your A's produce 1000, B's produce 250, C's produce 50, and the D's and F's produce virtually nothing. The traditional, mediocrity-oriented management model used by MBA programs, McKinsey, and typical software management doesn't work anymore, and no one (except, perhaps, the leaders of Valve and Github) has any clue what to do.

The issue for us is the expectancy-variance trade-off. The effect of "lean" management post-1975 was to make individual managers (and workers) more accountable for local performance, the result being that no one was paying attention to global, company-wide concerns. Everyone started becoming a lot more careerist, job tenures decreased, et cetera. People optimized for local performance and especially for low variance, because underperformance is punished more severely (firing) than high performance is rewarded (usually, small bonuses, if the high performance is noticed at all). Bosses no longer worked for "the company" or were rewarded for abstract contributions that make the whole firm a better place, but instead were measured by their bosses, and on a tight timeframe. It became more about "individual accountability" and less about actual value creation. The results favor low-variance work. In a concave world, this actually works very well, because low variance is observed at the top of the output curve. This means the work is reliable and meets expectations, and the interchangeable parts fit together well with no surprises. But in a convex world, the top of the output curve is more highly variable, so the result of low-variance management strategies is abysmally low bulk output. It doesn't look abysmally low-- software engineers are still contributing $200-400k per year of business value-- but it's terrible in comparison to what would be possible if the allowance for variance were higher.