How Google Sold Its Engineers on Management

One of the big things I retained from my stats classes is the idea that, once you deviate from a pre-specified analysis technique, the strength of your conclusion is strongly diminished. Also, sophisticated statistical techniques are often less robust than simple ones. Maybe some other ideas apply that I can't think of off the top of my head.

On the other hand, the author may not have appreciated the statistical iffyness of that phrasing, and perhaps misrepresented the rigor of the actual analysis.

'For example, in 2008, the high-scoring managers saw less turnover on their teams than the others did—and retention was related more strongly to manager quality than to seniority, performance, tenure, or promotions. The data also showed a tight connection between managers’ quality and workers’ happiness: Employees with high-scoring bosses consistently reported greater satisfaction in multiple areas, including innovation, work-life balance, and career development.'

If their scores predicted those things, then they were measuring something real, regardless of whether they went looking for what they wanted or not. The question then becomes one of whether altering those scores alters the dependent variable or whether you've just created a correlation by doing evil to your numbers.

Which... they did look at their results down the line and I'd imagine they'd have looked at turnover, it'd seem really odd not to considering the other things they looked at and the obvious business case for doing so.

That's actually how one performs experiments and develops theories in the social (and managerial) sciences, such as economics: given data, and a hypothesis, and try to develop a model that fits both and offers demonstrable predictive power for future circumstances and datasets.

Of course, one's math might be wrong, and the model may still be falsified by future data. But that's what makes social science interesting.

An example: say we want to know how a drug affects cognition. We give a simple test to a bunch of people on and off it, blinded, etc. The control group's average score is 74, and the test group's average score is 72. We can use a t test to see if there's a statistical difference, and find there isn't. We can't conclude anything about the drug.

Now imagine we have exactly that same data, but we were careful to give two tests to each person (in a random order, and different tests, of course). We take another look at the data and find out that every single participant scored lower when they were on the drug. With even a fairly small sample size this provides strong evidence that the drug impairs cognition, and probably tells us quite a bit about how much it does.

The article is probably talking about multivariate regression; the more important number comes a few sentences later---"retention was related more strongly to manager quality than to seniority, performance, tenure, or promotions". So presumably, they did the same sort of analysis, carefully pairing people who were similar in as many ways as possible, and found out that good managers are more important than seniority in terms of employee retention. The more variables you have, the more even large differences can hide in raw group averages.

-8≤----

Upon this layer of commercial surveillance activity two things, then, happen with respect to government: the complicity and the thievery.

The data-mining companies believed, by and large, with respect to the United States and other governments around the world with whom they deal, that they were merely in a situation of complicity.

Having created unsafe technological structures that mined you, they thought they were merely engaged in quiet—that is to say, undisclosed—bargaining with power over how much of what they had on you they should deliver to others.

This was, of course, a mingled game of greed and fear.

But what the US data-mining giants of the West Coast basically believed, until Mr. Snowden woke them, was that by complicity they had gained immunity from actual thievery.

What sent both Facebook and Google into orbit since we were all last together—or rather, what had come out two weeks ago on the Wednesday that we were last together—was the news that their complicity had bought them nothing. [...]

So the problem is that, for the data-miners, the situation is not controllable, just as for the American listeners it is no longer controllable.

And it will only be controllable for Us if we bend our attention closely to the environmental nature of the problem that we face because environmental problems—like climate change, or water pollution, or slavery—are not solved transactionally by individuals.

http://snowdenandthefuture.info/PartIII.html

It would take an epically shitty manager to not be able to sweep that kind of failure under the rug and not manage to blame some programmers / QA for their failure.

One thing I would recommend though is to spread out their 360-degree annual reviews into a daily 1 degree review which will net Google an additional 5 degrees of review every year.

I also can't help but add that they seem to go the long way round to figuring out what everybody else already knows in the end.

I can't wait till they do this kind of intense analysis on why their social network efforts seem to grind so many people the wrong way and then arrive at the same conclusion that's already been talked to death out in the world at large.

If anyone can turn management into a real science, it's Google.