This is interesting, but there are so many things that can be at work that I'm not sure how much you can draw from this. For one, boards may elect CEOs that are good for what they need at that moment - when a company needs to handle a large merger, they may choose someone with experience in doing that for that industry and their talents may never be fully utilized after that. And a CEOs performance isn't likely to be homogeneous over time either - they may get better or worse over time. I guess your basic problem is that CEOs aren't selected by a random process. So if a board of directors always did a good job choosing the right CEO, why would you expect there to be big, statistical differences anyway?

I'm not also terribly clear on what sort of strength of effect the authors are looking for. A lot of people work for a company, are they trying to attribute all of a company's earnings to a single person? Or are they looking for a difference that is not very proportionate to what CEOs actually do.

I should also point out that "could be due to chance" isn't in itself very interesting. That means that with the methods and data the authors happened to use, they didn't happen to find a statistically significant effect, and while that could be because they don't have an effect it could also be that the tools used to look for one were simply not up to par.

That said, it's my understanding (and some experience) that the upper echelons of a lot of companies can be... a bit incestuous, cliquey, who-you-happen-to-know.