Let's say someone X cannot understand much mathematics -- then, if someone else, person Q, solves a quadratic equation, and person F solves, say, Fermat's Last Theorem, then, it's not easy for X to know that F is extraordinarily talented (right?) whilst Q is more likely an ordinary person. Because to X both problems are equally hard -- namely impossibly hard.
Instead, like @dcolkitt indicated, X will look at how Q and F dress (better looking = better at mathematics), and which one has the most self confident voice?
(However in things like sports, when one clearly sees who, say, is the fastest runner, then it's simpler of course.)
So yes having a successful career (i.e. an above average career) in a field might only mean that one is above average at having a successful career. A pretty circular conclusion.
You are correct that many of the senior people (i.e. professors) are not very involved technically and often do overestimate their own technical contributions ("I gave the postdoc the idea to use that instrument, try that trick"), however many of them did start out as very technically and scientifically talented people.
The problem is that the way academic funding is structured really pushes academics into a path were they are unable to do any more direct technical and academic work.
I would estimate that unless they move into primarily administrative&teaching work, professors spend about 30% of their time applying for funding, and they don't do that just to build empires or stroke their ego. If you stop and get to a period of a 1-2 years without funding, it's incredibly difficult to ever get funding again, because when your applications get assessed you get judged on your track record (previous publications) and often reviewers want to see some preliminary results to show your proposal is feasible (the irony is that on the other hand proposals should be "high-risk, high-gain". I'm not kidding, I've had proposal reviews were I was criticised by one reviewer that there was not enough preliminary work to show the project is feasible, and another reviewer said that my I essentially had already done everything on the project and so there was not gain left).
The result is that when you start a project you already need to write your next proposal, leaving you no time to actually do technical work.