The very premise that 'smartness' is easily quantifiable is also flawed. Attempting to rank all human minds on some hierarchical scale is going to run into the same problems as with human bodies. Is it better to be able to run the fastest, or to lift the heaviest weights, or to have the best hand-eye coordination when it comes to throwing and catching objects? Will specialization in one area of mental skill inevitably create weaknesses in other areas of mental skill (e.g. the strongest weightlifters are never the fastest runners over long distances)?

Not only that, there's the whole cult of intelligence issue, which attracts people who are very skilled at presenting themselves as uber-intelligent even if their greatest talents like in the realm of social engineering and bureaucratic manipulation (many highly successful academics fall into this category). This is a kind of 'smartness', but not necessarily the kind that helps one grasp complex maths.

What we call 'smartness', or 'actual skill' is the product of some degree of native intelligence and the amound of time and energy spent developing that intelligence in a particular area via diligent effort.