Just because someone isn't good at mathematics (or whatever subject) doesn't lead them to a life of flipping burgers, nor does it mean they're less bright.
Those who succeeded in such a system tend to enjoy thinking it's a meritocracy, and implicitly consider themselves bright. Others obviously have a different perspective.