"Smart" and "intelligent" people are just people born in relative financial ease with enough time to focus on studies and very likely with parents in academia or similar fields who made their children aware of the inner workings of the fields.

Physics PhD people are not people fallen from skies with a little extra brain. They just got the time, opportunity, and tips to be there. Same goes for other "smart" people. There are, of course, exceptions like von Neuman, Turing, Feynman, Einstein, etc. But these people are exceptions. Perceptions abourt other people in those fields should not be formed with the image of these guys.

Source: very first hand experiences.