I'm not sure that's true. It would be interesting to see some numbers. My naive impression is that there are at least two things needed to make intellectual history: genius, and opportunity to practice your genius. Both are rare, and given that there are far, far more people who need to earn a living than people who don't, I would expect the absolute numbers of (Genius)(NeedsAJobHasAMenialJob) to be greater than (Genius)*(Doesn'tNeedAJob).
My impression is that gentlemen of leisure have, especially in the 19th, 18th, and 17th centuries, been a very disproportionate percentage of intellectual history, and that even the nominal jobs in that group (parson, Lucasian Professor of Mathematics at Cambridge) were wholly orthogonal to whether they produced.
What do you think having an unrelated, undemanding job would contribute to intellectual work?