Kafka was a clerk at an insurance company. Einstein was an assistant patent examiner. Wittgenstein was a gardener at a monastery and then a primary school teacher.
Van Gogh only ever sold one painting during his life, and that was to his brother.
There are hordes of other people widely regarded as "geniuses", who lived and died in dire poverty. Many others were cheated out of the fame and fortune they deserved by others who got the credit. And many more are probably still unrecognized.
The cold hard fact of the matter is that the world generally does not reward intelligence, "genius", or even hard work. True, some have the temperament and luck to scale the greasy pole of "success", but many others don't -- or choose to focus their energies elsewhere.