I had a similar sort of thing. There I was, plowing away in physics, doing pretty well. I was enjoying myself. They let me, an undergrad, take some grad level courses because I was doing good work. But then I took a long, very hard look at the people who were teaching me and their lifestyles. Their cars that were always breaking down, their second jobs, how much their wives worked, their clothes, and so on. Brilliant people working for a pittance.