Like being an actor or a musician or an artist is the fantasy of coffee shop employees the world over, making games is the job a lot of nerds want to be doing. In fact, so many people want to be doing it so badly that they'll do it even if they don't get paid (much) for doing it. Thus, you don't find a lot of people getting rich doing community theater or playing in a garage band, but you do find thousands of people doing it, anyway. Fun jobs only pay the bills for a small number of the most successful practitioners of the craft...and often they are so successful by virtue of skills other than the craft itself.
On the other hand, building your bingo card software would not be a "fun job" for the majority of software developers. Building enterprise software is not a "fun job" for most software developers. Building systems management software is not a "fun job". At least, not for enough people that there are thousands of new products entering the market every month competing for the attention of the available buyers.
Add this to all the other issues you've mentioned, and it adds up to games being a lot like hopping on the bus for Hollywood. You're probably not going to be a star, and the industry will still have a hundred more just like you to chew up tomorrow. Of course, most of us aren't exactly destitute. If any of the folks here make a game and it fails (I've started building a brain games company in my spare time, so I've voluntarily entered this market, knowing enough to know that the odds of it being profitable are astronomically against; so I'm one of those folks), we'll still be working on other stuff and making a good living doing jobs that are not "fun", by most folks definition. I only spend a couple hours a week on my games-related stuff, and 10-12 hours per day on my real company.