If the system is working properly, you should mostly be able to accumulate wealth only if you provide value to everyone else. That's the theory anyway.
There's this meme I see on HN sometimes that every use of time is equally worthwhile, and that any value judgments are arbitrary.
Highly skilled communities should value work that creates some value, above noodling around in games.
I've played lots of games. But I know they aren't real, beyond helping me relax, or maybe learn skills that transfer out. I agree with the OP that we shouldn't spend all our effort in games. I'm similarly skeptical of things like the fashion industry, super professional sport etc too but that's just me :)