> Well, statistically speaking
You do not have the statistics to back that statement up.
People usually have less time for games once they have kids, but they have less time for basically everything. Super Mario Bros came out 28 years ago; it's existed for essentially the entire life of anyone younger than mid-30s, into people in their 40s for it being around since their childhood. And that still doesn't even cover people that grew up with an Atari 2600 (36 years ago) or grew up in the culture of arcades (Pong was 41 years ago).
Even getting past the "competitive level" that the GP is going on about, the comparison to spending the afternoon "watching the game" is exactly right, and it's really not that big a deal for most people.
(the best statistics you'll find, by the way, put the median age of gamers at 30--meaning the every single person in the group you think soley cares about games is matched by a person older than that--though the methodology is so-so and it includes a wide variety of "gaming", including on the ever-present mobile device)