This is more than a bit unconvincing.
I have all the basketball forwards I need, but Channing Frye isn't living on food stamps. He's not the one people come to see, but he helps keep a competitive organization running so that there's a platform for LeBron (who people do come to see). Your personal interest in a mid-range competitor doesn't determine their value, because they're helping to enable the higher-profit performers.
If the Olympic competition isn't a value generator, then the coaches and functionaries enabling the athletes are even less valuable than the athletes themselves. God knows the spectacle and ceremony part of the Olympics isn't a money-maker, that's why cities aggressively resist having it come to town.
The assumption that any of this is a function of market forces is funny to me. Sponsorships are probably the only free-market aspect of the whole program (or they would be if they weren't exclusive and determined above the athlete level) - the IOC operates as a 'charity' drawing money from national governments. As a result they're completely un-accountable to market forces, and athletes can't make any effort to collect their value. The highest-profile competition in the world is funded by donations, and they can't escape it.
Jack Warner and his ilk at FIFA weren't making a fortune because they were so efficient and talented, they were making money because principal-agent problems and regulatory capture are real. If the people raising player funding are also deciding how to spend it, they're likely to declare themselves invaluable and keep it all at the top.