Music and architecture bring joy to people who like them. Even in musical competitions, most of the audience enjoys most of the performance. Sports seems to be zero-sum; people don't appreciate a good game, they appreciate one arbitrarily-labelled team beating another.

(Interestingly I'm far more able to appreciate professional wrestling, since (at least some of) the fandom for that is more interested in seeing a good performance than which side wins (and indeed often it's the "loser" who has the most important and hardest job))