For football(American), I agree, but basketball and baseball stadiums see much more use. MLB teams play 81 home games a year. NBA teams play 41 games a year, and basketball stadiums can also double as Hockey Arenas, and concert/convention venues.
NFL teams only play 8 home games a year, and they're usually very big - 60k-100k seats - much too large for almost any other event. These massive buildings might only be used 10 or 12 times in a year. Huge waste of taxpayer money.