The NFL has also oversaturated the brand. It used to be Sunday games and then MNF. Each week was an event. TNF ruined the event aspect, and has put on really terrible games showing how thin the league has become. Then there are the games in other countries. For a sport like baseball or basketball 1 game in another country is not a big deal, but the NFL season is only 16 games and losing a home game is a huge deal for the fans, players, and competition aspect.
So yeah, I think all those things (and the stadium issue you mention) have added to up to pushing fans away.