I'm not really sure why it never expanded beyond the "early adopter" crowd. Maybe the readers were too hard to use or the benefits weren't explained clearly. Part of me kinda hopes it's because publishers started caring a lot more about the design of their sites and people actually preferred the experience of reading on the original site instead of a "stripped down" version in an RSS reader.
RSS is still available though, but the email channel is clearly the winner in terms of people using and interacting with it.
I think it was almost entirely because the "follow/unfollow" experience was so clunky, especially compared to Twitter and FB. Even today, adding and removing feeds from my readers is unnecessary pain.
The hurdle is far too high for the casual audience.