I'm not an avid Twitter user and this is only my personal impression rather than a guess why anybody else might be abandoning your app. That said:

Initially the idea of filtered tweets seemed compelling. I think everyone has that one spammy friend who they only follow because unfollowing would be impolite. But soon after signing up I realized that it's simply too much work to be worth it: not your app specifically, but using a separate client and having to come up with filters. Filtering and organizing is simply not natural for Twitter - it's meant to be a continuous stream where missing something is no big deal. I could possibly use filtering if it was in a form of a browser plugin (so I could still use twitter.com) and had just one option - to suppress tweets from a specific account.