Since Matt is responding here, I figured this is worth a shot, no harm in asking. Matt, would love a response from you if you get a chance, since the Webmaster Tools appeals process gives no insight whatsoever to our situation.
Following on from one of the comments here, namely the idea that "value is in the eye of the beholder", I'd like to raise our own plight. I run a number of aggregator sites - the largest and oldest of them being celebrifi.com, which was a PageRank 5 until Google de-indexed us in December (along with some of our other sites, but interestingly not all).
A little background - the purpose of the sites is to aggregate, organize, rank and add context to what's happening in the news, with each site focusing a specific vertical. Think Techmeme, but with more context.
I'll be the first to admit, there is no original content, but I strongly believe that we "add value" by figuring out what exactly is going on in any given story or blog post.
We add value to publishers, by always linking to the original source (indeed, many publishers directly request that we add their feeds to the sources we track), we respect copyright by only displaying a short snippet of the original text and only displaying thumbnail images and we add value to users by giving them easy access to a lot more content on the same topic/story, all in the same place.
Google's Quality Guidelines clearly state that duplicate content is penalized, and that is totally fine with us, but is it right to totally de-index a site for duplicate content? I wouldn't even want to rank above the original source for any given piece of content, as I respect the hard work that writers and publishers put into creating quality content, but aggregators who add value have a role to play in the content ecosystem. Digg, for example, uses the "wisdom of the crowd" to aggregate and rank content - hence adds value. Topix takes a local approach to aggregating content, and uses comments to rank content - hence adds value. We take a verticalized approach to aggregating and ranking content, and hence I believe that we add value.
As mentioned above, we got de-indexed in December, and despite going through the appeals process, fixing a few things on our end to do with sitemaps, and clearing out some older "low quality" sources that we were tracking, we received no clarity into what our crime was.
Matt, I'd like to raise this issue with you - both as it relates to us, but also as a general industry question - are all aggregators going to be de-indexed? And if not, which aggregators are and which aren't? What is the criteria, and who decides? If its algorithmic, then I am very curious to know what on our sites triggered the de-indexing? And even more curious to know why some of our sites got de-indexed, and some didn't.
I have great respect for Google's efforts to clean up spam and low-quality content - and would always expect to see original content ranking higher than aggregated content. But to completely de-index an established aggregator site and strip it of its PageRank seems very draconian.
I would love to hear your/Google's position, and look forward to some more clarity on both our situation, and the future of news/content aggregators.
Respectfully yours, Niko