Gnip: Numbers + Architecture
blog.gnipcentral.com
blog.gnipcentral.com
Yes, if you've ever tried to create a mashup using data from Twitter, digg, delicious, etc., you know the problem is that you have to poll their entire data set (without exceeding their api request limits) to get at the data you are trying to track.
With Gnip, you get an abstraction layer so you can create a filter based on usernames, for instance. When these users do something, like tweet, we post that data back to you.
In this scenario, you no longer have to poll arbitrarily for needles in haystacks--we push the needles to you when they occur.
Hope that helps! If anyone is interested in utilizing data from the social graph to create their app (or if you are interested in publishing data you have for public use), we have a Group list: http://groups.google.com/group/gnip-community as well as documentation on our site, www.gnipcentral.com and a set of client libraries (we are currently evolving) to help facilitate such endeavors.
Cheers!