The Yahoo Firehose: Get access to our public Updates activity stream

http://twitter.com/random_walker/status/11917456790

"within a few years there will be a real-time stream that aggregates essentially all activity anywhere on the web"

While Twitter, Wordpress, etc. have firehoses, this announcement is different -- aggregation across multiple services. I think the concept is very powerful and is something I've been looking into over the past few weeks.

I'm curious how many people know about Spinn3r (spinn3r.com) -- they claim to provide a real-time hose of all updates in the blogosphere. They quoted me $6k/mo. Unfortunately, Spinn3r's main product was deadpooled last year, so I'm not sure what their future is as a company. I think they should have made the real-time stream their main product. In a way they were too early.

Another startup called Superfeedr takes a different approach. They charge on a per-feed (or per-entry) basis, so their up-front costs aren't as huge. They currently cater to customers who might only want a few hundred or a few thousand specific feeds, but as they scale up they should be able to offer a bulk stream of essentially the entire blogosphere.

Then there's Google Buzz, which is also about aggregating different services. They don't currently offer a firehose but apparently you can build one yourself using PubSubHubbub, although developers report that it isn't working so great: http://groups.google.com/group/google-buzz-api/browse_thread...

But the long-term trend is clear. We're going to have an "uberhose" at some point! I have a few applications in mind of things that such a pipe would enable that are fundamentally not possible today, but I'm curious what ideas other people have.

If they continue to aggregate 3rd party data and put it all in an easily accessible firehose, I could see this being really useful.

I haven't used them a lot, but Yahoo!'s developer stuff seems pretty handy. Would be nice to see them reinvent themselves or really push this division much like Amazon AWS has.

It's a very cool service that only a company the size of Yahoo! (and Google, Bing, etc) can provide due to the infrastructure needed for crawling and serving. It would be interesting if this was the first of many Yahoo! moves in which they shift from a pure search + portal to data providers. Their APIs already do that, but this is much bigger.

how comfortable are you on depending on yahoo services and why?