Search Goes Real-Time With Scoopler (YC S08) | Hacker News Reader

Search Goes Real-Time With Scoopler (YC S08)

techcrunch.com

67 pointsrokhayakebe35 comments

Cool. But until there are a lot of real time services their value is very limited.

Going by the current trend, there will only be more real time services in the future. It pays to be early in the game :)

Yeah. Soon I can imagine a time when we could talk to each other online in real time. Like twitter but without a delay. But maybe that's a couple of years off.

have you seen mibbit.com?

Or some sort of instantaneous messaging or how about a telephonic device that transmits voice over copper lines (or even wirelessly) or perhaps meeting someone for a malted beverage at a public house of some sort.

>> "how about a telephonic device that transmits voice over copper lines"

In real time? You're crazy.

This is a fun game to play, actually, any time a new technology is introduced.

Imagine the new technology was invented first, and the old technology is actually the new challenger. What are the advantages and benefits of the old technology you can promote over the new one?

this is a tradition in the Emacs community: for each release, they publish "antinews", e.g.:

    > For those users who live backwards in time,
    > here is information about downgrading to 
    > Emacs version 21.4. We hope you will enjoy
    > the greater simplicity that results from the
    > absence of many Emacs 22.1 features.
    > [...]

http://www.gnu.org/software/emacs/manual/html_node/emacs/Ant...

Does mibbit or chatterous or an Instant message client allow for search of conversations? If not I think all chat clients should make an API to search data, especially links shared in email & chat clients!

Members of Chatterous groups can search through chat history.

isn't Google pretty close to real time anyways? I know I tend to find things on Google that I just posted 5-6 minutes ago.

Yes Google is pretty fast for newly published static content. However, it's difficult to use for events happening right now. From the feedback we've got, people like real-time results because its engaging and effortless, a little like watching TV.

"One element that’s really nice, is the Peek feature, which allows you to take a look at pages being linked to in an overlay on top of the results, so you don’t have to visit the actual page."

Real-Time news for the writer: you actually visit the page. It's an iframe...

In the old days, there was something called target attribute. Neat when the target is not iframe friendly.

This feature is really curious to me... why would I want to peek when I have a back button on my mouse?

Good execution on the trending and the real power here is the deep search on live content. Anybody who does search right has value.

Thanks! We're working on adding more data sources to provide broader results.

What is Scoopler built with?

Scoopler is written in Python. We use Cloudant (YC s08) for storing our data. I'm planning to write a blog post about this on soon on our Posterous.

Cool. I'll take a note of that using Snipd, upload it to Scribd, then post it to Reddit, then tell my friends to upvote me using Heysan or maybe Chatterous. It depends on whether I'll have enough time, since I'm always playing on DraftMix.

Anyway, good luck with Scoopler. :P

If you are having problems with time management why not try RescueTime? ;)

Tu Chez!

http://en.wiktionary.org/wiki/touché

I have one question, are they consuming other companies search API, let’s say Twitter or are they crawling and aggregate their own data? Either way it’s a great job!

We actually crawl and index all the data ourselves. That's how we identify the most popular content.

Is this truly crawling the sites, or querying each site's search service?

e.g. for Twitter, are you crawling all the Twitter users' URLs (seems difficult to do - how do you find new accounts' URLs?), or are you fetching results from search.twitter.com ?

We take the raw data streams (e.g. a data mining feed from Twitter) and process them, ourselves.

Yeah. I'm curious if they developed something similar to Twitter's search functionality or they are using Twitter's Trends feed.

In its current form, most of the results are from Twitter which doesn't really differentiate it much from Twitter Search.

I don't think crawling and querying are mutually exclusive ideas.

Nice Work!

This is really cool - congrats. I'd really like to see google-trend-like features but on much shorter time scales. Even a weekly-zeitgeist tied to news stories instead of having to wait for the end of year.

For the most recent trends you can check out the hot topics. We also want to do news updates regularly - follow @scoopler and you'll get updates soon!

In case you haven't seen it, we do that for Twitter: http://twist.flaptor.com/

Nice start. I was expecting more from the twitter feed. It's an rss feed from a query. Maybe a better way is to group similar real-time results ? v2

awesome job dude! It would be awesome to have some sort of real-time alert feature, too!

It's in the pipeline! Thanks.

Reply on news.ycombinator.com