Hacker News Reader

Comment by FiReaNG3L | Hacker News Reader

My experience building http://esciencenews.com , which is automated news for science, tells me otherwise.

About the 'instantly obsolete' news - just put the latest of the cluster at the top. We see that a lot with shuttle missions to space : see http://esciencenews.com/sources/space.com/2008/11/12/space.s...

In the cluster ('More sources from other sites') you can see the mission`s coverage from start to finish.

As for bad grouping, just accept that you won't be able to find ALL stories belonging to a cluster and set your cutoff to accept the minimum of false positive while keeping the most stories belonging to a cluster.

View on HN

Hey FiReaNG3L, I've been scratching my head about this for a while. Over on eScienceNews, some of your news items are full stories. Some are just summaries. When I follow your source links below the full stories I find sites offering summary-only RSS feeds. How are you getting the full versions of those stories? Are you scraping the HTML from the feed item URL?

The full versions are press releases we get from universities and the like. The summaries with links to full stories are copyrighted (or not, as some site editors do the same press release cut and paste as our automated service does).

What's the workflow look like on those releases? Do they e-mail them to you and get "sucked in?" Do they have a special feed? Do they fill in a form?

We scrape them, and the universities and other organizations are pretty happy about it, we often get email saying thanks, links from professors and emails asking how much views they received, etc. These would get no exposure at all without sites like e! science news.

Cool. One feature that I wish that you would add is RSS feeds for each sub-topic. Other than that it looks great.

There is already; http://esciencenews.com/syndication

I just noticed the topic pages still link to the main rss though, will change that as soon as I get back from the lab!

Your site is very well done. I will be returning soon.

First thing I thought about when I saw Gabe's post was your site and how well it works, even though it is fully automated. Your last point is probably the most important one, and in my book that's an acceptable trade off.

One question. What's the speed of updates? I know techmeme wants to be the place where you go immediately when a news story breaks out. Would that be a problem (only computational I suppose)?

Updates are every 5 minutes, which is plenty. Faster than that I don't see the point.

How do you do the clustering? Open source tool/software? Nice site!

http://search.cpan.org/~mdehoon/Algorithm-Cluster/perl/Clust...

Full write-up on Drupal.org, awesome CMS we used to build this : http://drupal.org/node/261340

Reply on news.ycombinator.com