Hacker News Reader

Comment by danso | Hacker News Reader

God DAMN you put more work into meta-summary than most of us put into actual comments ;)

I know there's been talk about it, but has anyone hacked together a quickie-scraper tool or mini-app that would, given a HN submission, find similar submissions, and generate a plaintext list like the parent comment, but maybe with some metadata (such as date of submission, number of comments, number of upvotes). I'll put that on my side projects list but I bet someone more OCD than me has already done something like this.

View on HN

I have a bunch of half-finished, partially working, user-hostile tools. I can't bundle or release them, it would take too long to write what they do, and don't do. I could do something like you say, but again, as you say, I'd bet someone else has already mostly done it.

I wonder if I could cobble together from what I have something that I could make public. Next time I cross-reference stuff I'll leave a trail of bread-crumbs to see if I can regularize/semi-automate it.

"I have a bunch of half-finished, partially working, user-hostile tools. I can't bundle or release them"

This was never a problem for the Autotools people

If memory serves, someone built this with a semi-automated account, he got some (to him) surprisingly heated criticism from the community, and he stopped.

That was me.

Thanks! I was just scanning the comments from your non-DupDetector account so I could provide more info here.

The search box at the bottom of the page does a pretty good job, given a title :)

Depends on what you mean by "similar submissions", though. Similar titles (similar in topic? Sentiment? Levenshtein distance?), identical links, similar linked content (similar how?), etc.

Frequently you need to pick a single word, or perhaps two, from the title in order to find related articles. Using something like TF-IDF on submission titles (perhaps after mods have "normalized" them) then works well on the search. Then you want to make sure the articles really are the same, so you need to download the article, extract the relevant text, and do a similar sort of comparison.

Or just eyeball it. Or allow for mistakes.

Then there's the question of deciding whether it's worth having a cross-reference at all. Heuristics about number of comments, age of submission, etc, might be relevant.

In general, it's complicated.

Reply on news.ycombinator.com