Ask HN: algorithmically categorized curated news
The goal is to not create groups which are hyper-focused or biased by humans but to occasionally show articles which relate to what you're interested in so that you can slowly broaden your interests and see how it relates to other things. It's about finding things which are not posted on every single top website, but those niche blogs that are somewhere on the web.
However, the groups might be too broad/random, so there might be an option to subdivide a group.
Pros:
- the initial investment will be around $10 to train the k-means clustering model (on amazon ec2), after that it should be able to categorize websites on commodity hardware (my laptop) and be basically free. However, I'm forced to keep however many groups I've trained until I have to re-train it, costing around $10.
- I plan on using commoncrawl.org data, which means I can avoid using my web crawler full time to get fresh articles
Cons:
- it might be too random, and people won't really understand why they have to search through 100 categories to find something that they want (a high initial investment on behalf of the user.)
- or, if it's too focused, it might divide people into categories (which is the opposite of my intentions)
- spam might be a problem. I hope that the clustering algorithm will group those websites into a few other groups, and those groups could be hidden. We'd probably rely on crowdsourcing to flag spam.
The only way I could get a few bucks out of this is to put ads on there, or introduce a subscription-per-month model.
My question: would you use such a website? If so, how much would you be willing to pay per month? Is this idea hopeless or could it be restructured?