Hacker News Reader

> the fuzzy nature of the recommender problem means I am never going to get into the 90's

Right. If your AUC is in the 90's you usually either have a bug, or a predictive task that isn't very challenging.

Some tasks are hard, others are easy. I get in the low 60’s trying to predict if a headline will get a lot of votes on HN. Of course this is going to be bad because the same article can get submitted several times, hit it big once and not hit it big the other times. The content of the article matters too but the model can only make a wild-assed guess. I get in the high 70’s trying to predict if a post with a lot of hits gets a lot of comments, it is a cleaner problem. I get about 90% trying to predict if a post winds up ‘dead’, I found up HN is cleaning up a lot of spam posts when they go in but I know it is terrible overfit because it says an arXiv paper which has no chance of being flagged has a 30% chance of flagging.

For some other like “is this an astrophysics or materials science paper?”, getting 0.95 or better was straightforward in 2005.

Reply on news.ycombinator.com