Hacker News Reader

Comment by Bostonian | Hacker News Reader

Ideally, shouldn't this be customized to the user, so that liberal users get conservative messages sent to spam and conservative users get liberal messages sent to spam?

View on HN

The problem with that is an individual user doesn't generally mark enough things as spam to give the filter enough data to perform well on. It's the aggregation of every user's spam-marking that provides a large enough data set to make good predictions. If they switched to individual spam filters, a lot more spam would make its way into users' inboxes.

Not necessarily. After a bit of customer-specific training data, users could be grouped into cohorts.

So to give an extreme example: there would be one cohort of unsuspecting elderly people that can't recognize spam and therefore still get a lot of Spam?

Yea,that would be a weakness of the approach for sure.

So, train a classifier to create an embedding of Spam Types, then train a different Bayesian filter for each Spam Type with only the data from the users in that type class?

I mean, yeah, I guess that would be possible, but is it a good idea? Echo chambers tend not to be productive, so if the choice is "get rid of stuff that most people think is bad" or "get rid of stuff that most people who agree with you think is bad" -- and if it takes much more work to do the latter -- I can't justify putting in that work. It's counter-productive, and all to satisfy peoples' desires to hear what they already think.

It's way too expensive to have individual models for these things.

Ideally users should get what they sign up to receive and nothing more.

Reply on news.ycombinator.com