Hacker News Reader

Comment by ComputerGuru | Hacker News Reader

That would encourage them to try to generate spam comments in the slim hope that they'd get upvoted.... which would lead to a much worse spam situation.

For instance, they'd employ Markov Chains to re-hash comments in a post, or from older posts, or from the text of the article.

One thing I've learned: spammers don't give up. If you give them an easy way to submit that's even easier for you to clean up, that's better than starting an all-out war that'll just increase the spam content and make it more difficult to filter out.

View on HN

Very true.

The best you can do is allow the post to go thru and make it appear to the spammer that they have succeeded. Let them view their post on the homepage and go away with a nice warm feeling of satisfaction, secure in the knowledge that they don't need to improve their algorithm at all for this site.

It's pretty straightforward to modify your display mechanism to include spam posts if and only if they originate from the same IP address as the viewing client.

I implemented this on Blogabond, and it had two positive effects: My Spam corpus is growing at a faster rate (thus making it more effective), and the sophistication level of the average attack has dropped sharply.

The problem is detecting the spam posts in the first place. Your idea would be incompatible with the proposed method, because you don't want to do the same thing to legitimate new users.

Certainly, HN must have a bayesian classifier looking at the content of its submissions and deciding whether they are "Hacker Newsworthy". By now, the site must have sizeable corpi (corpuses?) of good articles and spam links to use in such an assessment.

Since this site was built by Mr Bayesian himself, it never occurred to me that it might have weak spam filtering.

> The best you can do is allow the post to go thru and > make it appear to the spammer that they have succeeded.

I read the cons on this suggest, but what about turning the approach just around: Keep the spam in by default, just the folks with higher karma get the spam filtered out? That way, the successfully submitted spam could not only be verified from the original source but also from every other node of, say, a bot net.

Spam bot generating comments that get voted up? Reaching the right level of sophistication, these spam bots would suddenly turn into quite acceptable members of the community, even if they are completely automated. If readers in general find these comments worth up-voting, than it would not matter much what the underlying intent was, because it is not harmful anylonger.

A spam bot can find a link that is published at the same time on reddit (by example) and HN, take some comments with lots of upvote and submit-it to HN. He will probably get some upvotes.

There are actually already spambots that do this on reddit -- they find a highly-voted comment in a reddit thread, follow the link to the article and repost the comment there (filling in the URL they want if the site in question allows it). Unless you look carefully at what people claim as "their site", or happen to read reddit as well, they can be hard to catch.

Isn't there an easier way to reach the 2 karma threshold? Like, creating 2 accounts, the second's only purpose being upvoting some throwaway comment the first one creates and subsequently deletes?

In the proposed scheme you can't upvote unless someone upvoted you first. So you can't create a second account to upvote the first, because that second account needs its own upvote before it's usable.

A human can use that account first to get beyond two votes, and then subsequently pass that off to a spam bot

Yes, but that would mean that he took the effort and used his skill to present a truly interesting comment. This isn't particularly hard, but I think harder than a bot or hired non-english speaking captcha breaker can perform.

We don't have to make an unbreakable system, just one that makes us less profitable than the rest of the internet.

AFAIK, most captcha breakers aren't hired. They are simply bribed with porn and similar things:

"To see naked pictures of Marissa Mayer, please type the words below into the box."

Honestly, Marissa Mayer would appeal only to geeks & nerds. And those might be more interested on what she can do, than look.

Would a spammer really go through that just to submit a spam link that probably wouldn't get beyond the new page anyway? It's quite a bit of work, even if you have the same framework that you can use across multiple sites. My experience is that a post that doesn't get upvoted will receive around 30 hits.

Surely there are easier ways of getting 30 hits.

They only have to write the code once. Then they can get 30 hits to a different site every day.

I don't think it's feasible to make "write once run anywhere" code for this. Input fields will be named differently, rules will vary and spam filters will trigger on different things. Even if you only have to spend an hour grooming your script for HN it seems like a lot of work to get 30 hits. You can of course reuse the script on the site once you've made it, but it won't take much before PG figures it out and bans IP-addresses, or some other measure.

Heuristics aren't terribly hard. Scan for an input text box of a minimum certain size. Then try each submit button independently to see which one gets your sample text on the page.

Thank you: I needed to hear that. Part of my brain likes to keep scheming about "more absolute" ways to keep them out (and thus avoid any clean up).

Your points suggest that my utopia-philic thinking is probably less useful than accepting that some clean up is just a necessary overhead (cf: "Eternal vigilance is the price of liberty."). Thank you.

Combating Markov Chain generated spam sounds like fun.

Reply on news.ycombinator.com