Hacker News Reader

Comment by TomGullen | Hacker News Reader

We got hit with a huge wave recently, that sent over 40,000 visits a day to our site and nearly ground it to a halt.

The number 1 effective thing we have found to do is to not allow hyperlinks to be posted if they are not trusted (not enough rep/point/score whatever)

Overnight it basically stopped the spam wave. Your removing the one thing of value for them, a hyperlink. I'm a big fan of accessibility and this works well with it. The only other technique we use is honeypot form fields which do catch a fair few, but nowadays a lot of spam I suspect is paid human spam.

View on HN

Blocking any spam that contains a link is helpful, for sure, but doesn't get everything. Every few months I see waves of comments like: "Really graet article. We need more people like you in the world."

Each comment has exactly one pair of transposed letters. There is no product being pitched, and no url (we don't display or link to email address either). It's baffling.

Some blog platforms whitelist comments from people who have had previous comments approved. I'm pretty sure these meaningless (but positive) comments are an attempt to get on that list.

yes. also some platforms (mostly forums, less of blogs) allow editing of posts, so forum spammers sometimes post meaningless crap only to replace it later with spam.

We should be able to detect that by looking for large numbers of posts with small edit distances. That will contain false positives, but looking for very large numbers should mitigate that.

I would guess that the transposed letters are used to keep tabs on where their comments are live. It could look innocent, as if it were a human typo, but later the spammers could run a search and see which sites are trusting their comments in order to either edit them later or use their trusted account to post spam links.

Those comments could be used to make automated spam filters less effective. Spammers could post comments that would normally be labeled as spam, but do not contain any URLs. Over time a spam filter would have a harder time distinguishing between cut-and-dry spam and real comments (assuming the admin is marking those spam comments as ham).

Also, I'm a fan of not allowing brand new accounts to post URLs in their comments. It's a no-brainer.

> Each comment has exactly one pair of transposed letters. There is no product being pitched, and no url (we don't display or link to email address either). It's baffling.

Sounds like they're doing what's known as "Bayesian poisoning" (http://en.wikipedia.org/wiki/Bayesian_poisoning) ahead of time to open the door for later link spamming.

Don't they still try to post links in the body of the comment?

Seems pretty effective indeed. If i look at the caught spam on my blog all of them have hyperlinks in either the 'website' field or the comment text itself.

Pity it also targets normal users that simply want to post a hyperlink :(

Maybe the barrier to posting a comment should be low if you don't have a hyperlink but if you include one then you have to jump through some hoops (captcha's, etc).

I implemented that with some success. Much more successful was to simply require Javascript.

Interesting. Why do the bots run with JS turned off?

To be clear, we're talking about an automated process that is looking for forms to fill out. It's far easier and faster to simply scrape the HTML of a page and look for the <form> and <input> tags to figure out what to submit than to actually fire up something that resembles a real browser.

>have hyperlinks in either the 'website' field //

Isn't the website field for a hyperlink? (or at least a domain that is converted to a hyperlink?).

If I leave a comment but refer back to a related page on my blog is that automatically "spam" in your opinion.

Hmm. I wonder if spambots check back on their work to see if it's worth continuing to attack a target. I guess removing the hyperlink would make them decide to drop it and move on.

I would say almost certainly yes. In my own forum I found that a short period after I had implemented spam filtering, the moderation queue length (everything that gets rejected as spam goes in to the mod queue) had very little spam in it. I would surmise that spam bots are at least smart enough to check that their posts are getting through, and if not, don't waste time posting content (just try one every now and then to see if you can slip one through and then hammer it!).

did you try, or do you know if merely adding nofollow was enough to have an impact? I've always assumed comment spam links were more about growing/stealing page rank than actually being clicked by anyone.

Reply on news.ycombinator.com