Reducing website spam
scirra.com
scirra.com
Back in the time when I was too lazy to install Akismet and manually managed comments on my blog (via moderation), I used to see from time to time annoying SEO comments - a fake profile with genuine-sounding comment, backlinking to a website catalog. When I was especially bored, I used to go to such catalog in order to find the SEO company who was spamming me.
On one occasion I contacted a private photo studio, asking if they're using SEO company XYZ, as links to them are all over the catalog. It turned out that they just hired a SEO company to 'position their website', without knowing anything about how it's done. I explained the situation (sending screenshots), and the studio immediately ditched the SEO company and hired another one. I was amazed by this attitude and featured a special advert-like post on my blog for them and we're in good contact since then.
Yes, you're right. I currently don't have any other idea how to deal with those small, spammy SEO businesses than to raise awareness about the subject and hit their customers directly. It's sad, because the customers here are usually not to blame - like you said, they neither know nor understand what's going on here. I was quite (pleasantly) surprised by the immediate reaction of that particular person, who 'out of respect for my space in the Internet' decided to take action that cost her money. I deeply respect such people. "Vote with your dollars" mindset should be encouraged, IMO.
Anyway, here's the story - it's in polish, but maybe HN folks from Poland will find it interesting (and get to know about one thoughtful photographer) - http://temporal.pr0.pl/devblog/2011/09/19/spam-seo-i-fotogra....
1. Base64-encode your form field names and decode them server-side prior to processing, or...
2. Create one-time use field names using md5 hashes of random numbers, map them to their true fields and store them in a session. Then process against those on the server side post-submit and clear the slate. (I used this method more often than not.)
3. Control the visibility of the honeypot field with CSS rather than "type=hidden".
Using #1 OR #2 in addition to #3 I've never had to use CAPTCHA nor human tests. A few paid spammers have come around from time to time, but since automated software isn't sophisticated enough to pick apart which field's which it either doesn't even try or it throws whatever it can at the fields, getting locked up in server-side validation.
The amount of additional processing power it'd take to surmount this (and the other methods) from a spammer's end at scale would be incredibly prohibitive, and that's kind of the point.
However hiding it completely with display:none should be safe and is what I think you mean.
In our case, just hiding it works fine. We can upgrade this to a CSS solution if we need to upgrade it though. The main point is if your doing something, you're ahead of the herd enough for spammers to generally leave you alone.
We had tried CAPTCHA and honeypots but the spammers broke through it (we were being actively targeted). Once we used email verification, it forced the business owners of the email (Gmail, Yahoo) to implement better verification on their side and stop so many fake accounts from being created.
Spam is now a rounding error in my system (about 60 for every 56,000 daily posts).
I should also add that I subscribe to the broken window theory, so we also implemented Akismet to check all incoming posts for spamminess. We hide all posts that Akismet marks as spam, and it cleans up the place enough to (hopefully) ruin any spammer's SEO tactics. Once it becomes a futile effort to post spam that's just going to get hidden, they seem to stop aggressively targeting us.
So while it's a barrier to entry, we attempted to minimize it as much as possible.
If you name the honeypot field something like "address" or "website", you get browser toolbars that will "helpfully" try to pre-fill the field for the user even though it's hidden. And then you're flagging legit users as spammers. I think an ideal system would simply require users who fail the honeypot field to submit a captcha rather than lock them out altogether.
Display:hidden on an input field would be OK, but there might be cases on some sites where the field loads before the CSS file. This could cause auto filling to happen. It's a lot harder to test this sort of thing with CSS rules, it's a lot easier just to use hidden fields.
I'd recommend using names such as "username" as honeypots as the likelihood of being filled are high.
Or course, if someone took the time to target this specific site, that would be easily thwarted.
It still puzzles me just how this custom-made form got infiltrated by spam to begin with. Do people go around picking such forms and submitting form-specific bot-code code to some vast pool sold to bot operators? Or are bots far more intelligent than I give them credit for?
I get comment spam on some sites that I'm 99% sure is being posted by a human using a regular browser at an internet cafe in China.
A better solution is to let them register, but check all registration/post IP addresses against the blacklist. If they match, unobtrusively move them into a usergroup that just seems like your site isn't working that well.
Works wonders.
http://www.codinghorror.com/blog/2011/06/suspension-ban-or-h...
Reddit uses a similar system called shadow banning. When a user is shadow banned, they think their content and submissions make it through, but they actually are hidden to other users. On rare occasions, real users can accidentally get shadow banned; this is not ideal, but shadow banning is effective enough that the sacrifice is made.
The ones that still get through on rare occasions copy legitimate content and usually link to sites that aren't inherently spammy (like grey-market pharmacies) for SEO purposes. I suspect these are actually posted by humans; a post can classify as suspect rather than spam or ham and these always do when they get through. When that happens, the user is asked a trivia question related to the site's subject matter. I think it extremely unlikely a spam bot would be able to answer correctly.
Are they trying to find some sort of exploit in your code that lets them send out emails?
I can't imagine there are any search engines important enough to optimize for that don't use the rel=nofollow cue.
Google "nofollow seo value" to find other insights.
Maybe include a link near the start of the form that says something like, "Screen-readers, follow this link."? The link would be to a one-time use page that is sane and accessible. I don't know...