97 karma · joined January 16, 2011
(You can keep Russell Crowe though...)
AwStats screen shot: https://unavailable.s3.amazonaws.com/20121130_WP-Super-Cache...
Google's own google-content-api-for-shopping@googlegroups.com mailing list has this as the "To:" field:
google-content-api-for-shopping@googlegroups.com
And at the bottom of the message:
To unsubscribe from this group, send an empty message.
I had to ctrl-u and check the "Delivered-To:" and "X-Forwarded-For:" headers before I could unsubscribe.
(I'd tried to unsubscribe previously but the subscribed email account forwards to my main account so replying with an empty message didn't work. This thread prompted me to dig a little deeper and finally get one less piece of email per day - thanks HN!)
Should it be "chown -R mysql:mysql" ?
Set your browser's user agent to "Google" to see. This is a common exploit to lift scam sites in SERPs.
Screen shot:
http://autoreverse.s3.amazonaws.com/2011110_Babson_Freep_hac...
Google cache:
http://webcache.googleusercontent.com/search?q=cache:JIadb7r...
Apologies for putting this as a reply on HN but I emailed their contact address twice and got bounced both times.
A few comments:
1) Please add one or more licenses so developers can include Akayame in their projects with confidence. A couple of examples:
https://github.com/jquery/sizzle/blob/master/LICENSE
2) The footer and menu are below the fold in my browsers in the Converter and Editor views so I need to scroll to see the menu and "Yet Another Markdown Editor" text.
3) The footer could do with improved styling : text overlaps the border, bullets and links use default styles. Minor quibbles but improvements would improve the professionalism of the site. Image here:
http://autoreverse.s3.amazonaws.com/2011006_Akayame_footer.j...
4) Landing page does not convey to the visitors what the site is about (and"Yet Another Markdown Editor" is below the fold).
5) Adding a page or two with keyword rich text explaining the project ("jQuery powered online Markdown editor") and background ("inspired by other Showdown online Markdown editor projects") could help with search engines listings.
6) Have you considered hosting the code on Github for increased exposure and link juice?
<img src="http://autoreverse.s3.amazonaws.com/20110906_freelance2.jpg alt="">
I'm not alone :-) see http://incredibill.blogspot.com/
FWIW I've been using robots.txt honeypots on two sites for the past few years to deter scrapers. Approx 1,600 IP addresses have been banned. Most of these are for data centers.
Check out bad-behavior: http://bad-behavior.ioerror.us/
Otherwise:
Use a honey-pot to catch browsers that don't obey robots.txt. Send yourself an email for every new IP address so you can catch any false positives.
Redirect requests with no user agent specified to your honey-pot URL.
Same for known bad/useless user agents: wget, curl, Bing, etc.
Validate supposed good crawler agents via reverse IP lookup - cache bad IP addresses - redirect to honey-pot URL
Filter remainder based on request rate over a given period (use a DB to cache requests). More than 1 request per second for 20 seconds = bot, more than 200 requests per day = bot : redirect to honey-pot URL
I've been doing this for a site with around 150K URLs for the past 4 years. Have about a 600 URLs blocked and around 35 user-agents.