Comment by kitcar | Hacker News Reader

Comment by kitcar | Hacker News Reader

My last business was ad-supported (premium, big-brand advertisers). One day, out of the blue, one of the companies which sold ads on our behalf sent us a nasty letter accusing us of buying junk traffic / creating false impressions and clicks - something we most definitely did not do.

Digging through the raw ad server log files, I discovered that the "suspicious" impressions and clicks were all originating from AWS IP addresses - most likely someone was using AWS to run a spider on our site, the spider followed Javascript links, and therefore clicked every ad on every page.

We ended up adding a rule in our ad server to prevent ads from being served to any IP address which belonged to cloud hosting / VPS providers - this solved the problem for us.

I've since sold the business and therefore don't know if the problem ever arose again, but I believe blacklisting IP address blocks which are highly unlikely to belong to real human beings* could be a good start for anyone running into these sorts of issues, either on the advertiser or publisher end.

*Yes, I know some people run VPNs on AWS or similar VPS instances, which means they are real humans - that was a loss we were willing to deal with.

That solves the bad bot problem, it doesn't solve the actual ad fraud problem. There are at least millions, if not tens or hundreds of millions, of ad clicks per day coming from botnets of ordinary Windows computers on residential ISPs with full JS-executing browsers. You can't detect this activity based on IP address, user agent, script execution, etc. I've seen individual advertisers targeted and get hit by hundreds of ad clicks per day, when they usually only get a dozen or two, every click from a different Comcast/FiOS/TW/RR residential IP, with unique user agents, and varying search phrases that match the ads.

It's much, much harder to detect and block that; if we could only see activity on our own individual websites it'd be nearly impossible to tell the normal clicks from the fraudulent activity.

Agreed - targeting bot nets is a whole other ball game - one that likely can only be combated at the ad server / exchange / DSP level, as they are the only entities with large enough data sets to tease out which machines are infected.

From the single advertisers perspective, the easiest solution I can recommend is working with networks which provide eCPA-type bidding* - as then sites which actively buy traffic from bot nets will over time be blacklisted automatically from your campaign. Back when I was on the buy-side of online advertising, we used that "trick" to great success with a major credit card issuer buying billions of impressions.

*What I mean by eCPA type bidding is when you tag your conversion page with the network's pixel, and the network uses your conversion data to optimize the campaign on their end to get rid of publishers which send click that never convert. I know there is a better term for this, but it's a Sunday night and I haven't worked in media buying for a few years now...

what do the owners of those AWSes win from this? did you relay second hand referrer info on the clicks to the ad network?

Reply on news.ycombinator.com