Robots, Not Humans, Fake 23% of Web Video Ad Views, Study Finds | Hacker News Reader

Robots, Not Humans, Fake 23% of Web Video Ad Views, Study Finds

bloomberg.com

82 pointsMaxQuentero50 comments

Not sure I fully buy the argument that the industry is actually genuinely losing that $6bn. If I buy plates in bulk but find that 25% are usually broken in shipping, I will tend to pay 25% less for each shipment on the assumption that this will happen - the supplier might even have already priced this in.

Online advertising is a much more fluid market and very metrics-focused - I find it hard to believe that most smart companies haven't priced this into their bidding. There's not some sort of intrinsic value per click or impression - it should purely be based on the expected propensity of it leading to a conversion. Fake clicks and impressions are just a factor which reduces this and therefore it should automatically be taken into account.

In fact, I'm surprised it's not the suppliers of inventory, like YouTube, who are complaining that this is erroding their value. If advertisers are getting caught out by this then they're not being very smart.

That's what I thought but the fraudulent ad views are probably not distributed evenly. You can't just assume that 25% of views on your ads are fraudulent. There's a lot of uncertainty here.

As long as it's not a hugely noisy in the way it changes within each exchange/source of inventory, you should be able to account for it. Don't forget plenty of other "legitimate" changes happen every day to affect the value of placements too - usage patterns, page layouts, tastes and fashions etc. They can be just as nefarious but all need to be accounted for - fraud is just another factor.

25% is an average => some small advertisers likely close to 100% broken in a way which is not immediately visible to them.

There's a big difference if 25% of your plates are accidentally broken in shipping vs being smashed with a hammer in the warehouse.

I imagine the supplier would probably care (and maybe the police too) but as long as I'm paying 25% less it's not really my problem.

Your 'big difference' is _intent_, a flighty, volatile emotion that no rational person will ever care to account for.

https://en.wikipedia.org/wiki/Mens_rea

The whole point of my post is that this kind of thinking is bullshit. Saying "the courts do it" doesn't contradict that, it's just an appeal to history or authority.

If I break the plates, you might be able to get restitution from me. You can't get that from a bumpy road. Also, you might be able to avoid broken plates in the first place by threatening me with jail time. You can't threaten a road into being less rough. The intention of a person completely changes the threat profile and mitigation strategy, that's why it's a big deal.

Only if you can predict intent. Which you can't.

Exactly, lets not judge people on their bad results, but their good intentions.

I think you're right. It's not "the end of digital advertising". It's just another factor among many that lead to lower PPC prices.

When GM left Facebook because they claimed Facebook ads don't work, it was a huge story. But when GM returned to Facebook less than a year later, that didn't get as much publicity.

Many people are asking why it's a problem, why don't advertisers just pay less, etc.

The adbots are making money off of this, otherwise they wouldn't do it, and that money has to come from somewhere. Meanwhile, they don't contribute anything of value, so someone's losing money to the adbots.

The burden of the loss will actually be distributed between the ad buyers and ad space sellers depending on their elasticity. The economics is very similar to that of a sales tax, with the proceed going to the botnets.

I think a clearer way to put it is this: The 23% number was measured across networks displaying ads on many sites. Some of those sites employ botnets to generate fake views, giving them much more than 23% fraudulent views, while other sites that don't do such things have 0% fraudulent views. The amount the advertiser is willing to pay is presumably set by their experience across the whole network, so if they were willing to pay $X per ad without fraud, they will pay 0.77 * $X due to fraud, which means that the honest websites get less than they deserve (based on the value they deliver) while the fraudulent websites get more than they deserve. So, the fraudulent websites are stealing from the honest ones, not stealing from the advertisers.

Not knowing the rate of ad fraud does make things worse, but is not necessary to explain why it's a problem. Even if the ad fraud rate was absolutely uniform across all websites, there would be a deadweight loss. The ad fraudsters are extracting money from someone.

If the ad fraud rate were uniform across all websites, then advertisers would simply pay less CPM because its less converting traffic.

If website A is all fraud, and website B is 0% fraud. Then on the same network, you'd have a 50% fraud rate, and website A would be paid money it doesn't deserve (as it deserves nothing for its 100% fraud traffic).

Fraudsters are being subsidized by legitimate traffic.

No. The price is determined by supply and demand. So you're saying demand would fall - much as demand for a good falls when you impose a sales tax. There would be deadweight loss from the lack of traffic and advertisers would still end up diverting funds towards adbots.

The money comes out of the pockets of publishers. Collectively, they are making less than they should be for a pageview or a click. The advertisers have a budget for ads, and they will only pay that. At some point, when the ad is not effective and profitable, they will redirect that same ad money somewhere else. It's the Internet publishers that are losing.

They're both losing, the advertiser's loss is opportunity cost.

Not really, many people price bids according to CPA, with a well managed campaign a click fraud site would receive a bid X% lower.

It isn't surprising. Click fraud has always been an easy way for a creative programmer to generate "free" money. That said, it has also always been a complaint for ever that advertisers don't think they are getting their money's worth[1].

One answer is to switch to a pay for performance model like affiliates do, basically only pay when something is purchased not just on click or view. But brand advertising doesn't really have a good 'performance' metric.

As others have said though, its a 'fake' problem, just like credit card fraud is a 'fake' problem. The costs are baked into the charges so the vendors get their money either way. But having a story that BigCorp is taking pennies from its legitimate customers to pay for ads being clicked on by robots is a hard story to get sympathy for the advertiser :-)

[1] "Half the money I spend on advertising is wasted; the trouble is I don't know which half." -- John Wanamaker

To all the people saying this isn't a problem: it really is. Even if advertisers bake a fraud premium into their buys, blindly letting money flow to bad actors creates an incentive to be a bad actor, and bad actors in this space do bad things that hurt everyone: waste massive amounts of bandwidth, spam the web and app ecosystems with worthless content, and worst of all exploit users to build and run botnets.

We've spent a lot of effort over the last few years fighting this stuff, and we're really happy to see the industry as a whole starting to agree that it's a problem. If you'd like to help make the web better by working to remove these incentives, we're hiring engineers; contact info in my profile.

I agree with the above sentiment. To elaborate on who gets hurt, the problem is that small publishers get squeezed. If you're ESPN, you can create a private exchange, keep a strong agency sales force, and keep up your prices. However, the smaller publishers will not have the ability to reach the buyers except through these compromised exchanges. If the buyers flee the open markets, the smaller pubs have to band together into networks again or try their luck with programmatic direct (Pubmatic is making a push for this with their recent acquisitions). We go back to a world of media buying that looks more like TV; we become a lot less fluid.

I'm surprised that no-one is talking about accuracy here.

How, exactly, do they determine who is and isn't a bot?

> For the study, researchers put software into the actual ads of 36 members of the advertising association between Aug. 1 and Oct. 1. The software allowed them to tell if viewers of the ads were people or bots.

I think that's what is being sold, and is what the report is promoting.

As a product and company, they remind me of http://spider.io who did a similar thing and were acquired by Google. I was interested in using the technology to fight comment spam on blogs and forums, but it was really tailored for detecting fraudulent clicks and impressions on adverts and Google were quick to acquire.

Some of the spider.io blog is still accessible and shares some of the approaches they used: http://www.spider.io/blog/

It also assumes all bots are intending malice. Some advertisers automatically trigger videos on sites so they can review/categorize the content (see if it's porn).

They do that with the intent of making ad purchases, so it's actually desirable bot traffic.

Someone please explain why this is a problem. The way I see it is that advertisers pay for results. If you spend $100,000 on ads and make less than $100,000 in extra profit - you're paying too much for ads. The fact that 23% of ad views don't convert just pushes down the price of an "ad view", ie those fake views are priced in to your CPM rates already. This does not effect your ad budget however.

People that get hurt aren't the advertisers but the quality publishers who may be struggling to sell ads at higher CPM rates, justifying those with fewer fake ad views. But this is nothing new, ad rates have always varied considerably across newspapers based on their quality and audience, and ad buyers should understand that. It's completely normal to pay hundred times more to show your ad on WSJ compared to PornHub for example.

I don't know enough about this, but surely ad buyers have the tools to track conversion rates of different advertising mediums and publishers, thus enabling them to ultimately compute the cost-per-sale?

Most advert buyers are not sophisticated. So fraud matters.

I don't believe that. I've worked with some companies that spend millions per year with online ads. They are sophisticated.

Being sophisticated might just mean you understand that the current cost of fighting fraud is higher than the cost of the fraud. It doesn't mean you're not getting bled.

Agree that they understand that fraud exists. And they are not immune to it any more than you or I. But they don't look at it as "being bled". They pay X cents for a click, and they get Y sales per click. And Y has to be greater than X.

Most people speak out of their ass.

Branded campaigns for example have no such easy way to measure conversions - and since multiple advertisers at the same time get hit by a fraud operation at different rates, the loss is not evenly distributed, not easy to track, etc.

Add to that the fact that a lot of companies only care if the ROI is positive, no matter the cost, with the middlemen making money both on bot generated and human generated impressions, and you've got a whole messy business on your hands.

Source: our company just won 500k in investment at Slush to battle this specific problem.

>Someone please explain why this is a problem. The way I see it is that advertisers pay for results. If you spend $100,000 on ads and make less than $100,000 in extra profit - you're paying too much for ads. The fact that 23% of ad views don't convert just pushes down the price of an "ad view", ie those fake views are priced in to your CPM rates already. This does not effect your ad budget however.

If you negotiate an agreement where you would make a 15% profit on a transaction, and a fraudster swoops in and takes 12%, would you be content about getting the remaining 3%?

Ideally whenever a transaction takes place, both sides would share about half the benefit. Liars should not be able to pull money out of the middle with the sole restriction that both legitimate parties still make some profit.

Advertisers, for example, should demand transparency from website operators about where their traffic comes from.

Ads are usually served from ad networks' servers so they should already have this information.

Yup, and also, it's oh so difficult to fake a referrer.

I wonder how mass usage of AdNauseam could affect this:

http://dhowe.github.io/AdNauseam/

http://www.informationweek.com/software/productivity-collabo...

Online marketing/advertising is funny. You'd think once bots could get up to 20% fraud they could be scaled up a bit more and get to 99.9% fraud (the old "machines won't be exactly as smart as people argument", people will be smarter for some time and then machines will be smarter after that). The cynical me also thinks as the fraud isn't likely to be distributed evenly and you don't know if you are the one subsidizing the market you would get a "market for lemons" effect ( http://www.win-vector.com/blog/2008/05/is-search-advertising... ). But I am guessing the desperation of people to be in the market keeps the prices up.

I'm confused about the motivation here. Are the companies serving these ads running the bots, or paying others surreptitiously to run them? Why not just directly fake numbers if you're going to cheat customers? I feel I must be missing something...

There's two reasons that spring to mind:

1. The companies faking adviews don't directly control the ad management system, so they need to fake a page view on their servers in order to get the 3rd party ad manager (e.g. Doubleclick/Adsense) to record it.

2. The most obvious way of detecting fake traffic is to see that your ad click-through numbers don't match up with the actual traffic hitting your server. So to obfuscate this, you'll have your bot click through the advert then land on the advertisers page and spider through a few links before bouncing.

Click fraud works like this:

1) Make shitty blog 2) Put some ads on it 3) Buy some traffic 4) Buy some click fraud traffic 5) Profit.

eg. If buzzfeed wants to make a little extra rev this month all they have to do is buy some click fraud traffic.

Ultimately, as long as advertisers see a positive return and receive value for their money, click-fraud continue. Once that changes, then advertisers will take their money elsewhere. Until then profit margins will just keep getting smaller and smaller.

I am genuinly curious how they manage that. In a world where every user is followed I don't think the robots have a chance to fool the anti-fraud algorithms. At least with the bigger players (Google, Facebook) they don't.

Imagine a botnet has managed to get itself installed on Grandma's computer and is injecting 1 false ad impression for every 3 real impressions. The botnet can install things on the computer. You can't (if you're Google, assume Grandma isn't using Chrome). To stop the fraud you'd need to work out which of the 4 impressions is the fraudulent one. That's an incredibly difficult problem to solve...

Most of these bots seem human by traditional tracking systems (Google, Facebook) because they posses the same browser cookies as the human user who's machine is infected.

Agree this is a problem, but if I'm making a list of companies I empathize with for being victimized by shady business practices, Wal-Mart comes in dead last.

Does anyone have a link to the actual study?

Heh, I'm one of the authors of the report. You can pick it up at http://whiteops.com/botfraud .

Just make the viewer click Google's captcha in order to show the ad ;)

Reply on news.ycombinator.com