Companies are paying huge sums to show their ads to bots | Hacker News Reader

Companies are paying huge sums to show their ads to bots

wired.com

619 pointsthm331 comments

When I worked in adtech, I proposed a simple solution to ad fraud on our platform - assume any UA coming from an IP range owned by a cloud provider was a bot and don't show them ads, don't count their impressions.

Worst case, people using a VPN hosted on EC2 might not see an ad, but I was certain they'd cope. Best case, higher quality traffic increases yields.

The business team vetoed that, it would adversely impact their numbers in the current quarter, and well, a significant proportion of their total comp was bonuses tied to those numbers. Also, the shareholders seemed to be of the mindset that has infected cryptocurrency speculators where no matter what, the spice must flow, wait, I mean, the number go up.

Then there were all the "bonuses" and "incentives" negotiated with big advertisers or publishers. We just referred to them as kickbacks, because they totally are, but labeling something in code as kickbackPercentage was verboten. Just in case.

Advertising has always been corrupt. Bots just corrupt it a little further.

I made an ad fraud startup which ran into the exact same problem. With video ads, advertisers will pay 10x more to be in a “user initiated” video player (the user clicked or there was a reasonable expectation there’d be a video on that page - YouTube, camera icon on preceding link, etc).

But more often then not, those ads are running in an autoplay context (sticky video players in the bottom corner, large players in the footer, etc).

I built a mechanism which could tell the difference (viewport, fixed positioning, fingerprint elements on the page, provide a screenshot of the offended page with your ad on it, and various other signals).

We ran tests with advertisers, agencies, demand side platforms (dsps), server side platforms (ssps). We’d compare their existing fraud numbers (very low) with our reports (80% fraud).

Within each company, we’d have a champion saying “this is going to change everything. Quality supply, better performance metrics, etc etc”. Then we’d get introduced to a team whose compensation was driven by “percentage of spend” - every deal died at this stage.

My opinion: if your company is using online advertising tied to conversions (someone buying something, or potentially signing up for something) then you’ll be fine as long as it’s economical. But if your ads are for brand awareness, be extremely careful running on any website which isn’t in the top 10 sites on the internet.

A funny anecdote that's not fraud but puts a bit more ambiguity into the alleged targeted ad eyeballs even from the big players. There's a good amount of "ambient" content on YouTube these days. Some not targeted at adults or even humans.

I have been playing YouTube videos on my TV of birds & wildlife, for my house cats to enjoy. There is one guy who makes 8 hour long videos. There is one content creator basically targeting this market of "birds for your cats to watch".

I have noticed that the YouTube app on my Samsung TV will play for an unbelievable number of hours without any "are you still watching" prompt. Easily 6 hours!!

I've been playing them videos almost daily for weeks and only got this prompt this weekend finally.

So some companies probably think they are paying for targeted ads to some category like "people interested in cats / wildlife", YouTube collects a cut and pays the content creator a cut. All this for feline-only eyeballs.

That's why my cat recently ordered a casper mattress

Next, the cat will upgrade that meager 55" to a 75". A new fridge with bottom freezer will follow. Kitty must be comfy, else kitty will find new home

I do the same thing, but in my case it's guinea pig eyeballs. But I'm a Premium subscriber so I don't get ads anyway.

Kids videos on YouTube[0] is another vector of techincally-not-fraudulent-but-very-low-value ad inventory. The stats on those videos also tend to be highly suspicious - i.e. basically no comments but millions of views and likes. Normally you'd call this bots, but then you realize that small children are literally incapable of commenting on all those Pregnant Elsa & Spiderman videos they are subjecting themselves to.

[0] Not to be confused with YouTube Kids, a subbrand/app intended for small children to satisfy US COPPA

What kind of content are your guinea pig eyeballs watching? I have 2 and would like to put them a video and see what they do.

Uh right now the YouTube recommendation gods are sending them Little Adventures, Guinea Pig Jungle, and LA Guinea Pig Rescue.

The latter is actually a really good treasure trove of care information, too.

I have played serval hours of an ambient ocean sounds vid on YouTube and the only advertising was at the beginning. That was years ago, but I suspect they intentionally avoid inserting advertising in the middle of such videos.

These videos actually have frequent ads throughout the video like any monetized content

And because no one is there to skip them after 5 seconds, my cat gets the full length ad served..

It's hilarious

Ironically, it seems like the best indicator an actual human is there to view your ad would be if they skipped them after 5 seconds or watched the whole thing.

Could be because some people actually watch these videos and click skip occasionally. The sea sounds was a static image so I doubt people just stared at the empty screen for very long.

That’s not fair to the kitty. Why can’t they skip ads too? Let’s get the European nut bags to draft up some ridiculous legislation like GDPR that forces advertisers to put another pop up warning potential ata they can’t skip these ads

Hmm, that raises the question whether marketing can be made cost effective. One would need to get the cat excited about a product that it can somehow instruct its human caretaker about...

Until somebody finds a way to use that. E.g. play scary ads to the cat, display ads showing cat anxiety specialists to the cat owner.

Maybe the ads are really effective, but you just need a cat translator to know how effective the ads are on your cats? https://m.youtube.com/watch?v=5JuVHCJVYf4

Actual cat translator: "I want to kill those pretty birds".

Don't undererstimate how much human will actually spend to please the feline fellow.

See I run a DSP and I'm not sure what incentive we would have to be against this? My primary goal is to keep advertisers spending, and the better their results the longer they hang around. I have a vested interest in eliminating fraud if followed to that conclusion. Then again we focus on performance campaigns not brand awareness so we're a bit a different. Most of our inventory too comes from large sandboxed audio apps so while not perfect on the fraud front there is less potential for it from less players.

Then again I am a startup and I have seen how some people at very large adtech companies operate. Less sophisticated clients (many of them with huge budgets) can't properly evaluate what they are buying and thus default to cheapest CPMs possible. It's unfortunate and a symptom of the incredibly opaque ads stack and inventory.

> See I run a DSP

Digital Signal Processor?

Another post mentioned "Demand Side Platform", which Wikipedia explains like this: https://en.wikipedia.org/wiki/Demand-side_platform

But yeah - I'm totally out of my depth here, and while the Wikipedia articles makes some sense if anyone here wanted to throw out (or suggest) a quick primer on this industry I know that _I_ would learn a lot :)

The acronym was defined in the comment to which they were replying (thus two levels above your own reply); it would have been redundant to again define it yet again.

No shade intended. Just found this amusing.

> would have been _redundant_ to _again_ define it yet _again_.

The Department of Redundancy Department informs you that was not an accident.

The acronym was defined in the post to which they were replying (two levels above your reply), so it would have been a bit redundant for them to define it yet again.

I read the thread backwards and missed the GP.

Amidst all this fraud there must be a huge opportunity for a startup with a different model that actually delivers verifiable results (eg. higher fee for a cut of affiliated sales would better align incentives and perhaps encourage a reduction of unwanted spam to end users).

But it isn't even a problem that experts don't know how to solve it. We know how to solve this issue. The problem is that at basically every large company someone's bonus is tied to keeping the current status quo. There are all these measures and statistics around ad revenue and spending, but the problem is that a lot of it is naïve and outdated, but nonetheless John's quarterly bonus is tied to a program that is defunct and wasteful, and often these are business people, they are not going to blatantly work against their own interests.

It is like with Google. They have a huge issue with the company spinning up new projects and then abandoning them at the drop of a pin, and it is crystal clear why this happens (people take on impactful[i.e. visible] projects for their promotion packet, but the incentives are for heading a project, not for maintaining it). It isn't a matter of figuring out a solution, the problem is that the problem will persist as long as important stakeholders are financially incentivized to resist change.

By top 10, do you mean google / Facebook ads? I have some friends whose online store got a lot do traffic from Facebook but I don’t know much personally

> I proposed a simple solution to ad fraud on our platform - assume any UA coming from an IP range owned by a cloud provider was a bot

Ad fraud is an adversarial game, and you can't 'solve' it with simple things like this. You'll temporarily cause a lot of pain for the bot operators (yay!) but then they'll adjust and start sending the traffic from botnets (hacked consumer devices). Which is already what they do when trying to defraud advertisers on networks that take fraud seriously (which it sounds like your company didn't).

(Disclosure: I used to work on ads at Google)

> they'll adjust and start sending the traffic from botnets

I'm not saying using a botnet is "hard", per se, but the difference in difficulty compared to using a hosting provider is significant. If bot operators are being forced to use botnets, I'd say the solution is working very well.

I ran a volunteer bot net that generated legitimate looking traffic to websites. Its incredibly easy for anyone with a large network of gray and black hats to pull it off.

My botnet was specifically for the optimization of bounce rates, we kept away from any ads of any sort, and only navigated around the internal website through clicking relative links or absolute links with the same domain.

If you wanted lower bounce rates, you had to also run this on your PC, and kick over $50/mo. It was my favorite service I ever wrote even if it did help rank websites that naturally shouldn't have been ranked higher.

This is one of the more interesting comments I have seen in 15 years of reading hn, care to elaborate anymore?

Maybe this deserves a blog post, but I'm lazy. So here is the back story from 10,000 feet.

I worked for a company that basically gave me 80% time. 20% of my time was supporting the products we already launched, the other 80% was experimenting and coming up with new products.

I was blogging kind of regularly back then (once every week or so), on a subdomain without any actual backlinks other than from a few "no-index" and "no-follow" links on social media. So technically my website should not have been in the top page for any search term and I shouldn't have had ANY traffic, but it usually was #1 or #2 for various WordPress and jQuery related searches (I had a couple of jQuery plugins and wrote some php hacks that eventually took down millions of websites), and I got 10k monthly visitors on average.

So I started looking into WHY I was the top result of those queries, and it was because when someone landed on my site with that query, the bounce rate was only 10-30% compared to most other sources being 85-90% bounce. The exact technical meaning of a bounce is lost on me now, but it had to do with how long you stayed on the site without leaving or if you click on other internal links for the site.

So I proposed to the owner of the company, I would create a "Click Faker". It would go to google on your local PC, it would then search for a term you wanted to rank, then it would navigate the top 10 pages, if it found you, it would click your link, then spend 2-5 minutes navigating around your site before closing the window.

I first tried this with Selenium (or an equivalent back then), and Google blocked it almost immediately. So then I hacked together a headless version of chromium, with some standard but randomly generated user agent, and eventually expanded it to IE and Firefox as well.

And the marvelous thing is it WORKED! And surprisingly well. BUT you had to get your site in the top 100 results, and be running google AdSense before it would work. It also proved what we had long suspected that google would rank sites with Adsense higher than a website without (probably because of telemetry data they could gather).

The concept worked, we launched the product, and got a few dozen subscribers over the next month, BUT the demand just never materialized, and after 6 months we started seeing diminishing returns as google started captcha-ing our requests, and eventually it was no longer useful and we shuttered it.

Without a large enough network of consumer PCs on consumer internet, it was doomed to fail. The network needed to be around 1000 users before it would work. We even tried giving away free limited accounts (20 visits/day for free if you ran the script, and it stacked, so if you had 3 different PCs on 3 different ISPs - home/work/mom & dads house/etc - you would get 60 visits/day).

Ultimately, I think there wasn't enough education around it, and nothing we did marketing wise really helped.

> long suspected that google would rank sites with Adsense higher than a website without

Google says that they don't do that. I don't believe Google based on personal experience and it's interesting to see that you had some experimental confirmation.

The word volunteer is what caught my attention so much, sounds like you were making a product to sell, so why use the word volunteer?

> If bot operators are being forced to use botnets, I'd say the solution is working very well.

If ad-fraud goes from "We accidentally ran these 'indexing-bots' against some websites, causing some counters to be off. Sorry about that!" to "We deployed our code to run on stolen or hacked machines through botnets paid for in crypto on the dark web", you've moved from legally gray to clearly illegal.

I can see no down-sides with such a move.

I'm not objecting to making the move (definitely do it) but about selling it as a "solution" when it's really beginning a perpetual fight.

People often don't recognize that they are in an adversarial situation, where taking a step that looks like it solves the problem does much less than you expect because other people will later counter your work.

> selling it as a "solution" when it's really beginning a perpetual fight

If your qualifying definition of "solution" is "magic bullet" then there are no solutions. Every solution is a component in the perpetual fight.

Well it:

-cuts their profits in half or more because they have to pay for the proxies(or if they own them they can't sell them since they need them)

-it prevents most low skilled people from doing it

-it prevents them from doing on it on an infinite scale, AWS have more than a 100 millions IPs, it's rare to see a grey market proxy provider with more than a few millions clean IPs, and it usually cost like 40 cents per IP, where it can be FREE on AWS

You add basic protection against headless browsers, behavioral analysis etc...And now 99% of the people who can fool you are already making 6 figures in legitimate jobs and won't risk 10 years of jail to earn just a bit more money.

When you’re competitive it can be enough to be just a bit harder to crack than the competition. The bots may just choose another advertising platform as the target.

I'd argue it's prosocial to mislead ad spenders into thinking they are reaching more humans than they actually are.

Each layer of difficulty reduces the number of adversaries you have to deal with. Check your spam folders to see how many are incredibly basic and easily caught with the simplest tools.

Moreover, fraudsters are generally lazy. If fraud that works against somebody else doesn't work for you, the fraudster won't go after you until they've saturated all the “somebody elses”, which may be a long time. I used to work in fraud detection in adtech, there was a lot of low-hanging fruit.

Whether the solution is working well isn't about whether you have caused work for fraudsters (though I'm all for that) but about whether you are actually preventing fake traffic.

Difficulty will always have a direct impact on scale: if you've significantly increased the difficulty for fraudsters that's going to have a knock-on effect on the amount of fraudulent traffic you receive.

There's no such thing as zero; a successful measure is one that achieves significant reduction.

Definitely! I was taking issue with your counting "forced to use botnets" as a success, but it sounds like you're actually saying it's just a decent proxy for success, because it is hard enough that you expect this to massively cut down on fraud?

(I think of botnets as not actually that hard a step for fraudsters, and fraudsters as being very determined, but it depends a lot on how much money people can make with fraud against your particular situation)

It’s not. You have several large proxy providers that use millions of consumers devices to proxy your traffic.

Ever wondered how free VPN services make their money? Lots of them use a portion of your traffic to proxy these types of requests.

Show the ad to identified bots, but don't count it might be a solution

This is like burglars. You don't need to scare all the burglars, you just need to make your house slightly more inconvenient to rob than your neighbor's door.

So obviously it is better doing nothing since it benefits us??

Of course not! The my parent's company is completely in the wrong, and running an online advertising business with a "pretend they're not there" approach to bots is a terrible idea. I'm just saying it's nowhere near as easy as my parent seems to expect.

> I'm just saying it's nowhere near as easy as my parent seems to expect.

Apple could eliminate most ad fraud that pretends to be its platforms, by generating tokens from its secure enclave for advertisers, and providing a REST API to validate the token is from a live device.

And of course, as you are in this industry, you know that Apple devices are the highest quality, valuable, converting clicks in the ecosystem.

Then again, PAT would diminish inventory, prices would rise, and people would get better, but nonetheless similar, ROI that they get today.

It is intellectually dishonest to make predictions about ROI. Nobody really knows. Bot farms, fraud, those are all red herrings. Most advertising has shitty creatives.

> Apple could eliminate most ad fraud that pretends to be its platforms, by generating tokens from its secure enclave for advertisers, and providing a REST API to validate the token is from a live device.

What stops someone from buying an Apple device and generating a zillion tokens?

But what if you (the party who bans the cloud provider IP ranges) don't have a majority share of the market, and your bigger competitors don't block these ranges?

Then it would be easier to bot owners to just move onto your competitors, and you would have higher efficiency than them.

What if the customers you want to attract use a cloud provider?

That's fair, it wouldn't have prevented all fraud, but it would've excised a decent segment of obvious fraud, and tbh, we weren't Google, so the transition from t2.micros to bot nets may have proven economically unviable for the cut-rate scammers.

All "well, actually" aside, my point was, and remains... ...we could've taken some action against obvious fraud, but we didn't, because the business team didn't want their numbers to go down.

Anything that makes things more expensive for the fraudsters than the defender is a win.

That might be a net benefit for the web if it’s more profitable to rent out botnets for ad fraud than sell them for DDoS-on-demand.

What he said, same disclaimer

there is some big logical fallacies in this line of thinking. better to say that google is totally fine bots traffic. that's what you said, after all.

Currently working in ad tech. Your solution wouldn’t work today because so many current and upcoming privacy solutions being rolled out across platforms are run through a VPN or VPN-like service, and many with IP rotation. Many of the bots are using these services so that their IP appears as coming from a legitimate privacy service like Cloudflare.

The privacy service providers don’t have good incentives to attempt to identify which users are bots because it would undermine their privacy claims. Tough situation.

We do think we have a solution to this but it’s unlikely to last in a privacy arms race. There’s really only one logical place this ends with the ad model surviving, and it’s not pretty.

People using a privacy service are also highly likely to be using an ad blocker, I don't think trying to identify which ones are bots is even worthwhile.

Ads that are pre-rendered by the first-party origin, embedded in the page, cannot be blocked by ad blockers, unless the ad blocker keeps a giant blacklist of rendered advertising content. I don’t know about you but I’ve been starting to see more of this kind of content. Ad blockers are a separate issue from bot traffic.

But more importantly, while your assumption is probably true today, it won’t be soon. On-by-default network privacy enhancement is coming, and already here in places. You can easily have these services in place by accident, today, without knowing the first thing about them or why you might want them.

Origin-served ads cannot be blocked by host-based adblockers, but can be blocked by hueristics-based adblockers.

I'd implemented a variant of this as banner ads began increasing in prevalence in the late 1990s / early aughts, using CSS to filter out standard banner sizes and path elements. Tools such as uBlock Origin incorporate far more sophisticated variants of this. It's also possible to block (or selectively permit) specific page elements with a CSS editor (e.g., Stylus), with web proxies (including SSL/TLS terminating proxies), or other means.

My experience is that if a site tries excessively hard to cram ads down my throat, I'll kill it and look elsewhere.

For a long time the argument was that this was a limited technically-sophisticated niche response, and that "the average person" didn't care and wouldn't bother. The increasing prevalence of ad-blocking tools and default incorporation (e.g., in the iOS platform) suggests that the barrier is knowledge and skill rather than of caring, and that people who can block ads in general will.

This itself should be a cautionary note to both advertisers and advertising platforms, as the trend will be to drive the audience bar further down sophistication and means capabilties, meaning what ad content is shown aims at less remunerative segments, and tends toward ever-increasingly noxious methods. That story ends poorly, as do most Gresham's Law dynamics.

(A similar point is made more pithily here: <https://news.ycombinator.com/item?id=33067889>)

Is the “one logical place” a Chrome monoculture with neutered extensions and even textual content being rendered to a DRM-wrapped canvas?

What percentage of people uses these privacy solutions?

It'll soon be most iPhone and mac users. The mac userbase can be ignored, but iPhones? Those are the people you want to target the most!

OP mentioned blocking cloud providers but didn’t explicitly mention CDNs, and at least with AWS their IP list allows you to filter out CloudFront. I don’t know of a list of second tier relay operators for iCloud Private Relay but I do know that Fastly[1] and Cloudflare[2] were going to be participating, and presumably other CDNs with their larger number of POPs compared to most VPS providers would be logical choices to run it. So blocking only VPS IP ranges might not necessarily affect iCloud Private Relay. 1: https://www.fastly.com/blog/icloud-private-relay-and-a-priva... 2: https://blog.cloudflare.com/icloud-private-relay/

12 year adtech veteran here. Ad fraud is a business problem, not a tech problem.

All the major solutions are well known but not implemented because of the massive effect it would have on business for everyone down the chain for advertisers.

The entire industry meets every year. They can figure this out and fix most big problems in a few hours, but incentives are not aligned for that to happen.

So basically the industry is incentivized to waste money and hide this from the boss, even within companies that are spending on ads?

Mostly yes, but also common in many other industries (with different kinds of spending).

You mean, hide this from the stock market.

Advertising is corrupt, but much of business (and public perception) is corrupt in a much similar way. What you are seeing is a lack of integrity shared by business leaders who don't ask for rigor when their minions report higher rates of "engagement". There's too much of an incentive for the people in charge to be satisfied with someone telling them that something is "up", and too much incentive for those coming up with the numbers to not perform due diligence. I've seen this multiple times firsthand.

Our number of subscribers is through the roof! Mr. Corneroffice will be thrilled! Wait as sec... 90% of them are people we gave away free trials to. Ahh... fuck it. Mr. Corneroffice isn't gonna know or care, and neither will the investors. We're BigCo! Everything we say is considered official by most people anyway. After all, Mr. Corneroffice told us "fail fast, fail early" and "fake it 'til you make it". If he secretly knew I was including the free subscribers, he'd be proud! But he can't know because then I might not get that sweet, sweet bonus. Besides, it's not a lie, right?

Mr. Corneroffice has an incentive to catch BS like that, which is that aside from subscriber numbers, he's probably accountable for retention numbers, or even revenue numbers. If he accepts the fake numbers this month, his retention numbers are going to nosedive next month and he's going to look for a neck to choke. If he's content to fake all his numbers, his boss will have a similar counter-incentive, and so on all the way to the top until you reach investors, who are strongly incentivized to detect bullshit, if only to unload the stock onto some other sucker.

If those MBAs could read they’d be very upset

Yes they would be upset because it wouldn’t work

> Then there were all the "bonuses" and "incentives" negotiated with big advertisers or publishers. We just referred to them as kickbacks

But every time I propose that market without rules at all leads to corriluption and inefficiency, people get up in arms

Markets are, in a sense, defined by rules.

Perhaps the argument goes better if you say competition without rules leads to mafia, ie., people stop competing and form gangs -- since this is more efficient for them. These gangs then monopolise trade, and use violence to stop people competing with them.

If they disagree, you can always point to the state itself as an example. But likewise, without a winning mafia to set the rules, others arise quickly. No group wants to compete, the want to have already won.

> No group wants to compete, they want to have already won.

Or to put it another way: a capitalist fears nothing more than a truly well-functioning market.

They don't "fear" it any more than you or I "fear" going into work every morning, but competing honestly in the market is how they benefit society, and monopoly rents are the carrot that they're chasing. It's up to us to get our money's worth out of them.

No, because a capitalist (in the friedman, etc. sense) distinguishes between free markets and ancap-competition. A free market is a construct of a state with a functioning legal system, requiring as it does, property rights (, limited liability, etc.).

Capitalism is not ancap libertarianism. Indeed, it seems quite clear that ancap is an iterative contradiction, as "rule-free competition" cannot produce free markets.

There is no market that does not adjust itself with new circumstances. Crypto prices fell and GPUs are now affordable without any government intervention. Amazing isnt it ?

No one claims markets don't adjust. The claim is that unregulated markets often don't adjust for the benefit of the society at large. It weren't as amazing when the GPU prices soared due to mining, was it?

That's not quite right. The affordability increased slightly with the falling prices. But the merge basically opened the floodgates. It was a change in technology rather than a "market change" (as in business incentives balancing over time) that caused it.

> ". Crypto prices fell and GPUs are now affordable without any government intervention"

https://en.m.wikipedia.org/wiki/Market_failure

https://en.m.wikipedia.org/wiki/Cartel

https://en.wikipedia.org/wiki/Bullwhip_effect

https://en.m.wikipedia.org/wiki/Demerit_good

https://en.m.wikipedia.org/wiki/Information_asymmetry

https://en.m.wikipedia.org/wiki/Wikipedia:Avoid_Parkinson%27...

And countless others.

This assumes these things don't operate in an environment affected by multiple nation states and their specific regulations

The markets are working.

1. Even with the bots, the price made sense for advertisers for so long because of how cheap ads were. 2. Its starting to not make as much sense and now we are getting these accusations thrown at platforms etc.

TLDR: Markets work but just not real time and not very perfectly. The issue with rules (esp excessive rules) is the corruption of the rule maker. To enforce these rules, the rule maker tends to have the ultimate capability for coercion.

And If you think "voting" prevents this corruption then you are r*ar*ed. Let alone the corruption of the masses which btw has resulted in the greatest acts of violence in history (USSR, Cambodia, Nazi Germany, 20th century Japan, French Revolution..)

> and now we are getting these accusations thrown at platforms etc.

What?! This problem existed for so many years! And it wasn't some niche knowledge but pretty mainstream already in 2014 (Veritassium video "Facebook Fraud" with 6M views):

https://www.youtube.com/watch?v=oVfHeWTKjag

I think the point is, the problem isn't a problem if it is priced into the product. It's cheap as hell to show ads on a cost per delivery model because the value is low, in part because of bots, in part because people don't click on ads much and in part because it's not clear how useful they are for building brand awareness. If there are more bots than previously suspected, sites will need to charge less to account for that. That's how the market functions, not a sign that it is not functioning.

Sounds like to you, even the casting couch and sexual favours are an example of working free market.

Something along the lines of secretary that provides blowjobs for the CEO ensures they are always at the top of their game?

>When I worked in adtech, I proposed a simple solution to ad fraud on our platform - assume any UA coming from an IP range owned by a cloud provider was a bot and don't show them ads, don't count their impressions.

Wait, is that not automatic on every single Ad network out there? WTF?

How long ago was that?

I run wireguard on an OVH box that I've had for years as my VPN. I just checked and I still get Google ads.

So it's not even automatic on the largest ads network out there.

Personally, I'd love it if using a VPN meant that advertisers didn't even try serving ads to me, but unfortunately that's not the case.

What, you just expect Google to show you a giant red box saying "Whoops, we detected that you were trying to do an ad fraud! Better luck next time!". Google will happily show you whatever ads you request, but they won't count them as monetizable impressions and they won't charge advertisers for them. The goal is to make it impossible to tell as a bot operator whether your fraud was successful. So why give bot operators more info about how to evade your detections?

They should not charge people for them but they definitely do.

I mean yeah but Google doesn't count those clicks for ad revenue at all

or maybe just on OVH or smaller provider,but AWS ips are definitely flagged for advertising

It is wonderful that after cable TV, where a few thousand of individuals' viewing habits determined who watched what, we now have an absolute stellar system where everything can be measured, yet, nobody wants to see the real numbers. :D

...

> The business team vetoed that, it would adversely impact their numbers in the current quarter, and well, a significant proportion of their total comp was bonuses tied to those numbers

Furthermore they were not spending their own money to buy those ads. Shareholders money is (almost) for free. In my experience companies run by the founders with little or no investors are always more careful about the money they spend.

> Then there were all the "bonuses" and "incentives" negotiated with big advertisers or publishers.

your firm's higher ups received bonsues/incentives from publishers and/or big advertisers directly?

No. When the ad spent increases the sales person, manager etc. get a bonus. And you don‘t do anything to improve quality which would reduce ad spend.

What kind of rational system would ever incentivize spending rather than results (aka sales?). Sounds like the marketing equivalent of being paid by the line of code.

> "What kind of rational system would ever incentivize spending rather than results (aka sales?). Sounds like the marketing equivalent of being paid by the line of code."

OP works for an advertising company. They get paid by clients to run advertising campaigns. Employees of the advertising company get bonuses for closing high-profile deals with clients. The employees are incentivized to continue bringing in more revenue streams and continue closing more high-profile deals.

The deals couldn't have been made in the first place without some form of trust between the parties. As long as the clients are seeing results (aka return-on-investment, ROI), the trust is maintained, and the deals are renewed. Different clients have different budgets, and have different ways of measuring their ROI (sales are one parameter, but there can be more). No one would be buying those advertising services if they wouldn't be seeing ROI.

It is generally acknowledged that "advertising works". In reality though, there are lots of factors at play.

Marketing ROI is very tricky to measure. On the other end, you would see that the more you spend the more customers you have. The question is: are those new customers due to your increased spend or not? There's all sorts of attribution magic that goes in the background to trace back adcampaigns to converting users (especially across devices). That's why the article mentions that most of the "big fraud" that was uncovered by simply turning off paid campaigns without conversions dropping. That's a pretty big giveaway that something is amiss.

There's an old saying in advertising that we know half of our budget is wasted, we just don't know which half. Ad fraud and bot traffic live in that area, kind of as expected waste.

Then there's the other issue - branding campaigns also cost an arm and a leg, and the most you get is vanity metrics like reach and likes and impressions and whatnot, which have dubious correlations with reality but they look nice on dashboards.

Its not tricky to measure at all. Thats why we have marketing mix models for like 20 years if not more.

In adtech you have to spend all the budget a client gives you. They get very pissy if you don't and it's not unheard of for clients to penalise you several multiples of the unspent budget.

Yeah, because their bonuses are based on spending all the allocated money (which is insane, so standard corporate behaviour).

> What kind of rational system would ever incentivize spending rather than results (aka sales?)

Is that a sarcastic question? I guess the part about humans creating a rational system is a dead give-away.

It takes a really good salesman to convince customers that they need to pay for something they don't need.

> Advertising has always been corrupt. Bots just corrupt it a little further.

I had friends in college who did door to door sales for a major cable company. At the beginning of every week they had to stop by the office to pick up a stack of flyers and at the end of every week they had to mark off the houses they visited on a map. They got paid per flyer distributed and every flyer was distributed directly into the trash can.

They got paid, no one got harassed by door to door sales people, and the cable company got their marked off maps. Everyone was happy.

> and don't show them ads

Wee, I'm tunneling by browser session through IP address on a cloud provider so I don't have to see ads!

Meanwhile, ads are shown to my cracked residential router which has been enlisted into a botnet.

When I worked at a company that got like almost 100% of their revenue from ads, we got a very serious warning from an ad agency to stop serving our ads to bots. They gave us like 7 days to fix our traffic or we'd get removed, effectively killing our company. Scary time to be an engineer that morning! That was like 6-7 years ago, though! It's kinda crazy that things have changed this much.

I’ve worked for a company that used an Azure hosted internet gateway/firewall that all their sites connected through using an MPLS VPN. So there are definitely some non-not users with traffic coming from there. But yeah, happy enough to not see ads when piping traffic through those ranges!

So how about starting by not actually blocking the bots, but publishing two numbers, the original calc including bots, and the adjusted with bots removed. Try to estimate said number for competitors as well. Then after some years try to move to using mainly the second set of numbers.

Fleecing the investors, who are fleecing the workers and consumers to entertain their gambling addiction.

"assume any UA coming from an IP range owned by a cloud provider was a bot and don't show them ads"

As someone who is trying to get into google ads for their startup, what's the best way to do this ? Is there a nice Google Ads 101 other than Google's own tutorials ?

> it would adversely impact their numbers in the current quarter, and well, a significant proportion of their total comp was bonuses tied to those numbers

So much about how terrible things are in general can be tied back to this one simple concept.

Ads are cancer.

We did do that when I worked in adtech. However even 10 years ago there were massive networks of residential proxies.

Ad businesses are in the business of selling ads. Not of selling less, but good, ads, if they can sell more but bad...

Great insight, thanks. Are there any 'honest' ad networks with integrity out there in your opionion?

that almost sounds whistleblower status. If they are a public company, and they are knowingly using bad data to look more financially successful, there may be something there.

If you are paying any bill for ads, make sure you only pay for conversions. Ie. unless the customer actually buys your product, then don't pay anything. Eg. instead of paying $0.002 per view of your ad, or $0.10 per click of your ad, pay $6 per person who clicks your ad and then buys your product.

That way, ad fraud doesn't affect you. Plenty of ad providers allow such an option. Internally, they convert your payments per conversion into payments per click and then payments per impression, but importantly this shifts the incentive to get rid of bots to them, not you - you no longer care. The provider also has an incentive to send you customers who will actually buy your product, and to send the bots and web scrapers to their other clients.

A problem is that deciding whether or not a customer was converted from an ad viewer is done by the people you pay for ads. Perhaps the conversion numbers are slightly inflated but in general it’s just really hard to tell. If you show a lot of ads to potential customers then chances are, most people who buy things from you saw your ads. Does that mean they came to you because of the ad? I think even if you had perfect information it would be hard to decide. There is some chance that you just have an added sales tax where you send $6 to each advertiser every time you make a sale. I don’t know how much (eg) google care about deciding whether conversion numbers are correct. But how do they even decide if the numbers are correct and what stops them from being biased towards reporting higher conversion rates?

Most online businesses have multiple ways to track conversions... It will become pretty obvious if Google says you sold 130 widgets but your warehouse guys only delivered 60....

Likewise, lots of businesses have a different landing page for people coming from an ad vs people from organic search and people coming from referral links/shared URL's. The business counts how many people visited each of those.

They'll know if the ad providers stats are lying...

The issue is not that you sold 60 widgets but Google tells you it was 130. The issue is that you sold 60 widgets with ads but you would have sold 59 widgets without ads: are you making sales because of ads or are you showing ads to your would-be customers.

As parent said, you can see that one sale was made from ../product_google_landing_page and 59 sales were made from ../product_landing_page

That still perfectly solves the bot problem.

It doesn't the problem of optimizing your ads, but it's ok to only completely solve one problem and let the others on the same situation from the beginning.

If I understand the premise, it should be simple enough to disaggregate my organic vs. paid-for conversions.

For brand lift campaigns though, without enough historical sales numbers, I can see how it’s harder to tell if your ad money is being well spent.

you don't even need a different landing page, though that makes it easier. Depending on the ad platform, you can (in the US) cookie them and connect their behavior after the initial click. For the first session, UTM's are often sufficient.

> A problem is that deciding whether or not a customer was converted from an ad viewer is done by the people you pay for ads.

If you rely on their reporting, yes, but you can aleays set up your own basic attribution (with some caveats around ios traffic due to the ATT changes, but in that case the reporting comes from Apple). There are also third-party providers that can do attribution for you, but somewhat costly.

This makes it sound so much easier than it actually is. Tracking where customers from on iOS device is extremely difficult. Now spread that difficulty over 10 different advertising channels (youtube, Facebook, direct visits, google, etc), each with millions of dollars in budget, and the problem quickly becomes impossible to track yourself.

Sounds like a legitimately good application for block chain - both parties can have access to the immutable transaction history. Similar to how Walmart applied block chain to inventory delivery disputes.

How do you verify the information being put into the blockchain is correct?

> If you are paying any bill for ads, make sure you only pay for conversions

Others have already pointed out that this is actually impossible. The big ad platforms charge on impressions you can setup any targeting you want but they charge on impressions that's it.

So Facebook charge based on impressions, but they rank based on conversions which tends to reduce the impression billing problem.

Conversion isn't the best metric though for many ads. The big advertisers don't care about conversion because they are not trying for customers today, they are trying for mind share so when someone thinks about buying they are on the list.

Conversion is a lot easier to measure than "buys your car sometime in the next 3 years", and is worth measuring. However it is often only a proxy for the real metric.

Question: how do they know if you're playing fairly and tell them of the conversion? Also, do they get the customer's personal data that is involved in the purchase?

They don't. But if you don't pay them enough, ad-companies will stop sending you any customers at all, and instead put their efforts into catering to other higher-paying clients. So you are incentivized to make a good ad campaign that generates a lot of sales.

That would be the correct answer. The ad-companies are auction based, so they shouldn't have any problem with it.

But somehow, people report differently. (Personally, I haven't looked for a while, so I don't know.)

Usually there's a magic pixel that you embed on the checkout success page or a callback you invoke. I suppose people could cheat, but if your ad never converts, the network has no incentive to run it.

This has major privacy and GDPR implications though.

I don't think ad companies care too much about that.

Such as?

They generally just trust you.

Audit: They click 10 ads and buy themselves and then check you compensate them for all 10 sales.

Conversion is not even sufficient. Conversions tell you how many people went through your ad and successfully bought something, but the value of ads is *how many people who weren't already going to buy something bought something through the ad*.

E.g. people might be seeing an ad for Target after typing "target" in Google. The number of conversions through that doesn't matter and should be discounted. People might be seeing an ad or hitting an affiliate link for a website they already use regularly, if they go there hours later it doesn't mean your ad is effective.

This has been around for a long time, but which major ad providers offer this?

Google adwords does:

https://support.google.com/google-ads/answer/6268632?hl=en-G...

That’s a targeting strategy. You still pay for every click, whether they convert or not.

Not true. We only pay for conversions in a few of our GA campaigns. I don’t think this options is available to most advertisers though.

Even before online, newspaper would "distribute" their papers to hotels and college dorms to "invisible" subscribers to increase the count of their subscriptions. And let's not even get into how radio and TV viewers are counted via Nelson.

Noone, except scammy "affiliate" post view rackets.

You can set max cpa bid strategies, but that's still paying cpms.

Even if it would be easy only pay per conversion, you would create a different type of perverse incentive in which ad platforms would exclusively want to target consumers their models attach a high preexisting (before exposure to ads) probability of conversion. That is OK for lower funnel tactics but it would severely reduce your ability to put your brand in front of a new audience.

Maybe there's two sets of ads being discussed here: exposure/top of tunnel and conversion/bottom of funnnel?

Yeah, I long thought that these kinds of fraud are ultimately filtered by measuring the return. Paid to Zonama web shop to show your wares, but all visits are from people comparing prices with the in-house brand? Well drop the shop. Bought an ad campaign, see zero conversion? Why would anyone continue buying it.

After speaking to many sales reps for ad networks I have decided to to so,however, these Cpa deals have negative value, say if you provide a service that will or with not have better customer life time value than the cost for the cpa, what would be a useful idea about that?

> ...make sure you only pay for conversions...Plenty of ad providers allow such an option

This is untrue.

Can you share which services allow to pay for conversions?

Would be curious to here, too, as far ad networks go.

There are many affiliates for online casinos, crypto exchanges etc, they usually have a cpa + revshare deal.

They scrap the non converting audience, its vastly more expensive, but I imagine converting an impression from an add somewhere on an app or such to a paying depositor with life time values north of 200usd is just not going to be economically viable.

So far, all I have spoken to ended up a cpm model slightly shuffled.

A few years ago, I tried to help out a locally owned gym do an experiment with Google Ad-Words. The place small storefront in a run-down strip mall straight out of the 1960s, but it had very good equipment for people that were in the know and it was 24 hours.

The owner was a nice guy but he relied entirely on word-of-mouth and his technical skills extended to making expletive laced, 128-point-font signs reminding people to put equipment away in Word. I got the sense the business was not going well after a protracted battle with a former business partner, where the business partner just wanted to liquidate the assets and call it a day. He managed to buy the business partner out and continued to operate the place---sans a few treadmills. I decided to try and help him out by volunteering my time and some money to see if a small, targeted ad buy from Google could help him out. It sure beat finding another place like this.

The dashboard made it look very targeted geographically. But, when I investigated where they were all coming from it was 95% international and from affiliates listings from very low quality sites with no information on them. A month and $300 later, I discontinued this experiment because the traffic was entirely bot driven.

From that experience, I've come to the conclusion that online Ad Tech is 99% a scam.

Online ad has negative effect. I never trust products or services that are showed up in online ads.

Everyone who thinks this way already blocks all ads.

In this case, it was the slightly less worse case where the ad was meant to be informative instead of manufacture consumption. If you were looking for a gym that met this criteria in this locality, finding information about this particular gym would have been limited to people willing to go through several pages of search results, or looking through specialized indexes, or just happen to be walking past the storefront on the way to the bar or gun shop next door.

This was the exact use case that online advertisers invent to justify that they're not all bad, and they failed to come through.

> Yet the digital advertising industry has grown so accustomed to working with inflated numbers that few are willing to unmask the fake clicks

That is a funny reason for why this phenomenon might have grown so big in the first place. The people in the ad-buying company who are in the best position to 'smell' when the numbers are off are also the ones most likely to individually benefit from inflated numbers (KPIs, number go up,...).

A principal-agent problem at its finest, that unfortunately might drag some white-collar workers into being at least morally complicit in cyber crime.

From my definition of white collar (office worker sitting on chair in front of computer all day), and blue collar (factory/shop/traveling/working with hands not computer), I would have thought white collar workers were the ones most likely capable of committing cyber crimes.

Two points

- I think it's worth pointing out anyway, because in my experience white-collar workers - especially outside of areas like finance, where they do get a lot of compliance training - often seem to be of the impression that their work is far removed from what could eventually be considered crime

- Hands-on workers can absolutely be part of cyber crime schemes. Money mules, various packing, collection and delivery activities, intimidation,... can be important parts of the cyber crime 'supply chain'

Reminds me of Graeber's bullshit jobs theory. We have people generating fake clicks so that the ad departments at big companies can feel important.

On top of that, IIRC he mentions advertising among the "zero-sum" variety of bullshit job. Not all of it, but a great deal is "we're spending more because [other big brand] started spending more, just to keep from losing market share".

They have award shows that give out awards based on these numbers. It's really silly.

If you ask me, we should simply delete the entire advertisement sector—influencers included—and start from scratch. The amount of wasted resources (i.e. attention, money, electricity, bandwidth, time, smart people) is just insane! The benefits I see are mostly lot's of jobs and means for the biggest businesses to sell more stuff. That can hardly be worth it.

I know I'm dreaming and it will never happen, but still... let me dream.

Here's a plan: Tax it. Tax the everliving hell out of it.

The problem isn't the existence of advertising, it is that there is so much of the crap advertising. Taxing ads would raise the price of advertising, which would lower the ROI on advertising and force ad buyers to be more careful about which ads they buy. Crappy advertisers who are not actually connecting buyers with sellers would tend to go out of business, shrinking the industry. The remaining ad industry would have to create less ads with higher quality to compete in the newer high cost marketplace.

Basically, the idea is to move the supply curve on the (on the supply and demand curves) to a place where they meet at a higher price and a lower quantity. See for example, https://www.economicshelp.org/microessays/equilibrium/price-... (scroll down to Tax Incidence). Note that for advertising the 'producer' on this graph is the company making the ads and the 'consumer' is the company buying an ad for their product. The part about moving the curve to a lower quantity is the key! The goal here being to reduce the ratio of 'attention used' to 'product-consumer fit created'. We would have a smaller, better ad industry. We can't have that with current market conditions; competition won't allow it.

The nice thing about a tax plan is that it is pretty easy to sell people on the idea of 'take money away from these people who annoy all of us and give it to [insert social good here]' . The best people in advertising would likely benefit from it as well, as they would likely stay in business and have better margins in a smaller industry with less competition.

So, keep dreaming, but feel free to spread this idea around. Maybe it will grow by word-of-mouth.

> If you ask me, we should simply delete the entire advertisement sector—influencers included—and start from scratch. The amount of wasted resources (i.e. attention, money, electricity, bandwidth, time, smart people) is just insane!

I agree there is lots of wasted time out there. But the advertising paid for Google, Facebook etc. It pays for Chrome, Android, React, PyTorch etc.

If free software that runs nearly everything could be done without advertising, so could these.

That's a small sacrifice for ending pervasive thought manipulation and surveillance.

The software would be cheaper if financed without ads

I'm just amazed ads work at all. Anytime I receive an ad I didn't explicitly request, I boycott that company. I'd have hoped enough likeminded people in the world would naturally defeat the ad sector, but evidently I'm an outlier. I'll never understand why the average consumer rewards and reinforces ad spend

> Anytime I receive an ad I didn't explicitly request, I boycott that company.

Can we see your list? Is it in Git somewhere?

I do something similar. I block every account that shows me ads on Twitter and Instagram. I wouldn't say I boycott them but 99% are companies I wouldn't buy anything from in the first place.

> I'm just amazed ads work at all.

Same, I don't recall a single time an ad made me want to buy something, if anything I think "if they have to shove this in front of my eyes and I don't already know the product it probably is bad or useless to me"

Actually curious, do you hold the same standard for print media (a newspaper/magazine ad), out-of-home (billboard), and television advertising?

How do you primarily discover businesses and products?

The internet and all related technology is largely funded by advertising. It would indeed be very fun to watch that all disappear.

Major websites are ad-funded, the internet isn't. Networking technology, IPV4/6, http, ssl and most website frameworks aren't funded by advertising ( and even if they are eg react, they won't disappear.)

Let FAANGs fail. Let them try to disguise ads into content, fail and go off the deep end. What is there to lose, really? Search? Someone will take their place. Instant gratification online ordering (not funded by ads but fuck it) orchestrating modern slavery? We'l be fine. Facebook products? Might be a bit of a panic as chat for 50% of the world population goes down, but we'll manage. RCS is good now, matrix is right there, we'll manage. TikTok? We'l be fine.

Pretty much any thing funded by advertising is not vital to society (otherwise they could make you pay for it) and like half of it is parasitic.

Photo & video hosting for all of your major life events that delivers the content to all of your friends and family?

Is there any cloud that is ad-supported? AFAIK, all of them (including google) is supported by charging directly for it and/or bundling it a different purchase (physical phone, software subscription, etc.)

I'm referring here to Instagram / Facebook. How are you going to send your wedding or vacation or dog photos to your network? Mass email? The social platforms have a utility for most people at this point.

Ads, military and porn, the three pillars of the internet!

But that advertising money still come from somewhere and someone, right? Surely there is some other way to make those someones to spend that money on the internet and all related technology?

That someone is the one watching the ad by buying products the person sees in ads

Back in 2015, there was this "The Website Obesity Crisis" talk that claimed [0], among other things, that in fact most money in the ad industry is coming not from the ad-seeing consumers, but from VC investors. Has something changed in 7 years? Or was it never true in the first place?

[0] https://idlewords.com/talks/website_obesity.htm#fatads

> If you ask me, we should simply delete the entire advertisement sector—influencers included—and start from scratch

The thing is Alphabet and Meta revenues are 95% ads. All the big players are either products to show ads or products to gather data to show more ads

Well, I have to say though, those little influencers do deliver results.

I have spoken with many of them and they come up with ideas regarding gamification, retention and acquisition being far superior to the ad networks.

They have a community and I would hire some of them on the spot, no need, though, some of them earn my salary many times over.

The ad networks might be ok for flight by night ops like selling viagra copies on porn websites, but they will always be that annoying pop up, I am not sure it they count towards the result if people have ad blockers.

Or let me ask, which businesses can actually convert measurably from ad networks? Assuming life time value of customers being more than 50 usd.

Its always sold as for "brand propagation", an excuse to inform you ahead that no conversion at all will happen.

If you have a fast paces business which relies on large throughput and a constantl flow of new customers it is not very useful.

I know facebook could convert very well, but they will not allow any product.

Agree. And this is yet another proof that lots of meanningless jobs exist!

The digital economy would doubtless take a hard hit if we knew the true scale of synthetic engagement out there. It's just easier not to look too hard. Reminds me of a joke:

"Doctor, my brother has lost his mind. He think he's a chicken." "Have you tried telling him he's not a chicken?" "No. You see, the thing is, we really need the eggs..."

> No. You see, the thing is, we really need the eggs.

Isn't that the opposite of the problem here? If the bots actually generated sales, the adbuyers would likely be quite happy. This seems more like the Emperor's New Clothes, if you're looking for a metaphor.

I read that as his brother doesn't actually lay eggs, they're just hoping against hope for it.

Same as the ads and the bots.

"We could really use the eggs" might have been a better way of phrasing it then.

ok, yeah, reading it that way it is a quite an appropriate metaphor

No one really wanted to do much digging when the market was good, but now that the market is bad, expect a lot of digging and a lot of suing.

Must be a European joke. In the States I'm guessing it costs more to see a doctor than just go buy eggs.

And, yes, i know my euro-taxes essentialy pay for my doc, but a week in hospital that cost me €13.50, I'll live with. As well as a year's worth of post-care and meds....€0.00.

Eggs are getting a bit pricey now, though.

> “After all, who cares if big brands waste their money showing ads to bots?”

I'm not so sure this affects only big brands.

I've experienced that in march 2020. I ran ads in a very popular platform and was fairly happy about the results -- modest, but steady.

When the pandemics hit, my target-market shifted their focus away from my sort of product, and such shift happened almost overnight. Since I used to get a reasonable amount of real visitors, I thought "well, I believe the clicks on my ads will decrease sharply now".

Guess what? I kept getting the same amount of clicks on average, and my budget of course kept burning at the same rate. The only difference was that the "visits" started to last only a few seconds, the pages were not scrolled anymore, and the "visitors" never sent an inquiry.

Your bounce rate went up. That's still consistent with your target audience losing interest. It may be that they care enough to click on your ad but don't engage with your content. It could also be that the ratio of bots increased at the same rate as customer interest wained, or it could be the ads were never effective. It kind of doesn't matter. As an ad buyer, the only thing you to need worry about is whether the value of the traffic coming from the ads is worth your money. Uninterested humans and bots are of equally little value.

They cared enough to keep searching for my keywords, to click on my ads, but then they realized they were not interested and bounced?

Also, like I said, the traffic the ads generated was good (it generated prospects fitting the profile I was targeting, and I was happy with the numbers), so yes, the ads were effective.

My point is, either the traffic got worse or your ad is no longer effective and a higher bounce rate can indicate either. Bots don't shoulder aside regular users, so regardless of whether there are more bots, there are less users. Otherwise you'd have seen traffic increasing but not conversion. If traffic is flat and interaction is down, you've lost interested users and you need to look at your creative, or your targeting, or possibly consider whether any design or content changes on your landing page could be responsible. It could also be that you are getting outbid and your placement is now not catching the best customers. The fact that what worked before doesn't work now is totally normal. Customer tastes change, competitor behavior changes, the traffic supplier is changing. Ad buying is a constant battle.

And have you checked what was the ip range they came from?

> The case started when Uber pulled all online advertising and discovered barely any drop in app installs or sales.

Uber is a bad example I think. Not only they've been in the news so many times advertising is probably irellevant, there is an actual "natural" ceiling for demand for Uber in an area.

They may have to compete with ads in places where they have competition, but not overall.

> The technology to detect and block bots already exists

Yeah, right. When I close the site because i can't be arsed to look for palm trees in 120x80 photos, I'm a bot.

I do think they are a good example, however , maybe they ran those less quantifyable brand name campaigns.

There are many reports out there confirming the same.

Dont give the ad networks a carte blanche sometimes they themselves are hijacked by bad operators.

It seems there is very little done a mitigation or prevention of bad traffic and a lot of sending beginner sales persons to the front lines.

I mean, they do provide some estimates for conversion rates, but never will they put that on paper. Where are their numbers for traffic cost even coming from if they do not know?

When they say lets run a test campaign for 1 or 2 k before I commit to 5 times that, I ask them what have my comoetitors done, do you have feedback?( I do, am on friendly terms with competitors) they always say they have no insight(is that so?). All I know that some of them create either zero or so much fake traffic that I have to justify the 400 percent page views month to month to some payment partners.

Honestly, I'd much rather they show their ads to bots than to humans. Hopefully, the day will come when every human is using a competent adblocker, but the ad industries numbers don't go down thanks to the bots, and people can get paid for content that everyone enjoys for free.

But there’s no free lunch. If businesses realize there is a 0% ROI to ad spend due to all the traffic being bots, they will stop buying ads. At which point sites will just migrate to a subscription model.

That's the endgame! Then websites will have to compete for your dollars, and the cream will rise to the top.

If businesses actually had the ability to correctly calculate ROI to ad spend, we wouldn't be talking about ad fraud.

Wouldn't the incompetent businesses keep buying ads, helping them eliminate themselves from the market?

No. Ads will be inserted server-side. Like in the old days of print magazines and TV.

It's possible to block TV ads: https://next.taiv.tv. Taiv is a startup for bars and restaurants that uses a model which basically detects when there are sudden color shifts and lets them insert their own ads or get paid to insert someone else's ads.

Although I don't know how you'd block print ads unless you had some kind of AR headset that blacked them out.

What incentive will the site authors have to produce content and design suitable for self-respecting humans?

Time and again people fail to realize that if they install an adblocker, they remove themselves from the target audience and thus from influencing the content and quality of web pages.

“Victimless crime” - I think not. Most affected are small businesses and startups with precious marketing dollars that can no longer afford to run an App Install campaign that reaches real users instead of bots. The big dogs can brute force their way through this. I’d venture to say that Ad fraud is a moat in itself.

Almost all advertising like grabbing attention without prior consent is already inherently corrupted, bots are just late fallout.

If they're getting decent ROI on their ad spend, what does it matter? TV ads were normally shown to empty rooms while their occupants went to make themselves a drink. Still worth buying.

It mainly has an effect on content creators who rely on ads for revenue. They're the only ones providing real conversions but they're forced to share their revenue with bots emulating fake conversions and so their total slice of the pie goes down.

Conversions should be tracked differently to (and weighted higher than) impressions, though?

Because diverting money from actual productive businesses that employ lots of people into the pockets of people who are going to spend the money on Lamborghinis is net bad for society? Why care about insurance fraud if insurance is still profitable? Someone is paying for fraud and it’s basically the rest of society in the form of higher costs.

Because the bad actors are bad and extracting illicit gains from the industry. Removing the fraud will lower ad costs, cost of goods and as a result the cost of products for consumers.

I now realise I should've read the article, instead of clicking back as soon as it auto-played a video. I didn't realise it was deliberate, and fraud.

It's amazing how much compute time - both hardware and humans - is dedicated to figuring out how to show which ads to people.. and it turns out that many of the "views" aren't even people.

there is no incentive to figure out the solution to this problem. The largest players in this space (Google, Facebook) require advertisers to spend money to make money.

If the big players were to solve the ad fraud problem, beyond a superficial level, it would hurt their bottom line —- which hurts stock price, which is tied to compensation for their employees.

Ad fraud is usually a niche conversation that doesn’t get much traction (esp as most publications are reliant on ads), but the twitter / musk lawsuit is helping more people pay attention.

That's not my problem. Advertisers are part of the problem, they take in huge sums of money and feed the Google+fb monopoly, without ever knowing how their money is wasted, they "trust" them. There is nothing more opaque than google, the company that controls both the supply and demand of search ads. If you willingly participate in that market,i m sorry, not sorry

I am not an advertiser or anything, but businesses dont "trust" them. The moment businesses see an alternate option with more transparency and somewhat similar results - They will jump ship. Although we are way far from anything like that coming up (on that scale)

There were a bunch of ad marketplaces that were pretty transparent - (buysellads.com? not sure, and many others) because they were pretty basic, you were buying ads directly on websites. Google is not transparent on the publisher or the advertiser side. Advertisers "bid" on "keywords" with the amount that google suggests (pulled out of their ass? who knows). On the publisher side, if you add "auto ads" on any website, a bunch of ads appear all over the place, most of the time breaking your page layout. You are given an RPM for your ads that , as far as i know, could also be pulled out of their ass. They control the inflow and the outflow, and adjust according to their cashflow needs. Advertisers like them because they provide a lot of bloated dashboards with fancy graphs as proof of their work.

I did always find auto-pricing for ads weird. Why would I trust the one entity that stands to gain from an increase to set my bid? The fact that it's impossible to verify the veracity of the system makes that a no-go for me, but clearly not for everyone.

Its not trust so much as monopoly access to the digital ad market. Google has so much prime inventory (search results + youtube) that it's impossible to avoid it if you want any kind of scale for your campaigns.

If anyone wants additional insights, I spoke with an ad fraud researcher on this topic during the pandemic. He had a lot of interesting things to say.

https://www.mql.fm/002-60-million-60-billion-ad-fraud-questi...

Just listened to this - it was great. Augustine Fou was a fantastic interviewee and it made so much about the ad fraud market clear. I have bookmarked it by submitting the link on HN - well worth a listen to folks.

Thanks! Always happy to hear some of the stuff I do is well received!

No worries - if you have a moment please post about the hideen side of making the podcast (equipment, finding / persuading guests, fitting it in the day etc) Always interested in the real how to

I’ve written a few bits about that here https://www.jacquescorbytuech.com/writing/starting-a-podcast...

"How bots corrupted advertising"

Ah, yes, "advertising," that fine institution representing the best the world has to offer in purity and honesty...

I worked for a print ad business many years ago. They were basically running the same business model with the paper. Ads were placed in glossy and cheap/free pos magazines (cheap traffic) that mostly landed in the trash after collecting dust on the shelf. The bots consisted of distributors claiming high quality distribution while throwing the paper right into the trash. Everyone knew and hardly anyone cared because incentives were mostly aligned (mostly big brands).

I was hired to help with the transition to digital. Guess how that worked out. Next I tried running my own online marketing business. I thought not deceiving customers and actually providing real value was something clients would pay for. Some did, but competing with all the fraud became to tiresome eventually. Not going to work in this industry every again unless there are alternatives (includes scrubbing toilets)

Companies should be doing testing to check the efficiency of their ad spending. If they're happy with the Spend/Sales ratio then it is irrelevant whether they pay $x for 1000 impressions->10 real views->1 sale or $x for 10 impressions->10 real views->1 sale.

If they are not doing their homework, that is on them.

And frankly, 90%+ of advertising is a blight on the environment whether it's online or broadcast or physical so who cares.

Years ago our company was running an ad campaign on social media. Curious on the results I spent time looking into a randomly selected pool of the “users” that had been clicking on our ads. It quickly became clear that at least half (if not a lot more) of the clicks were from what appeared to be fake users or bots. We shut the whole ad spend there off immediately.

I sometimes when an advertisement annoys me, I punish it clicking on it, clicking around on the website, maybe adding items to the cart or going through half of the click funnel, and then I close the tab and never come back.

It's great.

You could automate this with a script running at certain intervals.

Some will throw you a captcha sometimes, but not all and not always early enough.

I did not abuse this, I have observed this when developing a scraper.

Test headless before deploying was my most valuable take away.

I think fraud is ubiquitous in advertising and often the publishers are to blame. Frequently on mobile devices the ads cover the content and you wind up clicking on the ad when you are trying to hide it.

Also I think it is not so accidental that mobile web sites have so many layout shifts because each layout shift is a chance you click on the wrong link and ka-ching!

Layout shifts on ad sites are more usually related to the fact that ads come in various sizes and to make sure one is always displayed (and the company makes a tenth of a hundredth of a local denomination of currency) so you can’t prefix the ad space to take out of the page.

Then UX and your ad team will duke it out for if it should be a fixed size or should the company make more money and the user always loses :(

I dunno.

Half of the problem is knowing where people will click, the other half is knowing when.

People into Black Hat SEO circa 2008 would often practice "blending" where the point was fitting the Google text ads into places on your site where people would be likely to mistake them for content. This is related tactically to a certain kind of landing page which seems to make no sense, but some users will get confused and click on a random link which goes "ka-ching".

In that kind of site all the links are paid links, on the real web it is not too different, it is maybe 10 or 20 paid links per real link. Fitts' law is on their side. I guess the only reason they don't put mini-banners between the navigation links (as Fitts would have you do) on the bottom of the page is that goes too far and gets you kicked out of the ad network.

I have noticed the layout shifts that happen on certain sites when I am just about to touch a link on my tablet and thought... "I wonder if you can predict some clicks based on the accelerometer trace" but then, nah, they'd have to ask permission to use the accelerometer wouldn't they?

What blows my mind is how bots even care about this stuff.

Back in 2009 I started my first tech startup with a friend. We crawled the net and stored the locations of and contents of online ads. Aside from the mistake that this actually costs a lot of money and we were too young to effectively raise money, we also made another mistake: Ads were about to get very dynamic. Not just ads rotating on a site, but search retargetting and hypertargetting were about to take off.

So I did what any reasonable founder would do after learning that the business just wouldn't work. I turned down a just offered $200k (huge at that time) and closed up the business.

Years later I'm learning that bots are still interested, and I suppose there is some signal here, but its minuscule compared to what was available back then. Plus, there are fewer and fewer out-of-app websites people spend there time on these days.

I dunno. I'm happy that startup never went anywhere. It would have sucked to be swept into the ad tech industry over the past decade.

"How xxx Corrupted Advertising" is a really strange titlre.

It seems to assume that Advertising being corrupted is a new thing?

And that the Elon guy is somehow a genius for noticing a part of the problem?

The whole history of advertising seems to indicate that in fact Advertising was designed to make us buy things we don't need, not designed to talk to the better angels of our nature.

Edward Bernays, aka Freud nephew, was a pioneer in Public Relations (called Propaganda at the time, they were not afraid of the word) and one of his greatest success was to convince women to start smoking en masse. He did that by branding cigarettes as feminist "Torches of Freedom" https://en.wikipedia.org/wiki/Edward_Bernays

If that's not corrupt, I don't know what is.

I mean the whole industry would look really really different if they had to give up misoginy. Think "Cheat On Your Girlfriend, Not On Your Workout” (Reebook) and 42.000 other examples

Strange choice to stress misogyny. Women see attractive models in advertising, they think "unachievable beauty ideals, I feel pressured". I see the same, I think "it exploited my biology to grab my attention, damn it worked again". Is it misogyny or is it mis-everyone, let's make money?

> Women see attractive models in advertising, they think "unachievable beauty ideals, I feel pressured".

And they are right.

> I see the same, I think "it exploited my biology to grab my attention, damn it worked again".

And you are right.

> Is it let's make money?

Yes

> Is it misogyny or is it mis-everyone?

It is misoginy, and misoginy is always is mis-everyone - but a tiny elite who benefits from it.

Advertisers for example.

TODO: Add "why-not-both?" meme here.

> Is it misogyny or is it mis-everyone

The word you might want here is misanthropy "a dislike of humankind".

That's a useless word though.

It's like "all lives matter". Nobody but psychopaths disagree so why do people bring it up? What's the point?

Mostly to defend the status quo where some lives matter less than others.

> It's like "all lives matter". Nobody but psychopaths disagree so why do people bring it up?

I only have anecdata to go on, but in literally every case I've seen, the reason has been "because of (and in response to) someone else implying otherwise".

I'm pretty certain that the advertising of women's things using beautiful, scantily dressed, women must work, it's being going on for long enough.

Obviously, the primary effects are those that impact women, but things can annoy you and still be misogyny.

Ad tech is applied sociology. What you're talking about is a different advertising industry, one that deals with human psychology.

Ad fraud is essentially incorrect/fraudulent sampling that leads to biased results. (Usually with a hidden motive.)

Which is why in traditional advertising sampling and data collection is done by government-mandated entities. Digital advertising has been "disrupted" by entities like Google and Facebook, but eventually we'll get there too.

Advertising to designed to make you choose a particular company over another when you go to buy that thing you don't need.

This is a classic problem when relying on proxy metrics instead of real-world impact.

Proxy metrics are convenient, they are also easy to game.

With enough time, any system involving humans will be corrupted.

But in the grand scheme of things, I don't care about adtech being demonetized in the long run.

I've seen some paid solutions for this like https://www.clickcease.com/

Does anyone know of any free open-source solutions?

And how much of tech valuations are based off of the bs revenues this genetates? Id venture a lot since a lot of tech valuations are levered up based on other tech valuations in a daisy chain

This. I believe this is what really has google and meta in a panic.

Network effects for these companies have been built upon funding from advertisers.

They have failed to build consumer business models (B2C) on top of these networks.

There are clocks ticking as advertisers seek direct routes to customers.

People just like free stuff at the end of the day, and if you are zuck or Sundar this is absolutely terrifying. You have to either give something to advertisers or convince people to pay for “free stuff”.

In all honesty, the one bet google had right was gaming, consumers aren’t shy about opening their wallets.

People may debate paying a couple of dollars for twitter blue, or sharing a Netflix password BUT will happily fork over 100s of dollars for games.

Google however expected overnight success and now they going to try squeeze ads and all their freemeum toys for all the juice they can get.

Same goes for meta, which at one point was poised to be the king of online gaming.

I think a insane fraction of google revenue comes from lawyers, insurance and car sales. But maybe you are talking about smaller companies that buy ads being overvalued.

I worked for a while on an online ad measurement product, specifically on identifying and helping our clients avoid "invalid traffic", which is almost but not exactly a synonym for "bots". Reading this article, you'd come away with the impression that companies doing this work don't even exist? (Besides the venerable Dr Fou, who excels at getting his name in print whenever this subject comes up.)

I can't vouch that all the third party ad measurement vendors do a good job -- in fact I know from experience that some of our competitors did not -- but it was weird to not see any acknowledgement that there's a whole competitive field of the adtech industry with lots of talent behind it working specifically on this problem and which any serious advertiser, publisher, or exchange in the open web ad ecosystem is already a client of. (The "walled gardens" like Google/FB/etc also work with these measurement vendors, though the measurement itself is neutered to the point of uselessness, basically just passing their definitely totally trustworthy numbers through to another dashboard. Now that's a topic I'd like to see a journalist dive into!)

>>In June, the Association of National Advertisers (ANA), a US industry group, published a blog post that estimated that ad fraud is costing US advertisers $120 billion each year. Hours after it was published, those statements were removed. John Wolfe, the ANA’s director of communications, tells WIRED that the figures were removed because they were out of date, but declines to provide any new figures.

Right, because the up-to-date fraud numbers are higher.

>>But the industry is letting ad dollars flow into the pockets of cybercriminals, he adds, who can then use it to fund other illicit activities. It’s a major problem, he argues. “One that no one talks about, no one writes about, everyone thinks it’s someone else’s problem.”

Exactly, everyone is incentivized to accept the inflated numbers, and disincentivized to look at reality, so the result is unsurprising.

>>Fou believes the industry was corrupted around a decade ago, when a series of opaque middlemen entered the scene. “Prior to that, advertisers would buy ads from publishers like The New York Times,”

It's said in software that 'Everything can be solved by another layer of indirection'. Everything evidently includes inserting a fraud layer.

Solution? De-scaling and going back to advertising directly with specific platforms instead of ad "networks". What about paying for conversion instead of clicks? That concept seems to have completely disappeared. Anyone inside know why or what could be done to get back to that?

As a small biz owner developing physical products, I'd be interested in advertising, but I sure don't want to blow $$thousands out the door in hours for a mere illusion...

Even the smol internet is not immune. At Floodgap, I estimate about 2/3rds of the Gopher traffic is automated, based on the nature, spacing and progression of IPs through the site. Some of it comes from HTTP->gopher proxies that allow web bots to index through them (can't imagine how much money that can cost), but much of it seems to be connected to no known proxy.

I’m going to ask the dumb question: if you had a newsletter or some online store that you wanted to promote, and you had at least some budget for ads, what actually works in terms of ads?

All anyone talks about is how ads never work, but I know a couple of people who, for example, thrive on Facebook ads. Are they outliers then?

Broadly - conversions that you can validate. It's kind of that simple. Depends on the channel, as well. For LI and FB/IG, have tight targeting, tighter tracking, and really really good creative. As long as I'm making more than I spend, there's something there. Every now and then we turn off our ads for a month or so and analyze traffic and sales to make sure those ads aren't capturing leads that would've come in organically. Advertising is work - you have to constantly investigate and experiment and optimize and refresh. Then do it all over again when that particular well runs dry.

I have a similar question.

I am running a FOSS community in my local language. And we have a contest where we pay the best 5 articles in our forums every month.

We would like to market our contest but how can one market their website without supporting bigtech?(keep in mind, not in English)

Twitter is our biggest platform and the one we are going to try first, but that's big tech and not privacy friendly anyway?

With private access tokens demo'ed at WWDC2022 [1] as a better replacement for captchas, this problem can be reduced to a great extent. As this becomes an IETF [2] standard that Google and Microsoft and many others will also implement, this will no-doubt be used to solve bots-in-ads problem.

1. https://developer.apple.com/videos/play/wwdc2022/10077/

2. https://www.ietf.org/archive/id/draft-private-access-tokens-...

Botnets create traffic from trusted devices. Would the icloud token provider un-trust compromised devices? Sounds like a nightmare.

People that buy ads for companies want to keep their jobs too. And like most jobs the more money the company gives you responsibility for spending the more they pay you. Don't see why they care how many bots see their ads.

There are probably more problems about advertisement effectiveness than bots cheating the system. This article from The Correspondent was on the frontpage a few years back and I think about it a lot: https://thecorrespondent.com/100/the-new-dot-com-bubble-is-h... HN: https://news.ycombinator.com/item?id=21465873

To be fair, the bots were mostly impressed.

This is sorta a self-correcting problem because bot traffic is imputed into the price/bids for ads. Bot traffic lowers ROI, hence lower bid. If ads converted really well and were 100% human traffic, they would be much more expensive on a CPC or CPM basis than they already are (even though they are plenty expensive still). Evidently in spite of bots, companies still derive enough value to keep spending tons of money on ads and pay really high CPM and CPC rates.

To me it feels like companies have these ad budgets, and just feel the need to spend, spend, spend without measuring any sort of ROI.

Example, almost every GM commercial these days are for EV cars\truck that you cannot even buy, some you cannot even put down a deposit or the "pre-order" is meaningless (like refundable $100 for the Lyriq, but no specs, colors or pricing has event been announced). It is like GM needs to spend its marketing budget no matter what.

I run a startup that heavily relies on ads, and I can tell you for sure that we spend and measure ROI in detail. We have metrics that tell us exactly how much we need to spend to convert a user into paying user, so that we can budget ourselves accordingly; I think most low/mid tier companies do this. Of course, when you look at real giants, its a different story as the ROI is harder to measure.

I think for bigger brands like Coca-Cola it becomes harder to measure. Because it’s not like you see an ad for Coke Zero on Facebook and order it right away, whereas you might if there was a new company with a cool new product. That’s why they run “awareness ads” to make sure they stay in people’s heads but I don’t think anyone knows exactly how effective those are.

They have specific goals to spend x amount on y product every month. SO that is not a mystery. Also, there are video game sales of 10s of millions where only 10% ever finish the game. Yet the next version has the same mechanics and sales are higher and still 10% finish the game. Interesting...

Does anyone know which companies media methane was partnering with?

The following article explains the mechanics:

https://www.anura.io/blog/king-of-ad-fraud-sentenced

But who were these partners who were paying media methane for (fraudulent) clicks?

there's two ad clients: small business who want cheap ads to sell, and large incumbent who want expensive ads as a barrier for new competitors.

the players (dsp, ssp, etc) all want expensive ads, more profit.

the second group will gladly pay the inflated cost because that's what they want anyway. and they have weight to force even google to submit to known bogus accreditation that let all bots through.

the first group have to pay because they have no leverage and can't force Google to have a valid accreditation.

ads sales pay huge 20-30pct commission because everyone knows some of that goes back to the large client buyer.

sucks to be a small business.

Companies are measuring the results of their advertising pretty closely. I would (possibly incorrectly) assume they're still hitting their metrics despite serving their ads to lots of bots.

If a company counts as a person for purposes of filing lawsuits, why doesn't another company count for purposes of viewing web pages? Why do they have to be human? That's racist.

Add it to the huge sums that companies pay for misclicks. Traditional banner ads are a total scam.

Outside of advertising, how many business expenses regularly have single-digit effectiveness rates?

Hey, what if we added an x-i-am-an-ad-bot header to HTTP?

I'd happily set it if it meant lower ad revenue, and that way, the actual click fraud bots wouldn't be committing fraud.

Not surprised. I worked with companies that paid affiliates for purchases done by load testing in test envs. Budgets is not always a good thing.

When using Facebook ads I am fine with that.

First I argent for likes, after getting a lot of likes on the ad (from bots?) I switch to targeting the right demographic

I wish they only showed ads to bots. I find myself vigorously clicking "skip ad" on youtube. Is that "ad rage"?

That is known since the 1990s. And yet, nothing changes.

This has been obvious since the 2017 report by Homeland Security about the Russian disinformation campaign:[1]

> Bots are computer algorithms designed to execute specific online tasks autonomously and repetitively. They simulate the behavior of human beings in social networks, interacting with other users and sharing information and messages. Millions of bots spread information on social media platforms such as Facebook, Twitter, and Instagram. According to a 2017 estimate, there were 23 million bots on Twitter (around 8.5% of all Twitter accounts), 140 million bots on Facebook (up to 5.5% of all Facebook accounts) and approximately 27 million bots on Instagram (8.2% of all Instagram accounts). These three platforms alone contained 190 million bots—more than half the number of people who live in the entire United States.

I've been wondering (and posting on HN about it) this exact topic for years. All those bots are pretending to be online users and all of them doing everything possible to circumvent detection, including consuming ads.

I would bet that most big advertisers have contractual rebates and reimbursements covering the fake ads - there's no way they'd stand for this sort of graft filling Google and Facebooks bank accounts while they paid millions for ads to fake users.

It's the smaller advertisers that get really shafted. Some small franchise or website buying a few thousand ads only to get little in the way of return on their investment. They have no way to detect the false impressions and rely on the various ad networks to detect bots, of which they have zero incentive to do.

I'm amazed it's taken this long for any news to come out about this, and even more amazed Congress and law enforcement haven't been involved yet. Again, look at the numbers above - this is a huge number of fake users and that only covers the disinformation efforts. Imagine the various networks out there that are focused purely on generating ad revenue and nothing else.

1. https://www.dhs.gov/sites/default/files/publications/ia/ia_c...

Are the bots impressed though?

I been saying it, ain't I been aging it?

Advertising is corrupt by nature.

Huge companies that exploit their workers are themselves being exploited. Sounds like maybe there is a god.

Ad fraud overwhelmingly victimizes companies that pay for ads many of whom are small businesses whereas the big companies like Google benefit because it means more volume.

So in the end we're all being exploited by advertising?

Reply on news.ycombinator.com