Uber discovered they’d been defrauded out of 2/3 of their ad spend | Hacker News Reader

Uber discovered they’d been defrauded out of 2/3 of their ad spend

twitter.com

1,793 pointsrbanffy891 comments

A related article is https://www.forbes.com/sites/augustinefou/2021/01/02/when-bi.... (It was posted at https://news.ycombinator.com/item?id=25620707, but the topic and discussion are substantially the same as this one, so we merged them.)

Note: this thread is being paginated, so you need to click More at the bottom of the page to read the rest of the comments. Or via these:

https://news.ycombinator.com/item?id=25623858&p=2

https://news.ycombinator.com/item?id=25623858&p=3

We did a similar kind of experiment some time back and found both FB ads and Google ads were doing jack shit for us. Google was better. FB ads were absolutely useless. So we stopped our ad spend. We observed no change in our customer acquisition.

Another way these ad engines extract money out of you is by allowing competitors to bid for your brand name keyword. Try searching for Ozonetel on Google. There will be six competitor ads and then our website will show up :)

In our early days we used to be scared and bid for it.Lost a lot of money doing that. Then we talked about it and stopped. Again, no change in customer acquisition. Turns out, customers scroll down the ads and still click on our website.

Are our competitors getting some of our traffic?

Of course.

But my business is not built on the assumption that people will not find my competitors. Its built on the fact that people will explore options and try to find the best fit for them. So its ok.

> Try searching for Ozonetel on Google. There will be six competitor ads and then our website will show up :)

DHH (of RoR and Basecamp fame) has been beating the drum about this for a while now.

Yes, we have also been beating the drum since a long time. But unfortunately our drum is not as big as DHH :)

Curious: why do you feel your competitors shouldn't be able to do this? If your customers intend to go directly to you, why do they not go directly to your site but instead search for it on a general search engine? More generally: if I search for "apple", what should the first result be? Wikipedia or Apple.com?

    why do they not go directly to your site but
    instead search for it on a general search engine?

Do you honestly expect people to remember the domain names of the brands of everything they have or are interested in?

My grandmother still searches Google for "google images".

As I have mentioned, I have no problem with my competitors being able to do this. The problem is how Google buries ozonetel.com below everything else. It should be absolutely fine if Google puts Ozonetel as first search result(because there is a brand and company called Ozonetel), and then puts the ads below that. Its win-win for everyone.

"If your customers intend to go directly to you, why do they not go directly to your site but instead search for it on a general search engine?"

It's because now mostly people(including me) search for a brand we know and click on the site. Mainly we type in the address bar(without .com) and press enter. The default search engine brings up the list and we click.

>The problem is how Google buries ozonetel.com below everything else. It should be absolutely fine if Google puts Ozonetel as first search result(because there is a brand and company called Ozonetel), and then puts the ads below that.

Sure, and I want to watch a show and then have all of the commercials run at then end after I turn off the TV.

>More generally: if I search for "apple", what should the first result be? Wikipedia or Apple.com?

So what would your answer to this be?

"if I search for "apple", what should the first result be? Wikipedia or Apple.com?"

Apple is a common noun in the English language. So I would assume the first result to be Wikipedia or some site which gives the best details about apples. The second result could be Apple.com or whoever has applied for the keyword.

Thank you for answering. But that is inconsistent with what you said about your brand appearing first before any ads. In Apple’s case, are competitors allowed to bid on this keyword, since it is a common noun in English? Why or why not? I assume your brand is not a common word in the English language but what if it is in another language? As you can see, I can continue muddying the waters until it’s not so trivial to make a decision on these things — especially automatically.

Now you’re comparing apples and Ozonetels. Ozonetel is not a common noun.

I'm behind a Pihole and the top result for Ozonetel is ozonetel.com. On mobile, there is a "People also search for" bar that takes me to Google searches for their competitors, but it's mostly the Ozonetel home page, LinkedIn, Twitter, Crunchbase etc.

Normal users don't type domains, they type where they want to go into the address bar. my aunt will type "google" into chromes url bae to bring up the classic google search page she knows, and then will continue to type something like "jcpenney" into that box.

If the site is in her browser history she'll get an autocomplete suggestion for jcpenney.com .

One justification might be that it creates an environment where everyone but the ad provider loses.

If, hypothetically, Google was more pervasive than it already is, and made their ads more intrusive until consumers overwhelmongly clicked the highest priced ad for a given keyword, and Exotel bought the Ozonetel keyword, Ozonetel would have no choice but to buy ads for its own name. They could also buy ads for the Exotel keyword. The two could get stuck in a bidding war, wastefully sending money to Google when the optimal thing would be to just send users to the site they're trying to reach.

We're long past the point where you should expect users to try to guess domain names and to pretend that Google is just another general-purpose search engine. That universe could be nice but we don't live in it.

This is exactly what happened and a bidding war started(long time back).

And then we decided to step back from the bidding war. One reason to do so was because we did not have VC funded money. If we had VC money, maybe we would have continued the bidding war. But glad that we could make the right choice.

As a practical matter, search is how 99% of everyone gets to everything. Even if the browser fills in http:// for me so I can just type a name to get a complete URL why would I risk misspelling or mis-domaining a URL and possibly getting a fake or time wasting similarly named site? It doesn’t really matter whether the apple wiki or apple.com comes first because there is no chance I will confuse the two (unlike, say competing online furniture companies).

If you trust Google more than your browser and your own self, surely they deserve to make money off the hard work they put in to earn your trust.

1) Not if that goes to a place that isn't the best answer for it, as expected by a normal person not paid by Google or purchasing Google advertisements. Might this threaten the ability of Google to screw with smaller companies for money? Darn, that's a shame. They'll have to make only a transcendental amount of money instead of all of the money.

2) Related to the above: Google has decided they want to be a critical cog in the body politic. We as citizens get to have a say in how critical cogs in the body politic function. If Google doesn't like that, Google should not put themselves in that spot. We can help them with that if needed.

> If your customers intend to go directly to you, why do they not go directly to your site but instead search for it on a general search engine?

Plenty of people use the search engine as their homepage or reflexively open it when making a new tab, and searching for the name of the service (you don’t have to remember the TLD) becomes a habit.

When Google used to show organic results front and center and made advertising conspicuous, this was reasonable. Now that there is a pile of undifferentiated advertising on top of organic results, it's borderline fraud.

> When Google used to show organic results front and center and made advertising conspicuous, this was reasonable. Now that there is a pile of undifferentiated advertising on top of organic results, it's borderline fraud.

One of the things I would like the antitrust investigations to dig into is Google's decision to make ads look more like search results, after a long-held principled stance of clear differentiation between the two.

IIRC, Google never did comment publicly on that particular design change.

I'm sure any internal discussions on the topic will be fascinating reading.

I'm surprised that's legal, since they're literally referencing somebody else's trademark.

I first ran my such Google Ad in 2005. And yeah our competitor at the time was pissed.

Google makes money with these so they allow it as long as the ad text does contain the trademark.

You need to have a thick skin in business. What matters are whether consumers are happy.

There's nothing illegal about referencing somebody else's trademark as long as you're using it to talk about their product.

Having a trademark doesn't forbid your competitors from talking about you.

What I think should be made illegal is that Google (and others) is essentially collecting another toll even when users enter in the brand name of the target business in the first place. That is:

1. User types CompanyFoo (whose real URL is www.companyfoo.com) into their address bar, which lands them on Google (because Google owns Chrome and pays billions for the privilege on other browsers).

2. Since Google will sell this keywords to others, many businesses always bid on their own brand name to ensure it comes up first.

3. Thus, an ad for CompanyFoo shows up first, even though www.companyfoo.com is the first search result, so when user clicks on the CompanyFoo ad Google takes its cut. Again, this is also because Google has done as much as possible to make ads as visually similar to SEO results as possible - just the teeny 'ad' banner, compare to the big yellow box of early Google.

I don't have as much objection for other companies buying branded keywords, but the way Google has set up things up they are taking their cut from everyone even when they are giving very little value.

You are under no obligation to use Google's browser or search engine if you believe it doesn't provide you with a valuable search experience.

You have no control over the browser or search engine your potential customers use.

What do you expect should happen? The browser should automatically load the first google search result? Google should prevent companies bidding on their own name? Or that they should make bidding on their own name free?

How about they clearly separate search results from ads and don't put the ads first.

There was a search engine that got really popular doing that. I think they were called Google..?

Well, I think there are two options.

Either Google should take a note from the old days when they were not rent-seeking jerks but did no evil and actually distinguished search results and ads by placing the ads to the side bar.

Or their practical search monopoly and all the profitablity from there should be regulated to oblivion.

That's just what I think should happen. The sensible me understands that obviously neither is going to happen.

I'm pretty sure that if I typed foo in my browser back in the early 2000s I'd go to foo.com or www.foo.com. Now I type foo in Chrome, see the logo for foo, but end up at a Google search page. This is very clearly a step back in terms of user experience.

In modern Safari this pattern will often take you directly to the website (or some other, non-Google website when it guesses wrong). When it works it's very nice.

I wonder how it works, if its powered by google or if its their own search engine in use.

This Google squeez also happens when a company spends on offline ad campaigns as often the first action in response to an ad is to type the company name into the browser. The searcher then arrives at the Google gate where friction is applied to divert a percentage of those to click on competitor online ads. Google is also taxing global offline ad spend.

The problem is that a lot of times they act like they are the trademarked brand in the ad to trick the user. This is unethical and should be illegal.

That not only should be illegal, it is. Using somebody else's trademark in a way that could decieve a customer into thinking your product is theirs is like the definition of infringement.

But from what I've seen, the vast majority of ads that incorporate a competitors trademark are not doing that, and making it very clear they are advertising an alternative to the trademark being referenced.

This seems like a positive thing imo. A lot of adverts are formatted as "Tired of x? Check out Y!". I see no way this is infringement and its a net benefit to the world since it shows people new things without tricking them.

Yeah, there's lots of places where IP law is pretty dumb, but this is one of those spots where they struck a pretty good balance and things are mostly working well.

Tip: URL suggestions are rarely advertising, and pop up faster than the results page does. Check the suggestions before searching.

Too much to hope for: a Chromium fork to have the search hint dropdown provide suggestions + Duck Duck Go results. And maybe block the enter key unless a resolvable address had been entered to prevent inadvertent Google searches.

I had a hunch and looked at MDN's list of web extension APIs.

https://developer.mozilla.org/en-US/docs/Mozilla/Add-ons/Web...

It seems like you could implement your described behavior as a cross-browser extension, for everything except Safari (and Android Firefox -_-). Easier than forking Chrome! :)

Thanks. It does seem tantalizingly there. Now that I've pondered it a bit more I think what I want is a one-way direction from Google's suggestions into an ad-free browsing space. The omnibox is a bit of a misdirection.

What's the point of listing Edge or Opera? Is the same damn browser as Chrome after they forgo their engines.

Search me. Perhaps there are other factors influencing compatibility; perhaps these lists were created before the forgoing of the engines.

> searching for Ozonetel on Google. There will be six competitor ads and then our website will show up :)

Tried it. One ad. From ozontel.com

Same

I saw Dialpad

It appeared 2 ads for me, the first one very big with 3 clickable links

One ad from dialpad for my try.

I've got two ads: salesforce and weeloy.

ozontel.com doesn't exist. Is that a typo or did a competitor really buy ads to a nonexistent domain?

ozonetel.com

good eye. That’s a typo on my part

Hah, I saw Ozonetel Alternative - Dialpad® Official Site...

So Dialpad.

Pretty sneaky Dialpad lol

It's not sneaky to compete in a marketplace.

I'd argue it is sneaky to put the competitor's name first, as most people will click quickly to move on to the website.

Disagree in this specific situation (generally yes) - "XYZ alternative" is about as clear language as you can get. Nothing sneaky about it.

People click through search results really fast. "XYZ alternative" would be much more acceptable if it came after the real name, but putting "XYZ" as the very first word is guaranteed to cause confusion.

The result can be split two ways. Either the advertiser wasted money on a click because the consumer will seek out the true XYZ. Or the consumer will be satisfied with what they've clicked and choose to buy it.

Hopefully the metrics show that the latter is mostly happening.

It's only fraud if the advertiser counterfeits XYZ's brand on their website.

I'm worried about trademark law, not fraud per se, and consumer confusion is the gold standard.

The ad can violate trademarks even if the site behind the ad is squeaky-clean.

Trademark law does not stop you saying "my product is better than X". It only stops you saying "my product is X".

Generally, yes. But if a large fraction of people are getting confused in a specific context, then you can say "don't word it that way", even if the wording is true when you consider it in a vacuum.

If putting the other brand name first causes consumer confusion in certain places, that's a problem, even if they're not lying or even doing it on purpose.

If ozonetel is trademarked it's generally against Google Ads TOS for competitors to use the term in ad copy. File a complaint.

We recently launched in the US. Nice that Dialpad has noticed us. All the best to them :)

You still see ads on Google?

Ok you guys made me curious, so I tried to. I got Ozonetel at the top, zero ads! I win! :)

All results related to Ozonetel and zero ads

https://imgur.com/a/q70nShf

But we all tried it. So that was either a mistake or pretty good advertisement from the op.

Strangely enough my ad blocker had been disabled and I didn't get any ads for Ozonetel. Perhaps it's because I'm searching from Philippines.

We're probably not in your target market but most people here probably know anything marked with "ad" is very likely to contain malware and probably won't click it even if they think it goes where they want.

This isn't true. Google search ads are rarely malicious - Google does a pretty good screening them.

I disagree. When searching for crypto related terms, more often than not I see phishing sites as ads than legitimate ads.

What are some examples?

The whole cryptocurrency field is scamming. Phishing is just a little bit more obvious about it.

That is inevitable since 99% of crypto sites are scams...

I have recently been getting straight up spam (“you won 1 million dollars!” type) as ads in Gmail. No idea how these managed to show up when these would have been easily marked as spam if it was an email.

So would 99.9% of ads. In fact, that’s what spam is—an ad.

The difference is that Google got paid for one and not the other. Pay them the protection money, and they’ll let you through.

Google ads are just links to sites, there’s absolutely no malware.

While Google does admittedly do a much better job preventing malware-ridden sites from running ads (for very long) than most platforms, to say there's "absolutely no malware" is a bit disingenuous when there do exist some sites with exploits and/or malware that manage to get ads in search results. The ad itself might not have malware, but the site it links to sometimes does.

Yeah that's what I meant, clicking the ads will always be just as safe as visiting the sites themselves, you can't really ask for more than that.

EDIT: I suppose you could ask for more, you could do as you hint at and not even link to sites with malware I suppose.

Why you think that website are unable to control malware or other harmful payloads?

In form of directly executing code or something encouraging to download and run programs containing it.

If anything, FB ads were worse than useless for my business, all they attracted was 90% fraud.

Looking at what business Ozonetel is in (i.e. sales are directed towards business owners), I'm not suprised FB ads were also worthless for you. I get the sense FB ads are really only good for things a consumer might impulse buy (e.g. clothing, games, small gifts, etc.) but anything where the user is probably going to do some more in-depth thinking about the purchase, they're beyond useless. Which pretty much makes sense - even if you've drilled down your targeting well for business decision makers who may sign up for Ozonetel, it's highly unlikely that decision maker is going to really want to know more about Ozonetel while they're looking at cat vids and fighting with crazy uncles about politics.

Facebook ads are the worst from a users perspective. I bought a floor mat through them once and for the next week all the ads were for floor mats. I just bought one I’m not interested in buying another. I bought a PC chair recently with my credit card, now I see a bunch of ads for PC chairs, again, I don’t need another one.

Same thing with kitchen appliances, but not limited to Facebook. I'm not going to buy another refrigerator for at least 10 years, but still I'm treated as Mr Fridge.

Honestly this might be a good thing, it seems to show that ad networks don't have or at least don't provide your actual purchase history to the ad space buyers because given that info they probably wouldn't be spamming someone who just bought a $thing with ads for $thing_but_from_us. As annoying as it is it's probably better than the alternative of ubiquitous access to your purchase history.

Yup, this is the problem. There is no centralised log of what people bought, which is what you'd need to actually fix this problem.

A good argument, but it happens on Amazon when the search, research, and purchase were all on Amazon.

ML is pretty shitty at causal inference

Yup, I get "buy it again" prompts on nearly all of my recent purchases, even when they are all durable goods that don't expire or require restocking, like say a tablet case.

Hmmm, that's really weird. It's probably an edge case that would be complicated to change in that case (never underestimate the power of inertia, especially in ML systems).

I feel like it might be almost trivially solved with a “buyer only needs one [for X months/years?]” property on the items.

It would likely have to be added by a human, but still: there are a relatively small number of items that people only need one of.

Well we know these places buy your credit card history, so it’s more proof that their ad system is shitty and their customers should make a fuss.

Amazon enjoy that kind of thing too. Look at a product page, Amazon ads become for that product or similar ones. Buy it, and... they still want me to buy another one. And they wonder why people use ad blockers...

I've seen the theory floated that people return so much stuff that shortly after buying a thing you are indeed most likely to buy another, slightly different one of the same kind, and that explains the weird categories where it happens. (and for lots of things Amazon sells recommending similar items does make some sense, although I would expect a bigger spacing in time sometimes.

Also that brand satisfaction is related to how it performs relative to other offerings. If you just dropped $44k on a new vehicle, they want to make sure that you're seeing fancy cars, thinking about how nice and luxurious they are, etc.

And the emails! I keep getting suggestions I just brought a copy of a book why on earth might I want to buy another copy the next week.

One way to avoid/reduce this is to always place orders from a private browsing/incognito window, and to never log in using your normal browsing sessions.

Well no one has a plausible explanation on it yet but with some of the highest A/B testing ratio and best Data Scientist Amazon too seems to think this works.

And yes I agree it sucks.

Hah. I remember shopping for a wireless router and seeing things like “shoppers who bought this item also bought...”. Yes, we all buy three slightly different models of routers. Good job Amazon.

Actually there is. There is a certain chance that you're unhappy with your fridge and you'll return it. So the assumption is that you are more likely to buy a different fridge than any other item. That's why they show you other fridges.

I have an established personal opinion on this, and the opinion is that I'm glad that things are this way. It means that ads are pretty dumb and relatively harmless so far and I can easily ignore them. Floor mats? Tough luck bud, I just bought one.

Now, when the ads begin predicting what I would want the next day—that's when I start worrying.

Don't use facebook but surprised how my spouse finds instagram ads super relevant and interesting enough to check them out almost 100% of the time. They are mostly fashion/travel related that she is interested in checking out,if not buying.

Reading this thread I really start to feel people don't understand what fraud is. You shouldn't be calling it fraud when in the next paragraph you say that FB does work for impulse buys and not "ozonetel"

Just because your ads didn't convert, or 90% of the traffic you buy didn't convert, doesn't make it fraud. Fraud is malice, with the intention to defraud someone and not deliver what you promise, with intent. This is completely different to you setting a target demographic on FB and Google and it not converting.

I imagine most people calling fraud in this thread aren't actually talking about fraud, just non converting traffic. There could be a million reasons for this. You haven't set up conversion tracking, your targeting could be off, or your product or landing page is just crap. Calling it fraud is not helpful.

Here are the California jury instructions for fraud and deceit: https://www.justia.com/trials-litigation/docs/caci/1900/

There are definitely multiple categories this would fall under.

> You shouldn't be calling it fraud when in the next paragraph you say that FB does work for impulse buys and not "ozonetel"

I don’t believe they were saying the adds themselves were fraud, merely that they had a higher than average propensity to attract fraudulent customers.

I think you misunderstood what he wrote.

My business also sell only to other business, and Google kept calling us nagging us to do some "experiments".

One of them involved enabling "display network" ads on mobile.

Like the guy side... it "attracts" fraud.

So, what kind of fraud?

I am not talking about no conversion or no click, I am talking about all our clicks, or almost all to the point it was impossible to tell the difference, were fraudulent, for example most our ad spending was going into a musical keyboard app for kids, that has nothing to do with my business, despite me blocking the category multiple times, the app in question just kept being reuploaded in different categories to avoid bans.

When I went to check out the app Google Play page, it was rated 1 star, and lots of complaints similar to this:

"App is just a fraud, it keeps opening ads in the middle of the song while my toddler is playing to force my toddler to click the ads, then it automatically closes the page that opened and keep running, using all my battery and making my toddler see weird stuff."

So yeah, it is fraud, it is not "failed conversion", it is an app conning toddlers into clicking my ads that are selling stuff that shouldn't be anywhere near a toddler, both the parents and my business were being scammed.

You've misunderstood what I wrote. When I'm saying that "90% of what we got what was fraud", I mean that 90% of the traffic that did convert from FB ads were users that were trying to scam us (it was a financial services app), I mean literally steal money from us with things like stolen credit cards.

The general user quality you get from Facebook Ads is atrocious.

I work for a retail app and can usually tell when we have or haven't been spending on FB ads when I check our usage numbers for the week.

They do work but like anything it's about choosing the right tool (platform) for the job. We wouldn't run ads on LinkedIn and B2B SAAS companies shouldn't advertise on FB/Instagram.

> For ex, one ad network launched “battery saver” style apps in Google Play, giving them root access to your phone.

> When you type the word “Uber” into your Google Play, it auto-fires a click to make it look like you clicked on an Uber ad and attribute the install to themselves.

Isn't above an example of a criminal intent of deliberate fraud?

Yes, and? How on earth are you going to even explain it to the police, let alone find which jurisdiction to use?

You are buying it from some vendor, their company is invoicing you. You can use either your or their jurisdiction, whatever fits better and you can also sue them directly.

You explain it like:

>For ex, one ad network launched “battery saver” style apps in Google Play, giving them root access to your phone.

>When you type the word “Uber” into your Google Play, it auto-fires a click to make it look like you clicked on an Uber ad and attribute the install to themselves."

You don't have to be a genius software engineer to understand that.

Usually you will have to choose the jurisdiction of the party you want to take to court. This is part of the reason some regulated industries require you to have presence locally.

Ah right, usually you can sue on your home turf only if there's a treaty for it (e.g. when both parties are based in EU).

Ad fraud is HUGE online.

In display, at least. It's not massively common with either Google or Facebook (assuming you avoid their networks), which is part of the reason that both of them make so much money.

I don't think that's true, just that fraud is different on those platforms. I suspect it depends on how you define fraud but certainly if a platform tells me I'm bidding to show adverts to customers who match criteria X, then I expect them to match that criteria.

Unfortunately, that's frequently not the case.

https://pubsonline.informs.org/doi/abs/10.1287/mksc.2019.118...

Your reference talks about a different set of entities, the information from most data brokers is inconsistent and often wrong (much like display).

Because they have first-party information, neither Google nor Facebook suffer as much from this problem (although the most common school attended on Facebook is Hogwarts, so obviously they're not perfect).

> Because they have first-party information, neither Google nor Facebook suffer as much from this problem

Maybe not from the problem outlined in the study, but fraudsters certainly create fake high value "profiles" that they use to commit fraud on those networks.

I spoke with an ad fraud researcher recently for a marketing podcast I started and he spoke in detail about this fraud is carried out. IIRC he referred to it as retargeting fraud.

Sure, but both Google and Facebook have whole teams devoted to stopping these kinds of fraud.

Like, they are definitely not perfect, but they are so much better than the rest of the industry.

I am aware of a bunch of advertisers who refuse to use open-web/app advertising because of the high levels of fraud, and this is a reasonable decision given the state of the industry.

Luckily bidding on your competitors name is illegal in Germany. We did get our competitors to not do that with some letters from lawyers.

I always thought that it was clear cut decided by Interflora case in EU court, and that such bidding is only prohibited if impersonation happens.

IANAL

In Germany there is a law against dishonest competition, called "Gesetz gegen den unlauteren Wettbewerb".

From my limited experience, resources are better spent to SEO to boost ranking. Also, you can request review on google ads keywords to prevent competitors from using your name, especially if you have a trademark.

You can keep them from using your brand name in their ad text, but you can't keep them from bidding on and showing up on searches for your brand.

> Another way these ad engines extract money out of you is by allowing competitors to bid for your brand name keyword. Try searching for Ozonetel on Google. There will be six competitor ads and then our website will show up :)

Fact checking you: On Google, I see at most a single ad (on most refreshes for Ozonetel, on one occasion for something called dialpad, on some cases, no ad is shown) and then your website, whatever it may be. It would seem that you are trying to piggyback on this article's discussion to increase awareness of your brand, which is not cool.

Great initiative. Frankly most organisations continue spending on digital ads despite seeing diminishing returns because they have had hired people, often at the VP level to do just ads. These people obviously have to justify their salary so they forcibly convolute numbers to attribute some revenue to justify what they do- which is running ads.

They randomly change distribution models, ignore the bot traffic, and present eye candy line graphs to show "tremendous" impact from ads. And this keeps on running until cash graph goes below the threshold and CEO goes into the "introspection" mode.

One place where I worked spend a lot on ads, but they also hired some marketing people and a leads conversion person who seemed to be able to quantify how much return on investment they were getting from ads.

Thing is, everybody involved has an interest in making the numbers look good. And it's really easy to make them look good even without faking anything. Just omit the fact that you're counting post view leads, stretch the lookback window, etc

I think he did a bit more of an objective job than that. He compared platforms where we were advertising and I remember that Pinterest generally gave a far better return on investment (than Facebook, which was the main platform we used).

> Frankly most organisations continue spending on digital ads despite seeing diminishing returns because they have had hired people, often at the VP level to do just ads. These people obviously have to justify their salary so they forcibly convolute numbers to attribute some revenue to justify what they do- which is running ads.

How the hell do I structure my empire so that it out-lasts my reign as obsessive good emperor? I am the one guy who can finally say, “I’m sorry, you’re a good person and a great VP of marketing, but you’ve done such a great job that you’re now just marketing yourself really and you’re fired.”

Honestly, the way that ad companies treat brand specific search terms has a whiff of fraud to me. Surely Google is well aware that by the time an end user is searching for a brand, they have no interest in anything else; they just forgot the specific URL. If I ever found docs encouraging such behavior, I'd upgrade it from "whiff" to "pants on fire".

That's at least something related being offered :)

When I searched for a very specific pillow type (and added country name to the search to get local results) I get this as first result on Google https://i.imgur.com/Fxx9b80.png. It's a mattress/pillow start-up/brand in India. The ad shows the exact thing I am looking for.

But when I click, and I am redirected to the actual product page https://i.imgur.com/eI6sNtJ.png that has none of these features and the product description is completely different from what it shows on Google Search result page.

I had reported this to Google sometimes back and nothing changed in their SEO gymnastics the last time I had checked a couple of weeks later.

I think what they are doing is fraud (wrong? unethical?). But what really annoyed me is it wasted a lot of my time as I thought I was doing something wrong and probably they actually had the product I am looking for.

SEO isn't Paid its a different channel

Prefacing this with an acceptance both that HN culturally has US management principles and that I've got no idea how to run a business...

Why is it an initial assumption that customer acquisition and ad spend are linked short or even medium term? That doesn't seem right.

When I see an ad for washing machines it doesn't make me run out and buy a washing machine. It gets me familiar with Samsung the brand so in 6-12 months when I buy a washing machine I am familiar with Samsung and buy their brand. Or it means that if my washing machine starts to sputter I'm more confident knowing where to go to buy a new one.

It doesn't seem enough to stop advertising for 6 months and claim no damage is being done. It seems well understood that repeated exposure to a brand has a powerful effect on humans.

Don't know how much of that is true as I am not a practitioner but when I was studying marketing we were taught this (continuous brand awareness reinforcement) only works for FMCG.

So this is arguing against GP's point since a washing machine should last for several years and is not fast-moving, right?

Yes, that's correct

Uber is about as fast-moving a consumer good as it gets. Up to the local speed limit.

In online marketing/merchandising online advertising is very alluring for two reasons: immediate feedback and the few key players are well known. Essentially online advertising makes a marketing team’s life much easier even though it could be a net negative for the business revenue.

When I was doing some work in this space for Travelocity many years ago the biggest problem was cannibalism of data and traffic. Yes, the traffic coming in from ads was shitty and generally costs more than it returned, but even still cannibalism was the bigger problem. We would have been better off taking all the ads off the site and spending money in other offline venues. It was so apparent that ad spend was a drug addiction. I still don’t understand how this was a mystery to anybody in management considering the strength of the brand and the success of their television ads.

Eventually the business imploded and the partner/affiliate segment became more valuable than the core business partially because it was absent online ad spend.

You're correct on that. Large scale ad spend is divided into "branding" (creating awareness about the company and products) and "performance" (leading directly to a sale). The performance category has many factors including the path to the actual conversion/sale and how many steps are in the way and how long it takes. It can be instant (clicking a buy button in the ad itself) or many months.

Optimization is part of the process to ensure wasteful spending is reduced as analytics and sales data is generated but unfortunately the biggest issue is the people who actually run these campaigns. Lots of politics, perverse incentives, lack of skills, and general apathy.

That's fine - that's awareness marketing and if you want to do it with online ads that's up to you. But then you should consider it compared to your other awareness marketing channel.

But the promise of performance marketing is exactly this immediate connection between ad spend and performance (i.e. sales).

Well then it becomes a question of attribution. How do you with a finite amount of dollars to spend on advertising know if you are getting your money's worth when you cannot connect a dollar spent on ad with incoming revenue?

Sure in broad strokes you are probably right that brand awareness has ROI positive in the long term. But then why would you buy targeted advertising vs a giant billboard on the main road?

This quote is over 100 years old, “Half the money I spend on advertising is wasted; the trouble is I don't know which half.”

To more specifically answer your question; I imagine they have some sort of strategy with advertising. A billboard campaign might work. I always see Barracuda ads in airports, but nowhere else. Soap operas are named because soap and consumer products advertised to homemakers during the day. Even when using a broad brush you look for the best use of your money and target your audience.

> Well then it becomes a question of attribution. How do you with a finite amount of dollars to spend on advertising know if you are getting your money's worth when you cannot connect a dollar spent on ad with incoming revenue?

The problem with attribution is that it's entirely flawed. Just compare what Google Analytics reports for a campaign vs what Facebook is reporting for the same thing. They'll give you different results and in most cases you probably shouldn't trust either.

BUT, what they do is provide an easily accessible number that you can show your boss to show you had results. This is massively flawed but most marketers don't understand the mechanics of advertising or tracking well enough to understand why they're reporting garbage data.

In my experience, the loss of conversion to competitors was not the problem with google ads on your brand.

The problem was phishing attacks on our customers.

I think it's worth countering this with the fact that FB ads work very well for us at Thread, because in the HN echo chamber it's very easy to lose sight of this.

This is going to depend entirely on the product and the market. We're B2C, and a very "personal" service rather than a service for one's professional life, so that probably helps. Plus our target market isn't defined by being tech-savvy.

As with all channels, optimising for customer acquisition (not just clicks), and lifetime value (not just easy sales) is critical for the long term health of the channel.

Serendipitous but of anecdata... was curious where you were located (ah, CA), so googled :) aaaand my keyboard decided I wanted, uhhh, "ozometel" (???).

Did you mean: ozonetel?

Nice.

(Gboard bit me again while typing it out here too lol)

Twitter has been by far the worst ad spend i've ever seen. When I complained that the clicks i got weren't real people, they blamed the landing page. I was advertising to 3 keywords "free astrophotography data" and the landing page was 0 ads, 0 popups and a link to free data to download.

They charged me for the ads, I complained to credit card company and did a charge back and they never followed through...

in fact, they offered me more credit to try advertising more.

For the life of me, I'm not sure how you can pay for 1,000 clicks to a very specific keyword and not have a single person download anything. The download was also valid for any user type. If you were on a mobile device, it showcased data that you could see on your mobile phone vs requiring a tool to download XISF or FITS data - it allowed you to download the high res image(s) in native iOS/Android formats.

Beyond the improbability of landing page being junk, Google analytics didn't fire for the majority of the traffic and twitters response was "the landing page may be too slow"

the landing page was < 1 second load times even within the twitter iframe nonsense.

anywhoo... the same ads on facebook, bing and google all had 75% click through and downloads but there were still a ton of clicks that had no analytics, no web time, didn't load anything in web analytics that we got charged for too...

the whole thing seems like a scam

FYI, Ozonetel bid successfully for the top line ad in my google search [ozonetel], dialpad taking second. You don't get there if you don't bid.

There was a freakonomics podcast recently about advertising (online and traditional).

No one can actually prove it has any ROI at all. No one is willing to run the experiments necessary. In the few cases of natural experiments, where ads got turned off for some people by accident, there was no change in buying behavior.

https://freakonomics.com/podcast/advertising-part-1/

https://freakonomics.com/podcast/advertising-part-2/

> No one is willing to run the experiments necessary

The people that would have the power to run this experiment have their entire careers depending on things staying as-is. Running the experiment carries a significant risk of exposing that the advertising operations they're responsible for provide much less ROI than they pretend it does.

The unwillingness of anyone to run such an experiment is already an answer. Why wouldn't someone jump at an opportunity to prove the thing/service they provide actually works, unless they were unsure about it themselves?

Exactly. That’s what the episode says too. The only people who could run the experiments hinge their entire career and industry on the results.

"It is difficult to get a man to understand something, when his salary depends upon his not understanding it!"

~ Upton Sinclair

And, unlike most Internet Quotes, this one is apparently real!

https://quoteinvestigator.com/2017/11/30/salary/

"If you believed something different you wouldn't be sitting where you're sitting."

~Chomsky to Andrew Marr.

> Why wouldn't someone jump at an opportunity to prove the thing/service they provide actually works, unless they were unsure about it themselves?

Because everyone is already acting like they know it works, so the only way that experiment can change things is in a way that's bad for the person in question. In that situation, they should (from a local, selfish perspective) be resisting even if they're awfully sure it does work (and perhaps even if they're right!).

Given that, I don't think the behavior has already given us an answer.

Facebook's ad research team have run a lot of experiments.

The goal was to demonstrate an impact of in-store sales from internet advertising. They did the first studies in about 2008-10, and have continued running these studies ever since.

They even built a tool so that advertisers can run these studies, and get a sense for the incremental impact of their ad.

Google have a similar (less full-featured) system.

Really, it's the rest of the ecosystem that has much of the fraud, and I think that a lot of people in the industry are aware of this.

> The goal was to demonstrate an impact of in-store sales from internet advertising.

So the goal was to advertise advertising. A more scientific goal would be to see if there was an increase of in-store sales from internet advertising. Instead they were looking to design experiments that would show a positive effect, with the goal of giving advertisers a dashboard so they could run those experiments themselves.

To let those advertisers do the experiments themselves, and to relate the campaign to in store sales, would require Facebook and google to share a ton lot of internet-user tracking data

> To let those advertisers do the experiments themselves, and to relate the campaign to in store sales, would require Facebook and google to share a ton lot of internet-user tracking data

I think you are misinterpreting what "do the experiments themselves" implies.

Certainly there is a JOIN with Google's or FB's data, but it can still happen entirely in a hosted sandbox with no data sharing necessary.

A system to advertise how good FB's ad system is, built by FB who we know have fudged their numbers before? That isn't proof of anything.

Heh. Just today, FB served me an ad claiming to be Macy's and talking about their closeout specials, accompanied by dozens of mourning comments. It was all phony.

Of course they'd say that.

We've run Google's Local Campaigns, which are supposed to drive brick and mortar store visits, which they measure using GPS and "other signals". I checked with one of our most remote and least frequented stores, to see whether they could see any of the 100 daily visits that Google claimed to have generated, and they couldn't see a thing (avg daily visits were around 80 before during and after the campaign)

Not really, scientists for example also have their whole careers based on the truth of some theories that they use. However, they're willing to put them to proof in different ways. The reason they do so is that they have a high degree of confidence that these theories are true. This cannot be said of people doing advertisement.

I’m not exactly sure what you’re trying to say. Scientists’ whole careers are based on running hypotheses to prove or disprove their theories. That IS their career.

And disproving a theory with lots of research backing it would be great. Imagine if someone found a huge hole in general relativity. There would be a boom in the grant writing industry.

The reproduction crisis in sciences suggest the self- interest is pretty wide spread.

And the social stigma in science of being someone who tries to take down, discredit, or disprove your colleagues/superiors/ competitors theories is pretty substantial: IME there has historically almost been a taboo against attacking our speaking negatively about publications and your own sciences + faculties practices.

The saying of sciences advancing one funeral at a time doesn't exist because they're all such great skeptics and falsifiers, and current scientific practice is heavily biased towards positive findings and contains general publication biases.

indeed there's actually a LOT of common ground with advertising self-interest, since a lot of publication in science is effectively just advertising your brand...

Truth is relative. Sampling can skew results. Scientists are people too. We all have flaws.

If you really believe truth is relative, walk to the grocery store tomorrow without wearing clothes and convince yourself that it will be fine. Or run a lighter under your hand - maybe it won’t burn today? Of course you wouldn’t to those things though because, at the end of the day, we rely on truth for everything we do. We just don’t always know what it is. Relativism is not the same as “it’s hard to figure out because it’s so complex”

Are you implying that the probability of someone going to a store named and getting out of it without harm is zero? There’s plenty of photographic and video evidence of the contrary. Probably not the best example.

Same for a lighter, there are many factors involved, and playing with lighters that way was a party trick where I grew up.

We use truth as a concept every day as an approximation, but the universe is not bound to follow. Very unlikely things happen all the time.

Truth is an unhelpful concept anyway (in science at least). What’s more useful is the likelihood to get a given result from a given experiment and the ability to make predictions.

And yes, when we put things that way it brings a lot of possible issues with sampling, processing, and measuring (and who’s doing the measuring). Some of these things would be harder to control for in a study of the effectiveness of advertising.

> Truth is relative.

This is a dangerous misconception to be spreading. It is not at all related to your other claims, namely:

> Sampling can skew results. Scientists are people too. We all have flaws.

These are of course correct statements. But they do not influence truth. They may well influence people's understanding of what the truth is, but that's different.

The analysis is often performed by a different team, thus no conflict of interest.

And if the team doesn't get the desired result there will be a different team next time.

Depends which team has the power. Sometimes it's the team auditing the results that's more respected.

I work in the field. Incrementality testing is a big part of marketing measurement at any reputable agency. Any claims to the contrary are FUD.

That said, companies like P&G, Airbnb, and Uber, which are oft-cited as examples of digital not being worth it, fail to understand their own brand recognition and organic power, built through prior marketing efforts, as key to their current standing.

Sure, TODAY, it doesn’t have the impact they’d like it to have but the investments PRIOR were key to ensuring their success.

Quit and do something that isn't fundamentally detrimental to everyone's quality of life.

http://jacek.zlydach.pl/blog/2019-07-31-ads-as-cancer.html

Brand is a lot more than marketing spend, and not all brands are equal. Google and Facebook have companies that depend primarily on performance marketing spend over a barrel.

How do you determine a reputable agency though?

I worked next desk to people running a small ad agency. Because we shared office (and I found them an intern), I got a very good look at how they're working. What I've seen can be summarized as: people who have zero clue or interest in statistics writing "reports", with "graphs" they don't even understand beyond "pointing up = good", proving positive ROI to customers - who also have zero clue or interest in understanding the numbers in the report, and not enough visibility into the whole funnel to independently check attribution. Both the agency and the clients were engaging in a shared and completely unjustified fiction of positive advertising return - and as long as both sides were happy, the money kept flowing.

I've been long since suspecting a lot of advertising on-line looks like that. Every now and then, I see evidence in favor. Like that good ol' Optimizely debacle, where it turned out Optimizely was structurally optimized to help people make invalid A/B tests, that erred on the side of concluding the interventions were working[0]. And sure, big brands with some superstar ad teams probably do this right. But I think there's enough slack in most businesses that advertising spend can get quite far detached from actual ROI without anyone noticing (and with plenty of people happily riding the gravy train).

[0] - https://news.ycombinator.com/item?id=10872359

Have you ever engaged with the large advertizing holding companies? They have teams of HIGHLY acomplished data scientists, boutique vendors for specific niches etc. Even if you don't pay for it there will be some people skilled at statistics looking at reports. If you pay for it, there will be data scientists doing analysis.

The large ad firms deliver a very good product but it usually isn't cheap.

I haven't, and I don't doubt that's true (at least in many cases; I've learned not to underestimate the chances of big companies ending up running scams either).

I've personally dealt mostly with the other end - the individuals and small agencies - and what I saw revealed total lack of necessary competences for the reports to be corresponding to reality. Perhaps it's understandable - after all, people who have the required knowledge likely end up working for big advertisers, or in entirely different fields. But small business owners don't pick these big advertising companies either.

It would be fascinating if true if the big "brand name" companies whose entire business is based on charging more for the same commodity product as a generic store brand, but using advertising to justify the higher price in the customers' mind, weren't buying ads from people doing the same thing to them...

The question isn't are the scientists good, the question is if the vendor is honest.

> as key to their current standing.

It seems like it would be important to measure this standing and turn down spend once it is reached. I can see how there's a lack of incentive to help large advertising spenders understand this.

> Incrementality testing is a big part of marketing measurement at any reputable agency

In a huge number of cases these tests are run by people that don't have the statistical background to property run and understand these tests, in the remaining cases there is almost never follow through to ensure that the results of the test have persisted after the experiment.

People will say "oh this test shows 10% improvement, and this one does too! and this one! and this one!" But then you should see a nearly 50% improvement just because of ad spend and you almost never do. Nobody ever check that all the numbers add up, they just want the numbers that someone reported to feel like they are sound enough to hold up to scrutiny but scrutiny is never applied.

I used to work in this industry.

A small tech team investigated fraud on our platform and developed a system that was pretty robust at detecting and potentially shutting it down. But literally nobody was interested - even the people advertising don't want to know.

The people spending money are typically networks, media buyers, ad agencies, etc, far removed from the actual brand.

There are so many parties who want a slice of the brand's cash that they are all long past caring about whether the ad is being viewed by a human or not.

Agency or client side?

I assure you the big brands CFO's take a huge amount of interest in what is spent on advertising.

In my experience the Sales and Marketing teams of any large company have the loosest restrictions on how they account for their spending.

Other dept;s like IT have to justify every penny, but Sales and Marketing not sooo much

I think there's a very loose belief that sales is how money is made, while they people that build and test products are pure expense.

I've seen it not just in the phenomenon you describe, but also in layoffs and restructurings.

Yes. Cost centre and profit centre.

Yes. Sales and marketing : "profit centers" . Ever increasing budgets.

Engineering , R&D, Manufacturing, Tech, Testing : "cost centers". Ever tightening budgets.

MBA, Beancouting 101

That's the issue - the CFO might care, but the staff actually delegated to spend the budget don't.

I worked in adtech. I built a simple ML system that detected signals of fraud on our network. When I reported my findings execs told me the group that was facilitating the fraud was our only profitable division. Soon after, our company went bankrupt. My final paycheck was months late.

Sounds exactly like my former employer ha.

IIRC our system wasn't even fancy ML - just finding known user IDs with identical timestamps over many simultaneous clicks / impressions on really diverse publishers.

The experments are run. Why don't you look at the work that comes out of the Advertizing research foundation.

Uber was just incredibly incompetent to not audit their ad spend at all.

I worked at an ad company. It was an absolutely standard metric to eg geo-fence ads out of a state or two for 3 months to demonstrate the impact of ads. This isn't easily externally visible, but tests like this are standard practice.

Particularly in the app install space, which is sketchy as hell once you stop buying from the top handful of vendors, buyers should be auditing by a couple million in annual spend. To get to $150m without looking hard at big chunks of their spend is just plain arrogance and/or incompetence.

I partly work in this space, and I can confirm that geo-fencing over a time period is the probably the most transparent and convincing (to the client) way to do this. Essentially you mark out local regions (as small as zip codes or as large as states, or something else in this spectrum), and ensure that your advertisements only show within certain "fences". And then you compare old and new numbers within those fences.

The technical aspect aside, there are a lot of "soft" factors that help: regular communication of easily consumable numbers/graphs/metrics to the client, calling out inconsistencies, etc. In other words, this is far from a fire-and-forget exercise; this can be an intense monitoring and client-engagement exercise.

The regular client engagement sometimes helps you in pinpointing cause-effect in the observations too. For ex company X might decide to advertise less of product Y in a certain state Z because of new laws there. But the team you're interacting with might not be aware of this change, or might not be cognizant of its potential impact on an advt. campaign. Regular dialogue helps here since you might observe a change in sales trends, and bring it up in a meeting - and the client team might be able to then rationalize the change. This is healthy for both parties.

It’s like saying that no one who works on reliability of the systems is willing to run experiment to throw 100% errors for a month, because running experiment like that may show that reliability of the systems doesn’t matter.

And then using data from a one system that went down and nothing happened as a proof that systems reliability doesn’t matter at all, and it’s huge scam by engineers.

> Why wouldn't someone jump at an opportunity to prove the thing/service they provide actually works, unless they were unsure about it themselves?

Because advertising in some form certainly works. If you can determine that approach "A" that everybody is doing is actually a waste of money but approach "B" is effective, then you can develop services around approach "B" and market them based on these findings.

Nobody is arguing that advertising doesn't work at all, but my argument (which I think a lot of other people agree with) is that the effectiveness of online advertising (and maybe advertising in general) is overblown.

Advertising that does not actually reach the claimed consumer, by definition, cannot possibly work.

> The people that would have the power to run this experiment have their entire careers depending on things staying as-is.

Any large company could invest in some experimentation, whether their marketing directors want it or not. It makes sense that at least a few do but just don’t publish the results

This is no industry with as much snake oil BS as adtech.

It also happens to be the sole form of revenue all of the largest tech firms enjoy.

No wonder there hasn't been a real audit.

Also, all those marketing execs buying all the ads would be out of a job too, so they aren't speak up either.

> It also happens to be the sole form of revenue all of the largest tech firms enjoy

I don't think this is true unless you qualify it a bit. Apple, Amazon, Microsoft, don't make most of their money on ads. Even Google has other revenue streams.

> Even Google has other revenue streams.

How long do they survive in their current form relying solely on them?

Days, probably, but that's not the point

It's exactly my point. Outside of Apple, all the biggest tech firms are 100% reliant on a form of income that is, at best, partially fraudulent. More realistically, almost entirely so.

Yep. Like the old adage goes, You cant make a man understand something, if his job depends on him not understanding it.

> The unwillingness of anyone to run such an experiment is already an answer. Why wouldn't someone jump at an opportunity to prove the thing/service they provide actually works, unless they were unsure about it themselves?

Or they have run the experiment and the results haven't lines up with their own personal biases so they were discarded.

“Half the money I spend on advertising is wasted; the trouble is I don't know which half.” - John Wanamaker

Half was true in his time (a century ago) but I think our numbers are way worse.

> No one is willing to run the experiments necessary.

Not without reason. Even without the conflict of interest that Nextgrid points out in a sibling post, there's still a significant financial barrier to attempting to measure this stuff. According to a former professor of mine who spent a large chunk of his career studying this stuff, the size of study you need to conduct in order to get any kind of statistical power at all on an ROI study is just absurd. See, for example, the treatment starting on page 15 of: http://www.davidreiley.com/papers/OnlineAdsOfflineSales.pdf

Gabriel Leydon of Machine Zone spoke to the general topic at Code/Media back in 2016. Basically discussing that they'd gone through the trouble of building internal expertise and tools for optimizing their ad spend to better ensure specific outcomes they desired/required in a way that would inevitably lead to more sophisticated ad buyers and putting a nail in the coffin of traditional media advertising.

https://www.youtube.com/watch?v=oXBqzpExvrk

Alternatively, they dumped hundreds of millions of dollars into ads based on a wildly unrealistic notion of a customer lifetime (much like the rest of Silicon Valley).

I didn't say they achieved their goal. Though I do think it was/is an admirable one. It would be nice if marketing budgets weren't a near limitless accountability-free money pit.

This made the rounds in adtech in 2016 and it's really nothing special. His entire speech comes down to a single quote: "media will be quantified".

Sure, everyone wants that. Precise attribution has been the holy grail for a long time and the struggles are far larger than just a few technology products. The new battleground is first-party data and clean rooms vs privacy regulations. And that's after dealing with all the politics and perverse incentives that happen in such a massive industry.

> No one is willing to run the experiments necessary.

Or, no one is willing to share the results of the experiments.

It's unclear how this experiment would be done. In the case of brand advertising, it's likely that brand awareness would decay over some period of time and in turn purchase behavior would change.

It's not currently possible to run an A/B experiment with a hold out group of potential customers across all channels, let alone for any longer duration experiment. So how can we separate cause and effect? (although pay per conversion channels do get gamed left and right)

Come up with some new product that requires some personal data for usage (eg. age, gender/sex, address). Start to advertise this in just one country to one demographics, and look how many out-of-target orders you get.

Maybe it's even enough if you simply just sell it via mail order, you can then look at the addresses.

There's probably a natural information spread in any market (word of mouth, trade magazines), and there's probably a physical dispersion of the target group of people too (people move, visitors/tourists saw the ad/product and order it at home), but it still should be a valuable to see how much effect just one campaign has.

Maybe one of the best products for this could be a car. They are pretty standard, really not much difference between them, they are in all price ranges, and regularly new models come out. Advertise one in a few major US cities but don't in others.

For most vendors, not possible to accurately target age/sex/address online. They will certainly sell you the option, but they won't tell you how accurate the data is - or they will provide it a high accuracy with extremely low coverage.

To see this in action - spend a day watching pitbull videos on youtube and see how many spanish language ads you'll get.

Hm. How about simply buying ad space at a few local newspaper sites? Sure, it's noisy as hell, but maybe still a decent proxy variable.

Plus, maybe it's possible to somehow offer a coupon for those who buy via the ad, and so on.

After all the ad/tracking industry probably have a thousand tricks to increase accuracy of this.

But, yeah, I'm not holding my breath for a conclusive answer on this.

Traditional brand advertising testing (TV, newspaper, etc.) would be geography segmented as I understand it. So half the cities got the campaign and half didn't. You can mimic that with IP based geo-location although you'd get more leakage than pre-internet.

Why does reddit have ads if all ads are a money pit?

I understand you no longer work there, but ads started in 2009 I believe, so you perhaps had some input on this?

Owning the pit into which other people throw their money seems like it would be pretty lucrative.

This is incorrect. Companies do the experiments. They just don’t publish the results. Why would they? The idea that Amazon doesn’t know the ROI on their ad/marketing spend is laughable.

But then why do they keep doing laughably bad advertising?

Because 50% of advertising spend is wasted, but it's hard to figure out which 50%.

If you're doing an $XY,000 ad campaign, it's a waste of your time and money to figure out which 50% it is.

If you're doing a $XY,000,000 ad campaign, you employ people whose job it is to figure out which 50% it is. The thing is, you usually don't go ahead and pay them to blog about it.

I’ve thought and said so before: The ads racket is be a case of the emperor with no clothes, at best.

At worst, it’s a front for building “profiles” of everyone.

I can’t recall the last time I bought anything -because- of an ad.

If anything, ads have sometimes actually put me OFF from buying something!

But someone did. "Brand awareness" is a thing.

For example we can probably agree that for completely new companies spending on ads makes sense. Or giving out free samples, etc.

Similarly for big companies doing media campaigns to keep the new ones at bay makes some sense.

Even if word of mouth is a thing, even if there are organic searches, and even if it seems like a race to the bottom if everyone just tries to outspend each other.

It'd be great to make experiments about how to sensible prevent/regulate this ad arms-race. But first better data privacy laws.

Oh look, yet another news about how the advertisement economy fucks up more shit than it benefits:

https://news.ycombinator.com/item?id=25624112

“URL shorteners set ad tracking cookies”

I don't understand this line of reasoning. P&G, Unilever, Cocacola, etc have never, not once in history, had a gung-ho C level exec who said "Screw it, I'm going to find out if our advertising works". And then either found it works and kept spending, or found out it doesn't work and saved literally billions of dollars.

There is so much money at stake that could be either saved or generated, its simply not possible that no one has looked at it. I used to help Pepsi/Fritolay set up tracking to tie advertising on youtube to in-store sales. They spent millions of dollars to measure their ads, Google had a clean-room data center specifically for pepsi/frito. The idea that no one actually checked if this system works is simply not possible.

For brands that large the majority of their spend isn't campaigns specific to new products, but overall brand management. The experiments would need to run for years or decades and if the prevailing belief that these are Red Queen's Races is correct, the cost if they're wrong could be the whole company.

Fortune 500 is a cesspool of insane inefficiency balanced by equally ~~insane rent seeking~~ insanely secure revenue. The sooner everyone understands this, the better.

First, care to define your terms?

Second, inefficient relative to what?

Third, so far, what you've written here looks like a rant, not a predictive theory nor a powerful explanatory theory.

Would you like to quantify your claim? Or at least make it more precise? As it is, I don't think it advances your argument.

I'm interested in strong logic, data, explanations, and persuasion. I see none yet.

> Second, inefficient relative to what?

The sad thing is it's not inefficiency relative comparable things. But anyone that has worked at these places or sold B2B to them just knows it on intuitive level that they are garbage, and need a heavy sedative to think there are no alternative.

> not a predictive theory nor a powerful explanatory theory

It's not. It's about letting go of some efficient market ideal and then finding new ideas.

We can look at Fortune 500 case-by-case to learn new things

> Cola cola

Sugar drug cartel. Despicable business with very stable revenue despite being a net drag on society. (At least "regular" drugs have a lot more upside!)

> Proctor and Gamble

Just as restaurants are reaching down market, and the inefficiency of everyone cooking and cleaning is starting to have market implications, we should see their reign finally dwindle. Wash-and-Fold should follow laundromats. The specialization means that stupid differentiation between products for uninformative consumers (c.f. https://en.wikipedia.org/wiki/Monopolistic_competition) should go away and restaurants and laundromats optimize.

Personal soaps and cosmetics (of course many soaps are cosmetics) however will stay as cultural reasons ensure people will continue to clean themselves and not contract that out for the foreseeable future.

Thanks for elaborating. I don't think I disagree with many or most of the points you've made.

It might be interesting to define a systematic and testable definition of organizational efficiency relative to an ideal market. Just like combustion engines can be compared to their theoretical maximum efficiency, it would be interesting to compare a particular organization against its maximum potential.

Here is another angle on the topic of relative efficiency. Consider a particular organization. Coca Cola will do as an example. Given its environment (financial incentives, regulations, cultural norms, etc), are we surprised by its behavior? This raises the question of how much an organization's particulars (e.g. leadership, history) affect its efficiency.

I suppose I take a systems view of it. Many deplorable businesses survive in niches. But can you "blame" them? It is a matter of perspective: organizations, like life, adapt to their environment and modify it to their liking. Such organizations can stretch the environment to or past its breaking point for a relative long period of time before having to deal with the downstream consequences.

> the inefficiency of everyone cooking and cleaning is starting to have market implications

Elaborate on your reasoning here? While I can maybe imagine a more centralized food preparation system being more efficient, the currently growing market solutions (DoorDash, etc.) are certainly not, and I can't think of a system in which transporting clothing somewhere for someone else to clean would be more efficient than an in-home washing machine. The wasted energy alone would be colossal.

How is Coca Cola rent seeking? How about Apple? I suspect you aren’t using that term correctly but maybe I’m missing something.

Effectively every restaurant in the US has to pay either PepsiCo or Coca Cola. Similarly, if you want to buy a non-alcoholic beverage at the grocery store it's mostly down to those two (with Dr. Pepper having a much smaller but still significant stake). Any competitor that emerges just gets bought up by one of the three.

Apple's half of the phone Duopoly. Either you pay them or pay Google if you want a phone and want to buy software for it.

Yes. But in fairness I retracted rent sneaking as a combination of monopolization, bonafied rents, addiction, and just shear damn inertia contributes to the malaise.

Calling it all "rent seeking" is not a hill I want to die one.

"Rent sneaking" is an interesting typo. Going to have to give some thought to how to use that term in appropriate context.

Restaurants only pay because consumers demand it and would not visit the restaurant otherwise. That’s not rent seeking. That’s market forces. And what they pay correlates with how much their customers demand coke/Pepsi.

The rent seeking is buying up competitors until its a duopoly and then protecting that through shenanigans like exclusive agreements. It has nothing to do with consumer demand.

Buying up competitor does not immediately make it a monopoly or duopoly. Especially in industry where barrier of entry are low.

Exclusive / Special agreement with serving only one kind of Soda ( or no Pepsi where there is Coke ) are Anti-Competitive arrangement. Not Rent Seeking.

The exclusivity contracts are what some consider to be rent seeking. I don't know if this is the correct term, but I do believe they are anti-capitalist and shouldn't be allowed in a capitalist economy. If freedom of choice is so important, than large businesses shouldn't be able to use their ability to corner the market to force you into exclusivity agreements.

I'm not sure this is quite right. I would have thought that most restaurants very much WANT to sell sodas, as the margins are enormous--much, much higher than the margins on cooked food.

This page [0], for instance, says the cost of goods sold for the soda itself is a penny an ounce for the syrup and CO2. Iced tea is apparently the margin champion, as the same page indicates it can cost as little as a penny per glass.

[0] https://www.restaurantowner.com/public/Restaurant-Rules-of-T...

Coca Cola sells water with flavouring and quite astonishing amounts of sugar. Apple sells nice devices made by what might as well be slave labour.

Both are propped up - actually maintained - by huge and vastly expensive brand management strategies.

The criticisms of individual campaigns here are missing the point. It's not about micro ad spend, but the perception of value and manipulation of behaviour created by the combined effect of multiple PR and advertising efforts - which include traditional print and TV/radio ad campaigns, guided advertorials disguised as news in the MSM, interviews with prominent figures, political campaigns of more or less obvious relevance to the core business activity, state, national, and international political lobbying, direct political campaign donations, advertorials masquerading as "freelance" journalism and blogging, managed astroturfing on social media, shareholder relationship management, product placements in movies and music promo videos, articles about commercial visual design elements in trade journals. And on and on.

That perception of value is - unsurprisingly - extremely valuable. And it's very much a US way of doing business. Instead of producing products that are inherently superior, produce something that is functional but glossily packaged, brand it as a premium lifestyle commodity, and charge exorbitant prices for it.,

The prices are traditionally far out of proportion to its actual utility. In fact real utility may well be negative - see also diabetes and any number of other health issues, depression associated with social media use, debt-driven spending on lifestyle products. Etc.

So the rent seeking comes from a kind of cultural squatting. There is value in dominating discourse in all of these different ways, because discourse and narrative define markets and ultimately control behaviour. And while this is happening other kinds of discourse - which may well have more real utility - are diminished at best and crowded out at worst.

So in this case it doesn't matter if Uber "wasted" their money. Uber have their own branding thing going, and explicit ad spend is a tiny part of that.

And even if all online ad spending ended tomorrow, a small number of corporations would have a difficult time for a while, but the marketing industry as a whole would inevitably interpret the change as minor damage and route around it.

Because the ideology of capitalism is "cooridation failure, whatever", It sure would be nice to find a way for one of those companies to freeload off the culture-shaping the others do, and bring the whole enterprise crashing down!

Trademark laws exist to prevent such freeloading. In this context, they ensure only the ones paying for the culture-shaping can benefit from it.

> Both are propped up - actually maintained - by huge and vastly expensive brand management strategies.

In the case of Coca Cola, I agree - in the case of Apple, not at all. "Keyboardgate" aside, their products are vastly superior to the competition in build quality and life time. A typical Windows laptop is unusable after two, three years (almost all are made from plastic which breaks or looks extremely ugly rather sooner than later), and has next to no resale value while in contrast even old cheesegrater Mac Pro's fetch many hundreds of dollars today. iPhones are supplied with firmware updates far longer than most Android phones outside the flagship Pixel line are. OS X is, while it has gotten locked down a fair bit over the last years, still way better than Windows as it doesn't do tracking bullshit and start menu ads and decently better than Linux because no matter which device, stuff usually Just Works (tm) without having to fiddle for hours to get something as basic as a Bluetooth headset working.

Apple built themselves a loyal following with pure quality superiority over the competition, unlike HP or Dell who are only surviving because of enterprises who follow the "no one got fired for buying IBM" line.

And that doesn't even touch that it's Apple who's at the forefront of innovation. iPods, iPhones, iPads, Apple Watch, AirPods, now the M1 CPU - these entire device classes were created by Apple. When was the last time you heard about something truly innovative from the Windows/MS/Google side of IT? Only thing I can remember is the Google Glass.

What Apple is doing is not a "brand management strategy" per se - it is delivering actual value and innovation instead of rent-seeking. And for what it's worth Tesla are doing the same thing.

Apple isn't repairable. Apple devices have high prices simply because of the brand ecosystem. Old Thinkpads use commodity parts - but they command good prices because they are repairable.

Marketing the new thing is what Apple does. It is good at it. That is not the entire spectrum of customer value.

Thinkpads are pretty much the worst form factor for a laptop that doesn't transition into "toy" space.

Source: forced to use one daily. If I could get a Mac laptop running Linux but with the input smarts of the Apple trackpad and keyboard, I would in a heartbeat.

That shows that 1) people are different or 2) you are a fan of Apple hardware. It doesn't show that Thinkpads are bad. IMO they are light-years ahead of anything Apple sells.

Source: Me.

Out of curiosity, how do use it without bumping the trackpad while typing?

My trackpad is automatically muted/unresponsive while I'm typing. I suggest you figure out how to get palm rejection to activate/work on your setup, and to look if a different Thinkpad series might better fit your body (ergonomics wise).

There is no option to enable "palm rejection" in the Settings dialog on Ubuntu under "Mouse and Trackpad", as far as I can tell.

It might not be that easy, and could require manually installing some packages.

I recommend a dell xps 13 developer edition

The trackpad is superior to apples and it can come preloaded with linux

I've been using my windows laptop daily for 5.5 years without issue.

>A typical Windows laptop is unusable after two, three years

This shows that you are just spreading FUD. A Windows machine is no slower after 10 years than it is the day you bought it. Sure you might have added so much software-crap that the software runs slow (which is easily fixed) but the hardware is exactly as fast after 10 years as Apple hardware is. Do you think Apple builds their hardware out of magic and unicorns?

>but the hardware is exactly as fast after 10 years as Apple hardware is.

Capacitor, DRAM, Heat Sink, etc.... There are quality difference in hardware. Although the two / three years remarks were definitely exaggerated.

> A typical Windows laptop is unusable after two, three years (almost all are made from plastic which breaks or looks extremely ugly rather sooner than later), and has next to no resale value while in contrast even old cheesegrater Mac Pro's fetch many hundreds of dollars today.

By "typical windows laptop" do you mean a cheap model where the macbook equivalent is buying nothing at all? Of course it's going to be lacking in that case.

Well-made windows laptops exist, and hold plenty of value. But you have to pay for that, no matter what brand you choose.

> tracking bullshit

Like checking signing certificates by pinging servers almost every time you run a program?

This analysis excludes glossy packaging and status signalling from the inherent superiority, when (in terms of overall market value) they're part of the superiority.

And in Coca-Cola's case (among others) the advertising is much of what creates the value. When you know it's a Coke, it tastes better. I've had a lifetime of absorbing ads that make me think of good things whenever I know I'm drinking a Coke.

That’s not rent seeking by any accepted definition.

The answer is not complicated, and the logic on this board is strained by people I think that don't work in marketing.

1) Advertising works.

2) Coke, a sugary drink, is largely kept alive by smart marketing, that involves mostly advertising in the end.

3) Much of advertising is quite wasteful, because it is a fuzzy instrument - but on the whole, good marketing works.

4) There is a lot of self-awareness in the industry on this, and no doubt, a lot of borderline fraudulent actions by participants willingly spending money they know doesn't work.

5) Ad networks will happily sell you ads that don't work, and look the other way when there are shenanigans.

The problem here in the thread, is that people are having a hard time grasping how all of these things can be true at the same time, but it's not that hard really.

That some ad spending turned out to be 'not useful' is 'not news' when everyone knows that easily 50% of ad spending is probably wasted, we usually just don't know which 50%.

Agree with most,

>2) Coke, a sugary drink, is largely kept alive by smart marketing, that involves mostly advertising in the end.

Not sure I agree. I haven't seen any Coke Cola Ads for years, not online digital Ads and I rarely consume any traditional media. But I will still buy Coke Cola every once in a while.

They are also on all of the convenience store with decent positional shelf space or visual merchandising. Something I learned that no one on HN actually knows anything about during the discussions of App Store.

Edit: All of my friends can taste the difference between Coke and Pepsi. It is just different. So I was surprised to read comments and youtube video about "blind tasting" on the two. No, I didn't drink Coke because of Ads, I drink it because it taste better. (And some people prefer Pepsi )

Your personal situation doesn't represent the market, moreover, you spent 20 years watching coke ads growing up and having coke, it's impossible to tell how that affected you.

You saw 1000 'product placements' for coke without being aware of it in films other things. Taste is associated with familiarity and comfort.

> Taste is associated with familiarity and comfort.

I respectfully disagree, I used to work in F&B industry. And product marketing and placement has absolutely no affect on how one think something taste is good or bad. It does however remind anyone next time the walk into store they should buy coke, which does affect sales number. But not taste difference.

My 7 years old nephew hasn't watched a single Coca Cola coke in his life time, but he prefer coke over Pepsi. While my friends daughter ( and her mum ) prefer Pepsi.

"And product marketing and placement has absolutely no affect on how one think something taste is good or bad"

Yes it odes.

"My 7 years old nephew hasn't watched a single Coca Cola coke in his life time"

1) It's ridiculous to say a 7 year old 'has never seen an add for coke'.

How would you possibly know that? Unless they are living as the Amish, that kid as seen ads.

2) That someone will prefer one drink over another is not relevant to the argument.

Nobody is making the claim that 'ads make you do stuff'. No Coke ad is going to 'force Pepsi drinkers to like Coke'.

Ads are influential. They root the product in feelings, emotions, those have impact, which is why advertising is there.

3) Your nephew was probably drinking code from some age, maybe as a treat. It was Coke and not some Italian cola, because Coke has such a powerful market position.

In Italy, they drink Brio and it's quite good, you can't even get it here, if you could, people would drink it.

Coke spends billions on ads because they work not because they don't.

The trick isn't figuring out whether advertising in general influences people's behavior. The trick is in figuring out if any particular advertising campaign generated more profit than it cost to run.

Also, there's one alternative that's often forgotten in these discussions: Perhaps game theory is at play. It may be, for example, that, across entire industries, advertising costs more money than it's worth. But that everyone has to do it anyway, because anyone who chooses not to will start losing ground to everyone else. IOW, just like in the standard prisoner's dilemma, choosing to act is less about increasing your potential gains than it is about limiting your potential losses.

There is an interesting long-running natural experiment in the pharmaceutical industry that suggests, albeit inconclusively, that this is the case.

Distilling it down into "any particular advertising campaign" is pretty myopic in this world. It's about strategy as a whole over decades. Not every ad campaign is to drive immediate sales or signups (direct response). Branding and awareness campaigns can take years to run, and these are ALL diligently measured at every stage. Losses do occur do to negligence, malice, and poor execution all the time, but the brands should take some blame as well as they often feel compelled to spend. They have huge budgets that they NEED to spend regardless of they perform - sometimes due to accounting shenanigans, sometimes because they don't know any better, etc.

No matter what time scale you pick, there's still a point where ad spending costs more than you benefit. As time passes, the boost from any campaign decays. For any particular person you only need to show them so many ads to maintain awareness.

> these are ALL diligently measured at every stage

Mmm. Sometimes. Sure, you can obsessively measure audience and sales numbers, but if you don't have any controls then it becomes nearly impossible to figure out how much of that came from ads and how much came from everything else in the entire world.

it is true that that profit/loss point exists, but it may be impossible to measure. But it's very clear that 75+ years of Coca Cola marketing strategy has produced the situation where any human on earth immediately associates "Coca Cola" with soft drink.

If Coca Cola cut advertising 90% this afternoon, they likely wouldn't see the consequences until a new generation or two grew up.

I suspect that a similar dynamic can be in effect with many industry events. Everyone would collectively perhaps be better off with pulling out of or at least scaling back on big industry shows. But that doesn't mean it makes sense for just you to pull out. (And, certainly, your events team probably isn't going to push for scaling back.)

There's also a huge mutual back-scatching thing going on. I remember in a former life we wanted to pass on a big software vendor's user group show because, while we sort of needed their software for some important customers, we got very little traffic at this expensive event. Their CEO called our CEO and basically said to him "Be a pity if something happened to our partnership."

> IOW, just like in the standard prisoner's dilemma, choosing to act is less about increasing your potential gains than it is about limiting your potential losses.

You seem to be assuming that the potential gains are limited to current players of the game in this comparison.

It's possible that Coke, Pepsi etc. are all losing more money on ads than they're gaining in market share against their current competitors and in increased sales. But that doesn't begin to address the question of whether advertising is a net loss.

For one, the advertising also serves as a collective moat against new competitors, which could enter the market and eat up both Coke and Pepsi's market share. Maybe their biggest gain from it is that a bankrupt soda company from the 80s didn't replace them, and that a similar company today wouldn't have room to enter the market.

It also doesn't account for more complex effects like a net increase in sales for all companies in that market. E.g. maybe Rolex, TAG Heuer etc. all benefit in the long term from expensive watches being seen as fashion accessories.

If Rolex stopped their advertising spend one month and saw little immediate change, then TAG Heuer etc. might follow suit. But both might only experience the real loss years later as "luxury watches = must buy" faded from the psyche of their current and potential customers.

Part of the claim is that the people who are checking are ad execs who, if PepsiCo stopped buying ads, would shortly be out of a job (or have their budget and influence slashed). A counter to this might be that different advertising channels are likely not identically effective, and a TV ad exec has a big incentive to poke holes in non-TV ads.

PepsiCo has third-party auditors that they employ to check the checkers.

Yeah that is what is usually given as the reason so I was getting at a gung-ho CEO, CFO, CRO, Consultant, etc.

See the Pepsi Refresh Project of 2010. Where Pepsi redirected a sizable amount of ad budget (including super bowl ads) to community projects. They abandonded this strategy after a while because they lost market share.

This is the biggest large scale test of advertising I'm aware of. But it probably doesn't apply to all products.

They only lost market share because everybody else was still using advertising. Therefore, this is not a test in favor of advertising (only in the prisoner's dilemma sense).

So advertising works? If it didn't, those buying ads would not take the pepsi market share

If advertising is an arms race with more total cost than total benefit, that would be an argument that advertising works. Because anyone, at any given time, will benefit from running advertising.

It also means advertising is a net loss to society, but that is separate from the question 'does advertising work'

I think you rephrased my comment, yes.

> net loss to society

Not so clear, especially before internet search engines existed. As a consumer, I find it useful to know what's available for me to purchase.

If you're running a company, "just stop being in a prisoner's dilemma" is not one of your options.

How do you look into it though? I don't know, so i'm asking - but my immediate thought is that you treat it like science. You isolate an environment, advertise, and see if it has an affect. But the implication there is that if it doesn't have an affect, money is on the table.

If this is even remotely close to reality then it makes sense to me. Companies are more concerned with constant growth than strict efficiency, imo. They're throw as much money around as possible, and every cent lost or left on the table is panic inducing.

I also imagine different types or products and/or markets behave quite differently. Eg a new product might very well benefit from advertising - since no one can buy your product or visit your store if they don't know it exists.

> Google had a clean-room data center specifically for pepsi/frito.

Why?

My company provided sales data from stores to a specific team on google that compares it to users who’d been advertised to. This team didn’t want my company to influence results, and didn’t want YouTube to influence results.

More complicated since there are multiple device graphs/etc in play, but effectively tens/hundreds of millions of dollars at play and no one trusted anyone. The largest thing at risk was YouTube’s ad revenue.

But does it need dedicated datacenter? I wonder they just run different service instance rather than datacenter.

>> The idea that no one actually checked if this system works is simply not possible.

Have definitely heard that one before. No disrespect.

Yes, the advertising industry is just obfuscated basic income for the twitter class.

> No one can actually prove it has any ROI at all. No one is willing to run the experiments necessary.

Depends enormously what sort of business you're in. I used to work for a company where all of our sales came from ads, 100%. It was trivially true that if we stopped advertising we would have no sales. We were also committed to running experiments: we knew how well all of our many advertising channels performed, and we ran A/B tests for every change.

Listening to the podcast just a minute ago as I found this post... What a coincidence. I second that recommendation.

Great subject matter, but boy does reading the transcription remind me why I hate podcasts. Stop infantilizing the audience with these coo-coo-ing sounds bites and get to the damn point already!

These experiments are run all the time even before the internet. I remember reading about how for broad brand advertising they used to segment by city. Half the cities got an ad for Coca Cola and half didn't. Then you compare sales of Coca Cola. No one will publicly publish numbers because it's a mix of sensitive sales data and competitive advantage (ie: otherwise your competitors don't have to burn money running their own tests). Also, smaller shops turn their online advertising campaigns on and off all the time to test impact.

> No one can actually prove it has any ROI at all.

That's a philosophical question of whether you consider statistics to be "proof".

> No one is willing to run the experiments necessary.

You're nuts if you think this is true. I assure you that companies in traditional industries (i.e., without venture capital) can and do run these experiments.

The Uber story is about venture capital and its anti-market incentives, not about the ad industry.

Read the podcast transcript. An economist proposed turning off print ads in one market to find out if they mattered, and the head of marketing said he’d rather not know.

Only because the head of marketing realized it would expose his past misjudgments - not because he didn't realize it would actually be a good thing for the company. You'll see this type of behavior in every department - people trying to cover up their mistakes.

Wait, so their marketers are unwilling to do anything that would reveal ad ineffectiveness, and you consider that an argument in favor of the claim that ads are effective (the topic at this point in the thread)? Like, you take that as a signal that marketers would converge on effective ads?

My point is that we have to differentiate between bad employees and the overall effectiveness of ads.

The latter is all about incrementality. When you adjust for it, you get the true effectiveness of ads. In some instances, the incrementality will be significant, in others it won't. It looks like that in Uber's case, the incrementality was low. Btw, if you're not familiar with this term, please look it up - there's a ton of literature on it: https://www.adroll.com/blog/marketing-analytics/beginners-gu...

Which brings me to the point about bad employees - marketers who are not adjusting their performance measurement based on incrementality should be fired (and it certainly looks like this was not being done at Uber at that time).

You think Uber's marketing people are extreme outliers compared to the typical large company?

I interviewed at Uber for a marketing position back when they were small, and decided not to take the job (huge mistake). At least back then, the people I met were super smart (a lot of them were ex Facebook, again back when those teams were super good). I never met Kevin Frisch and don't know if anything he said was taken out of context, but the first impression is not good. Also it's a bit strange for a well-respected Uber guy to go to Intuit - it's not a common career path.

"Chief marketing officer of Unilever" isn't the guy running experiments, he's the guy that hires ad agencies and auditors that in turn have teams of data scientists running various experiments for Unilever.

I guess you could propose a conspiracy theory that all these people are conspiring to put together fake reports and data sets, but that sounds extremely unrealistic - ad agencies are essentially money-counting businesses like banks, and they take data security very seriously. (Because usually they'd be audited at every step.)

In short, smart people have already thought about these issues, and there isn't any inherent pro-advertising bias in industry, rather the opposite.

"No one is willing to run the experiments necessary."

Tesla is one natural experiment about not spending money on advertising in the mass media compared to traditional car companies that spend HUGE amount of money advertising.

https://www.motorbiscuit.com/gm-spends-an-embarrassing-amoun...

"Hyundai spent $4,006 per Genesis vehicle sold in 2018. Ford’s Lincoln brand came in second with $2,106 per vehicle sold. After Jaguar and Alfa Romeo, GM’s Cadillac brand came in fifth with $1,242 spent per vehicle sold. Tesla was the lowest at just $3 spent per vehicle sold."

Tesla's marketing spend is whatever it costs them in legal fees and fines to keep the mouthy celebrity CEO - and their high-profile campaigns in 2018 seemed pretty effective.

Telsa is in an enviable position of selling most of their cars before they're produced. In that position, you don't really need to advertise much. They also get a lot of PR to keep up brand awareness.

If Hyundai couldn't keep Genesis vehicles on the dealer lots, they'd advertise them less too. Having a dealer network means dealers that want manufacturer support in advertising to keep dealers happy, even if the new cars sell themselves, dealers need to get people in to sell used cars.

> there was no change in buying behavior.

I suspect that if our browser isn't blocking ads then our brain is. It's complete conjecture, but I assume that adblockers eliminate this cognitive load explaining a portion of why they became successful before they were necessary for security/malvertising.

Beware extremes like "no one can actually prove". One difference between internet ads and their more ethereal tv and radio predecessors is that adviews, clicks and purchases can be tracked. Also, there are techniques that can make even TV, radio, print and even digital to physical world ad performance more visible: coupons, response codes and campaign-specific phone numbers and URLS. That Uber was buying hundreds of millions in ads and could not attribute performance (sales, for example) speaks more to poorly designed campaigns and potentially very bad actors in the supply chain.

There was a good piece of reportage on this subject last year:

https://thecorrespondent.com/100/the-new-dot-com-bubble-is-h...

This story was on the Freakonomics podcast too. It definitely seems like a bubble, but there's no external pressure to make the bubble pop. With the mortgage crisis, eventually, people can't make the mortgage payments and the bubble bursts. But with people happily spending money on ads that don't work, there's no external pressure to stop. Will this bubble ever burst?

Back in 2017, Uber sued Dentsu for ad fraud

https://www.japantimes.co.jp/news/2017/09/19/business/corpor...

I'm only about halfway through the first of the podcast episodes you linked (thanks!), but just thinking about this logically for a moment:

I would not be shocked to learn that advertising for a specific product or event is not particularly effective. However, I'm inclined to believe it has a huge effect on overall brand recognition.

Let's say you go to Amazon to buy a roll of toilet paper. How do you choose from the literally hundreds of options? You could spend a day of your life reading reviews, and trying to parse which ones are fake. Or you could buy the toilet paper from Scott because you recognize the brand.

As I see it, buying brand advertising is a lot like buying an expensive suit. It's not that the suit makes you more productive, but it is a sign of professionalism, and—frankly—of wealth. If a brand is advertising everywhere, you know they aren't a fly-by-night company, and their products likely meet some standards of quality.

There's also the bit about luxury (mostly) car commercials. Half their purpose is to remind you that you made a good decision buying your <brand> car, and maybe your next car should be the same brand.

Additional resource discussed in the past:

https://news.ycombinator.com/item?id=21465873

I'm a fan of the podcast but one argument they cited seemed to have a pretty glaring error - they looked at the case where eBay was comparing incremental gain on search ads over no ads. It's methodologically hairy because eBay is a very major player with significant brand recognition.

"When Tadelis was working for eBay, the company was in the practice of buying brand-keyword ads. Which meant that if you did an online search for “eBay,” the top result — before all the organic-search results — was a paid ad for eBay."

This doesn't show that advertising doesn't work per se, it shows that eBay didn't hire a competent ad buyer. Whether or not you can prove the efficacy of advertising as a whole, this is not a valid approach.

Yes of course they can. They can run trials in different areas, and see if there’s an increase in spend. They run tests all the time.

it’s funny how trillions of dollars in market cap are built on top of some service with unprovable and questionable value.

I work in digital marketing and this episode made me want to tear my ears off.

First of all, they didn't differentiate between display or PPC advertising. PPC you only pay if the user actually engaged with the ad. Nearly all of the anecdotes they used where about online display advertising - a known crock.

And we absolutely run experiments all the time! In fact, we tie our ad spend directly to conversions. If anything, the market is too efficient - it's really hard to get more than what you are paying for.

But the experiments you run pit one ad type or channel against another. Have you ever run any experiments comparing ads to no ads?

I have. When I run ads, I grow. When I stop running ads, I don’t.

Absolutely. You can give a campaign its own website or URL if you want to test something like the effect of a specific billboard or radio ad.

Or you can do the equivalent of an excess mortality study but for client acquisition or product sales. You take something like the month of January and you say "this is what we expect for web traffic given our standard traffic for that month, and extrapolate it with your average web activity growth". And so long as you have "clean" months to compare it to (with no ads running), you can get a pretty decent back of envelope about what your ad campaign did.

Like, there's so many different ways you can cut the apple, it's patently ridiculous they made those claims.

To give them a bit of credit, there's something to be said for there being a lot of bad ad spending out there. But like the stock market, there's a tacit understanding that it's more or less efficient.

  PPC you only pay if the user actually engaged with the ad

Who defines that "engagement" legitimately occurred? The ad network.

Example: TVguide.com/listings. On most devices, I get a nondismissable audio ad when I'm looking at telecast schedules. I kill the audio immediately.

I never experience the bulk of the ad, and even if I can identify the product being advertised, I'll form a strongly negative impression of them.

I don't count this as "engagement", but it's charged as that.

since you work in the industry - how about your opinion on the Uber case in the original post? that is PPC.

I'm struggling to follow the actual narrative presented in the chopped up Twitter style. It sounds like there were was a third party ad network involved?

One thing we learned a long time ago is never to trust "engagement" data from platforms (be it Google, Facebook, LinkedIn, whatever). There are enough random user behaviors that you generate a ton of noise. You need to build larger, more robust attribution models that tie ad spending directly to revenue sources - new leads, new accounts, purchases, etc. Patting yourself on the back for how many shares or clicks you get is not great.

as a nonexpert its hard for me to say too but perhaps you will benefit from hearing directly from source - the performance marketing guy at Uber who found all this out: https://www.marketingtodaypodcast.com/194-historic-ad-fraud-... honestly i am not sure if he did a good thing by asking all these questions, or was partially to blame for gross incompetence managing a 150mm ad budget, or both. (the chronology of when exactly he took over matters, it seems he was only in the job about a year)

and of course for more detail the full text of the fraud suit is public https://www.adexchanger.com/wp-content/uploads/2019/06/Uberf...

This is really, really not true. Advertising lift has been well studied, especially in metastudies spanning hundreds of digital campaigns across Facebook and Google. These are independent academic metastudies, with hundreds of millions of impression data samples.

Positive lift in the range of 0-20% is very common, and many statistical aspects of causal inference on ad impacts are well understood.

Negative lift and flat campaigns are real phenomena too, and it does deserve more widespread publicity that negative lift happens in an appreciable number of campaigns, but that doesn’t take away from the overwhelming evidence that digital advertising works and that the mechanics of positive lift are well studied.

Here are two of the foundational papers in this area:

- https://www.kellogg.northwestern.edu/faculty/gordon_b/files/...

- https://www8.gsb.columbia.edu/media/sites/media/files/Garret...

Particularly Figure 1 (page 26) in the second link. That figure alone utterly refutes any nonsense claim that digital advertising doesn’t have provably positive ROI.

Did you read the transcript the for episode and the associated links and notes?

They back up their claims with studies of their own as well as metanalysis.

I did, and I think there's a lot of p-hacking going on. Leading with TV advertising is disingenuous at best.

The ebay example twists the concept of user acquisition (new customers) and purchases (new or old customers). It is a common tactic to buy advertisements defensively, for example, if you're a product manager, and have determined that some of your user base are more transactional rather than frequent.

Another pet peeve I have is how they conflate direct response advertising and marketing.

It would seem their backing research and metastudy totally missed the boat and ignored well-known academic experts on the matter.

I definitely trust the academic studies over an entertainment podcast.

I get downvoted whenever I propose digital advertising bubble will pop once people wise up to the fraud.

The ad buyers aren't smart enough to measure the actual effectiveness of their ads, and the ad sellers are not incentivized to teach them how to do it. This can go on for an arbitrarily long time.

I think the next major recession is going to be triggered by this bubble popping.

Why would it cause a recession? Google and Facebook would report lower numbers, literally every other company on earth would save more money.

IDK. It's a risk worth chewing on. eg If Google and Facebook tanked, bringing down the stock indexes and all the institutional investors, triggering a sell off... What would the knockon effects be? I don't know (no clue). The 2008-9 crisis largely happened when CDS stuff popped. Is there a similar house of cards on top of FAANG stocks or the digital ad market or...?

2008 was bad because of excessive leverage. Top tech companies have little to no debt and there aren't any leveraged securities to do with ads.

Ecommerce sites have very fine grained measurement of their advertising spend and know exactly the ROI (which is why they focus so much on retargeting)

This is completely false. Large advertising platforms have many A/B tests that show significant differences in consumer behavior between groups that receive different ad treatments.

Ads might be less efficient than some believe, but it's super easy to see that they "work", and advertising platforms do it constantly.

That's comparing one type of ad or channel to another.

But none of them test "no ads" vs "ads".

Some of the A/B tests I have analyzed are exactly "no ads" vs "ads"

This is nonsense. I had a startup completely powered by Google Ads. Ads brought in nearly 100% of the traffic. When my billing information with Google got mixed up and the ads stopped, the traffic went to 0 immediately.

That means you have zero loyalty among your consumers. Don't you see that as a problem?

Nope. They needed me for a specific job. Got their money, gave their thing, and they moved on.

Though not an intentional test, we found out what happens when a movie gets a wide release but does not advertise. In 2008, the movie 'Delgo' was released on over 2000 screens with nearly no advertising. Because the production company could not find a distributor, they spent their ad money on renting the screens for a week, with the hope that some people would randomly see it and word of mouth would spread, leading to the theaters wanting to keep it for additional weeks. It became the lowest earning wide release movie up to that point in time. Each screening averaged two people per screening. More people saw Conan O'Brien making fun of the movie in his monologue than actually saw it in the theater.

The reviews for the movie were poor but it had lots of household names as its voice talent, including Anne Bancroft in her final film. Good or bad, advertising likely would have lead to more people seeing it than two per screening. Of course there's no way to know for sure how many more would have been enticed by the ads but we do know that going with zero advertising resulted in a huge disaster.

So the movie was bad, and people didn't go see it? Doesn't seem related to advertising.

Eminem released his last two albums without even announcing them. Word of mouth got them both to #1 on the charts.

If you tried that same movie experiment with something like Avengers, I bet the results would look a lot closer to Kamikaze than Delgo.

These are not fair comparisons. Eminem and the Avengers have already established identities. Music cannot even compare to movies due to the sheer quantity of modern musicians. There are plenty of fantastic (IMO) bands out there that release music without advertising and they get nowhere.

As for movies, Avengers got to where they are today through lots and lots of advertising.

> It became the lowest earning wide release movie up to that point in time. Each screening averaged two people per screening.

On the other hand it got displaced by Oogieloves which had $40 million in marketing costs. Critics found it mostly bad and the only award it won was for films produced in Brazil. Maybe there was a reason no one wanted to spend ad money on that movie?

Maybe not for companies of uber’s size/current reach, but small businesses definitely do benefit from ads. They see an immediate uptick in sales when they start advertising on various platforms.

Ads are somewhat like propaganda. There's no proving it will work or not in short terms, but nobody can deny that persistent propaganda has a long-term effect on the whole population. You have think about a kid/youth who listens to some propaganda for years when growing up. If say you show an ad for a new Apple product. A small part of it is to inform consumers about it, the longer larger part is to enforce the Apple brand which has been happening over many years. You can't say well it is not quite working so let's stop doing Ads.

This article also sheds a lot of light on the whole issue of why people don't realize when digital ads don't work: https://thecorrespondent.com/100/the-new-dot-com-bubble-is-h...

> No one is willing to run the experiments necessary.

This is incorrect.

Depending on what kind of advertising you are talking about. Direct response advertisers measure ads to the cents. And they know exactly what the ROI is and where the customer is coming from.

They mention the common retort to this which is very bizarre to me: "If online ads don't work, then huge companies wouldn't spend billions of dollars on them. Therefore they must work. You academics are just missing something."

Ok, how does those companies know it works? They don't have any real data to show that it does, just the fear that if they stop they'll lose a lot of business.

They do have data. It's pretty easy to see sales without ads vs sales with ads and then do further testing to narrow down results from there. This has been done for over a century since the first billboards were put up.

The amount of data generated by adtech today is staggering. The problem isn't data or advertising, it's the wrong people running the wrong campaigns for the wrong reasons.

>No one can actually prove it has any ROI at all. No one is willing to run the experiments necessary. In the few cases of natural experiments, where ads got turned off for some people by accident, there was no change in buying behavior.

This may be true but it's a separate issue than the fraudulent clicks.

Brand advertisement (the kind you are talking about, as opposed to closed-loop direct advertising) is an investment in a brand, that doesn't get paid off in a day, or a week, or a month, or even a year. It adds fractions of a penny onto a customers value, every day from today into eternity. It's an investment in your mind, but the long time horizon makes it, as you point out, nigh impossible to measure ROI.

However, this is one of those cases where despite being immeasurable, it still works.

Imagine doing a study on low-fat or low-fat diets and trying to measure health outcomes like lifespan, or heart disease, or cancer after just one month. You can't do it. The best you can do is measure markers of these outcomes, like insulin resistance, blood triglycerides etc. Brand advertising is similar. You measure markers of long-term purchase intent. It's not perfect.

And just like actually doing a long-term study of diet for example is riddled with confounding variables and very hard (nigh impossible) to do well, so it is with advertising.

> No one can actually prove it has any ROI at all. No one is willing to run the experiments necessary.

This is entirely false. I work in adtech, and almost all companies run experiments in order to optimize their ad spend (everything from Ad A vs Ad B, to Ad vs No Ad, to Channel A vs Channel B, and more).

This isn't to say that advertising always produces ROI. Quite often, experiments will be run that will show a certain strategy isn't performing well, and the company will adjust accordingly. It's incredibly naïve to think that companies are flushing half a trillion dollars a year down the toilet on advertising without any attempt to validate their investments.

That's great, thanks for your experience, but can't comment on the argument provided?

Isn't it possible that these experiments you talk about are fatally flawed? I have serious doubts that a company can run and do well designed statistical experiments when academic experts are plagued by p-hacking and other foot guns.

That’s kinda a weird take when there’s every incentive to be the the person/manager/director who saves tens to hundreds of millions of dollars in ad spend.

Sure marketing people want to protect their jobs but the odds of “advertising is entirely worthless” being this closely guarded secret kept by millions of people in the industry or being a collective delusion is pretty darn low. The reward for defectors is just too high.

IMO large social media platforms are actually much more capable of running these experiments than academics. (Disclaimer: I work on ads at one of these companies).

Platforms can accurately determine who engaged with an ad (basic logging on their sites), they have infrastructure to create statistically balanced ad experiments, and can also accurately determine whether a conversion happened (either through a conversion pixel or through data brokers).

Running these tests on behalf of advertising clients, or for internal research is fairly standard. If we couldn't prove statistically that our ads were working, I would have left a long time ago.

I think the difference from academic studies that fail to replicate is less about capability and more about the fact that experimentation in adtech is fundamentally straightforward and occurs under far more idealized conditions - those you mentioned, plus massive sample sizes and easy iteration - than most academic research.

Of course, this just reinforces your point that experimentation in adtech is largely not subject to the same issues that have fueled the replication crisis in academia.

(I'm a data scientist but not in adtech.)

I'm not sure what argument you're referring to? I quoted two claims from the parent comment and refuted those based off of industry experience.

> I have serious doubts that a company can run and do well designed statistical experiments when academic experts are plagued by p-hacking and other foot guns.

By "company" are you referring to individual advertisers or digital advertising platforms? Why do you doubt that either are incapable of running well-designed experiments? It's in both parties best interest to spend ad dollars as efficiently as possible.

While it's certainly possible that every one of the 100 million+ ad experiments ran over the past decade have been fatally flawed, it's highly unlikely. I guess the alternative scenario is that digital advertising is one giant hoax being kept secret by the millions of employees who work in the industry, but given how easily tech news leaks, this also seems extremely improbable.

Another option is that the experiment design was wrong for every single one of those experiments. Sounds unlikely right? Running the same broken experiment any number of times won't change that it's broken. So if these "100 million+" ad experiments are more or less copies of each other they could all be wrong. How likely is that? I don't know, but I do know that at a big Silicon Valley tech company that you've definitely heard of I was involved in an exercise where we were told to create an experiment. The type A take charge person in our group did what they do, which in this case was talking over and shutting down the Los Alamos trained particle physicist who was trying to explain to her why her "experiment" could never show anything.

> without any attempt to validate their investments.

I've also worked in adtech, for quite a long time on all sides, and my experience is that far more companies do research to justify their ad spend not validate it.

I have managed plenty of A/B tests in my day, each one claiming to show some improvement, 10%, 15% etc (some, as you said, showing no improvement). Even though these tests are often run "correctly", essentially nobody goes back and asks "wait, we had a 10% increase multiple times in the past year, but is our <metric> really showing the cumulative improvement we expected?"

The greatest trick in the industry is that because VC are pouring money into everything, everywhere, every metric appears to grow. At every startup, everywhere, pre-pandemic, numbers were going up because people were pouring money into the system.

I've worked at companies who I know for a fact their adtech product cannot and does not work, yet their business continues to explode because in recent months nearly all ad spend has been on digital ads.

I've talked to companies whose entire function is bidding optimization who literally do not understand how to optimize bidding given the information you have.

Absolutely people are running "experiments" but the function of the "experiments" is to justify that the ad team is worth having, and then that the VP of marketing is doing their job and then that the CEO has hired some super smart people, and then that the VCs might have really found a unicorn. Everyone sees what they want and no one really wants to ask the question "wait, does this really work? are these tests really able to capture the complexity of the environment?" And if you are one of those ornery people that insists on probing into the details and seeing if any of this is working, you will eventually get fired.

Ad tech is largely a scam, but a huge number of rich and smart people benefit from the illusion that it is not, so we continue to see experiments showing that everything works as expected.

I agree that there a lot of (smaller) players in the ad tech space who are scamming companies out of money. And especially if you've been in this industry for a while, this practice was much more prevalent a decade ago.

I don't think your view is representative of how most F500 companies, or many of the new DTC brands, invest on major ad tech platforms though (FB/G/Snap/etc.). Experienced marketers try to measure metrics as close to core business KPIs as possible, and comparing ads based off of simple A/B tests is increasingly becoming a technique of the past. Measuring the incrementality of ad campaigns through long-term holdout groups gives companies a much more accurate read of their investment, while bringing higher statistical rigor as well. So rather than looking for cumulative increases on <metric> after advertising for a year, you could instead point to group A (who saw no ads) and see that group B (who saw ads) drove a 1.5x higher <metric>.

I'm not sure how you can say advertising is a scam when there are clear examples of popular brands that most likely wouldn't exist today without digital ads? Take Allbirds as an example. There are dozens of consumer shoe brands that are already in physical stores and have higher brand recognition; how can you argue that advertising didn't help them cut through the noise and grow their bottom line?

> I work in adtech, and almost all companies run experiments in order to optimize their ad spend ... It's incredibly naïve to think that companies are flushing half a trillion dollars a year down the toilet on advertising without any attempt to validate their investments.

We have a very large and conspicuous example here that suggests that Uber hasn't been doing this.

Running experiments like you suggest is difficult and to do them right requires some fairly specialized knowledge to do correctly. I'd doubt more than a small percentage of companies have the expertise to effectively run any kind of advertising experiment that returns useful data. Most businesses lack that knowledge and rely on metrics provided by the people selling the advertising—who are likely to provide self-serving numbers.

While I'm sure some of the bigger F500 companies do a good job validating their ad spend, I most companies don't even know how. Certainly the majority of small businesses have no idea how effective their advertising spend is aside from very crude word-of-mouth feedback. I suspect this creeps way up into the Fortune 500 as well.

> We have a very large and conspicuous example here that suggests that Uber hasn't been doing this.

I'm not sure this article is the smoking gun that the author makes it out to be. By 2017 Uber had reached saturation in what could reasonably be considered their addressable market. From my perspective, turning off app install campaigns and not seeing any dip in acquisition validates this position.

Meanwhile, their other major app, UberEats, is a business that exists in a highly competitive market, and they continue to invest in app marketing in order to grow that division. If the takeaway from this article was that Uber discovered app marketing was useless, I doubt you'd see them make that same mistake again.

> While I'm sure some of the bigger F500 companies do a good job validating their ad spend, most companies don't even know how.

If anything, I think bigger companies are more prone to overspending/spending inefficiently. While they do experiment to try and optimize strategy, there is some business inertia that gives them the flexibly to move a little slower, since one poor marketing decision will not sink the business overnight. Conversely, small businesses don't have the resources to investment as much in marketing. This leads to poorly run ad campaigns, but rather than continuing to invest in a bad strategy, they typically just kill the effort altogether.

Most small businesses don't need complex experiments to understand how marketing effects their bottom line - poor ROI is a lot more apparent when you're low on funds. That's also why it's in an ad tech platform's best interest to make efficient advertising as accessible as possible.

> No one is willing

Counter-example: I know of at least one consumer goods company that has studied the long-term effects of certain kinds of sponsorship deals on consumer behavior. The study had tracked people for at least 10 years at the time I learned of it. Of course, this company would never publish the results, because they provide a competitive advantage in structuring and bidding on sponsorship deals.

You are conflating value of the marginal ad with value of any ads at all.

You are conflating value of the marginal ad with value of any ads at all, and also exaggerating what those articles say

One of them said tracking cookies only boosted conversion 4%, and another said P&G did better with some traditional media advertising than some digital advertising.

> No one is willing to run the experiments necessary.

Because advertising is like military spending. It takes a lot of money to maintain the status quo.

This is patently false. I have friends who have worked years developing solutions for doing ad effectiveness comparisons. They are A/B tests in most cases but some methodologies require a lot of sophistication because of problems tracking conversions.

Traditional ad platforms like TV, newspapers might not have done this but online ones surely do. Infact that is one point they consider an advantage as you can measure effectiveness unlike the traditional platforms.

This is blatantly false. It’s incredibly easy to prove that digital advertising is effective using A/B testing.

I think they did the A/B testing by switching off 100m of ads and having the same results.

until you realize that ads are meant to make FB and Google rich, not you, you're going to have a bad quarter.

This was a superficial analysis at best. Adtech produces petabytes of data every day proving that advertising works. There's a reason why two of the most valuable internet companies sell ads.

The problem is knowing exactly which formats and campaigns are working down to the dollar, but part of that is just the reality of fuzzy attribution and it's only getting harder as privacy regulations get stronger. However you can definitely tell the difference when turning everything off, and if you can't then you were advertising to the wrong people in the first place.

Uber's mistake isn't that advertising didn't work, but rather that they didn't vet their vendors or even bother doing any checking and optimization of their own.

The implication here is that money spent on ads is largely wasted.

Companies like Uber and Ebay have turned off all their adspend and saw little to no change in their acquisition metrics. You can argue that they were just doing it wrong. But the point is that, if even they are doing it wrong - and getting nothing in return for the millions they're spending on ads - then it's very likely most others are in the same situation.

You are right that this doesn't mean _all_ advertising is useless, there are absolutely profitable usecases. But the larger points still stands: most money being spend on advertising right now is likely not returning anything.

We now have some strong precedents being set. I believe this will cause more major companies to run the ultimate experiment: turn off all ads and see what happens. It's too early to tell, but it's not impossible we'll see adspend drop significantly across the industry once everyone finds out they're just burning money.

Why the ad spend isn't working is a critical detail. It's definitely not most money but yes there's a lost of waste because it's a 12 figure industry with lots of politics, perverse incentives, thousands of vendors, and a complex global supply chain.

Fraud is a special case because it's criminal activity and has nothing to do with advertising. It happens in every industry but it's especially easy with online technology spanning multiple countries and data that can be easily faked. Uber was exceedingly oblivious here but I wouldn't extrapolate advertising efficacy from these examples.

At least in e-commerce it’s very much possible. The company that I work for does such experiments regularly (there’s a dedicated Data Science team for measurement) and I’ve personally been involved in lift studies for Google Ads. They work, you just have to be careful with the ‘how much’ combined with ‘for what’.

Happy to chat with anyone who is interested in the topic (pfalke at pfalke dot com).

Haven’t had a chance to listen to the podcast, apologies if that made me miss the point of the parent post!

It's an arms race. In theory advertising should give you an edge over non-advertising competitors, but if everyone is doing it demand remains unchanged and also you're wasting money

Lewis and Rao published a meta-analysis of 25 large scale controlled advertising experiment [1]. Here's the abstract:

Twenty-five large field experiments with major U.S. retailers and brokerages, most reaching millions of customers and collectively representing $2.8 million in digital advertising expenditure, reveal that measuring the returns to advertising is difficult. The median confidence interval on return on investment is over 100 percentage points wide. Detailed sales data show that relative to the per capita cost of the advertising, individual-level sales are very volatile; a coefficient of variation of 10 is common. Hence, informative advertising experiments can easily require more than 10 million person-weeks, making experiments costly and potentially infeasible for many firms. Despite these unfavorable economics, randomized control trials represent progress by injecting new, unbiased information into the market. The inference challenges revealed in the field experiments also show that selection bias, due to the targeted nature of advertising, is a crippling concern for widely employed observational methods.

[1] https://academic.oup.com/qje/article-abstract/130/4/1941/191...

This is now at 700 comments, and no one is talking about the math in the Twitter "thread". Found no mention in the Twitter replies either. Is it just me or is the math hilariously off?

[0]: "10% of $150M is $15M". Correct, but do we need a screenshot of a calculator for that, let alone Google? (why do people rely on an internet connection and Google for this?)

[1]: Talking about "2/3", but putting in "0.75". So which is it?

[2]: We are at $50M now. "Knock off" $20M, leaving us at $30M. Yet she puts in $20M into her Google calculator, gets 13.3% and calls it "a little over 1/10". Fair enough, but the real number is 30/150, so 1/5, 20%, double of "1/10".

Come on, this was not rocket science math! Should have double-checked before publishing, especially if you're a business consultant [3].

0: https://twitter.com/nandoodles/status/1345786303263219712

1: https://twitter.com/nandoodles/status/1345792380184760321

2: https://twitter.com/nandoodles/status/1345794089825021963, https://twitter.com/nandoodles/status/1345795168105066496

3: https://twitter.com/nandoodles/status/1345853158279565313

Yes, but marketers don't really care about math. Only about the numbers!

You would expect a few hiccups and a steep learning curve when the product they are trying to buy is a bid for visibility on insanely complex networks of platforms using said rocket science to guess where one's mind is at.

Disclaimer: worked years in the space on the topic of performance, only one time ever have i seen an AB test correctly run and correctly understood by a client. Most requested false numbers because they couldn't understand the difference between branding, attribution and incremental sales. To be fair, neither could most anyone working in the field... Seems like that doesn't change too quick!

Yes, but marketers don't really care about math. Only about the numbers!

This is hilarious.

And sad. Same thing happens in data science now - people are happy to have any number regardless of correctness. They just need something to report on and keep the work going.

That famous saying known to those of us in scientific computing: The purpose of computing is insight, not numbers.

As long as data science remains subservient to Product, the inevitable result is cherry picking and post hoc rationalizing of some PM's intuition. To blame is the "embedded" model in which DS works within (and takes direction from) a cross functional team driven by a PM.

Hah, exactly what I was writing about here

https://www.jacquescorbytuech.com/writing/marketers-addicted...

Hi, please don't do this. If the comments become just replies saying "I wrote about this on my blog!" with a link, they're unreadable. If you summarized your thoughts and posted it as its own comment, without a self-promotional link, I'm sure it would be very well received.

Sounds fair! Thanks for the feedback. The article did spur an interesting discussion previously on HN when I published it but seems relevant again given the ongoing discussion.

https://news.ycombinator.com/item?id=25016532

This is one of those things that isn't a big deal in isolated instances, but becomes an issue if it gets more prevalent, so it's always hard to know when/if to even say anything :)

The other problem is that the worst offenders are the ones least likely to respond to getting or seeing this sort of nudge, so there is some adverse selection going on.

It's probably self correcting over the long term, but meanwhile we lose the sort of links that would (or rather, might) enrich the conversation through overcorrection by posters.

It's the obverse side of the slippery slope, I guess.

I had the same experience when marketeers talk about statistics, all of the most basic errors are made (e.g. when A/B testing or channel attribution).

You are right, but also you asked for it by being pedantic: "common" should be "come on" :)

Or "c'mon," if you want to abbreviate it.

Oh yes, fixed it.

Some one had mentioned it in the thread. Looks like it's deleted now.

As for the calculator screenshot, that kind of stuff feels like a consequence of having 'show your working' drilled into your head at school. Which is great to show your teacher that you understood the question and didn't plug it into a calculator, but unnecessary later on.

Send it to your university professor and see how it goes.

You can’t really blame academia for this one.

academia != education

The education system incentives students to be teachable to conserve teacher resources.

Transparent calculations are more important to help folks with low numerical skills.

I think this is actually a really hard challenge to communicate simple calculations easily to a wide audience.

>[0]: "10% of $150M is $15M". Correct, but do we need a screenshot of a calculator for that, let alone Google? (why do people rely on an internet connection and Google for this?)

She knows her audience.

Thanks it was causing me headaches when I was reading the thread. I thought it was some marketing terms and calculation I had no idea about.

What how much did they actually saved? Apology in advance because I really couldn't grasp what she was saying.

>(why do people rely on an internet connection and Google for this?)

Because most computers lack a copy of bc in /bin.

You think if every computer had bc in /bin, or the equivalent, most people wouldn't still be using Google for math?

It's not the lack of tooling, it's the lack of knowing how to quickly get to the tooling. For most people, especially non-technical, Win+"calc"+enter is not intuitive. Ctrl+T+$formula+enter is. That's the difference.

Win+calc+enter searches being for a calculator on my workstation, Win+cal+enter opens a calendar, and Win+calc+enter works fine for me! Yay!

I use google for maths because it's the fastest thing I can get to and TYPE my maths in without having to open a new application. You can even just type it in the search bar of Chrome/Edge and see the result immediately

It‘s ok to use google for calculations. But kind of strange to show a calculator screenshot as proof for a 10% calculation...

Both Windows and macOS support calculations in their respective system-wide search bars (Win key on Windows vs Cmd-Space for Spotlight on macOS). They both support pretty cool stuff like sqrt(9), 13^37, but Spotlight edges it slightly by also support bit shifting like 1 << 10 == 1024.

I did not know this - thanks!

Is this a Cortana feature? Nothing pops up for me (WIN key), and a search online reveals it seems liked to Cortana, which I seemed to have disabled.

Start typing "calculator" after pressing WIN.

Win+R - Type in "calc" hit enter. Works every time, not only when Windows search decides to work.

Nice tip, I just realized spotlight has currency and unit conversions that I typically get from google.

> why do people rely on an internet connection and Google for this?

It's faster than opening a dedicated calculator program. If you're in a browser already, just ctrl-t and type your equation.

CMD-Space is just as quick and works system-wide. There is a Windows equivalent as well. Works when the net is down too.

why are you so focused on people using the google calculator? They built it, people will come.

who cares!?

I want to give her the credit of making some sort of dramatic point, but yeah it felt a little condescending.

Edward Tufte seems to be here on earth to suffer for some offense in a previous lifetime. To know how badly statistics get abused, to be able to describe it, and be heard by a fraction of people who can't manage to fix the problem either.

No wonder he fucked off to a farm to concentrate on sculptures.

You can A/B test ads pretty easily, and this is quite common. With some degree of statistical certainty, you can tell how one ad performs to another.

You don't have control over your SEO results as well - but you can also measure against SEO traffic with a high degree of certainty.

All big companies do this.

Sure, you're never going to know exactly how many people you advertised to would have organically, eventually found your product and bought it.

But that honestly doesn't seem that important compared to the other metrics - which most functioning large companies have decent data on.

You're also never going to know how many of your customers ate Green Eggs and Ham for breakfast. It's irrelevant. You have decent insight into your direct-online ROAS, and that's unique to direct online advertising, and it's important!!

Yes, you're never going to know if that million dollars you invested in online ads was the best use of that million dollars. But that's not much different than building a new factory, either.

Edit:

Specifically to Uber's case - they did NOT turn off 66% of ALL ads randomly to no adverse effect (implying that all ads are worthless).

They DISCOVERED that a certain type of ad (paying for installs on dubious ad networks) was mostly fraud. After turning off 100% of this type of ad - they found no adverse effect.

This is a fail on their analytics team. They should've been measuring this type of ad better - especially given how big a portion of the total spend it was - and had insight into something not being right. They should have been able to do this - and if they couldn't, because the network somehow didn't give them enough data to do it, they probably shouldn't have been spending this much money for exactly these reasons!

> Sure, you're never going to know exactly how many people you advertised to would have organically, eventually found your product and bought it. But that honestly doesn't seem that important compared to the other metrics...

Wait, why wouldn't that be important? It seems like the most important question since it asks whether advertising has any significant impact at all.

It's critically important for adtech companies' customers, since it's an essential part of determining return on investment.

But it's also critically important for both adtech companies and data scientists who work in the space to direct people's attention away from those sorts of metrics. You generally don't want to call the attention of the person who signs your paycheck toward the fact that it's all but impossible to really know for sure if your service has delivered them any net benefit.

This is creeping pretty close to the legal definition of fraud.

But advertising has had a fair amount of safe-harbor carve-outs for over a 100 years or so that are not available to other industries. So it is no surprise it continues today.

edit: I hate to say it but the auto-downvoters here on HN are approaching reddit levels. Everything I said above is true.

This is the industry whose job is to manipulate and deceive people. It should surprise no one that, if they do that to others, they'll do that to themselves as well.

> "edit: I hate to say it but the auto-downvoters here on HN are approaching reddit levels. Everything I said above is true."

It's not that you're wrong, it's that you're boring.

"I knew this years ago, look how my cynicism is superior to your naievite" is a boring comment to read. You could explain why you think this is any closer to fraud compared to selling anything else and claiming it's the best in its class, or what safe-harbor carve outs by whom, or why they matter more than other effects, or what you think should be done to change them, or anything more substantial than "I'm not surprised". Oh aren't you? Great, cool story bro. Except it's not /even/ a cool story.

It's old man "get off my lawn". It's every tech forum's old-man status grabbing which is now approaching Reddit /r/SysAdmin levels on HN. The only question is whether aggressive downvoting can curb its growth here before it gets a choke-hold.

Please don't cross into personal attack and name calling. It only makes things considerably worse.

https://news.ycombinator.com/newsguidelines.html

I will rephrase as "your comment is boring" rather than "you're boring" in future; my intent was to describe the comment and not to attack the author's person.

That would certainly be better.

It's sad to see the downvote culture come here. Downvoting is meant for removing contributions that don't add much to a discussion, not for indicating whether someone agrees or disagrees with you. It seems to be a norm that is spreading though, sadly.

It would appear jodrellblank is perfectly demonstrating how to misuse the feature, by calling you boring. Not exactly the kind of attitude that we want to have around here.

The downvote button should be replaced with an "off topic" flag. If downvote is opposed with upvote and upvote is understood to mean "I agree" or "this is informative" then downvote would be expected to mean "I disagree" or "this is bunk". It's semantically misleading.

> The downvote button should be replaced with an "off topic" flag.

I hope not. Some of the most informative, useful comments I've seen on Hacker News have taken the form of wild tangents.

I do see where you're going, though, that if "I agree" is sufficient cause for an upvote, then it's only reasonable that a vote in the opposite direction should be allowed to carry the opposite semantics.

It's just that, from a purely social standpoint, that's not actually how human beings typically think, feel, and react. Perhaps it could be that simple on the Vulcan version of Hacker News.

Oh for sure—wild tangents are fine as long as they're unpredictable, which is what makes them interesting. The offtopicness we try to avoid are generic tangents, the kind where people get sucked into the same argument about some topic that they always get sucked into. Those are the black holes of internet forums—it's best to steer clear.

https://hn.algolia.com/?dateRange=all&page=0&prefix=true&sor...

https://news.ycombinator.com/item?id=117171

To be clear, "off topic" here's just an example. My main point was the semantic implication of a downvote - I've seen the argument of "downvotes are not meant for disagreement" expressed here previously.

> It would appear jodrellblank is perfectly demonstrating how to misuse the feature, by calling you boring.

I don't have an opinion on what "correct" downvoting culture would be, one way or the other, but - if you say "contributions that don't add much to a discussion" should be downvoted, then isn't a "boring" comment precisely one that _should_ be downvoted?

I also always thought that downvoting should be reserved for low quality comments, but I had been corrected and even dang said:

Downvoting for disagreement has always been ok on HN.[1]

I don't really like it, but it has always been the case on HN. The question is if the community voting behavior has actually changed.

[1] https://news.ycombinator.com/item?id=17666145

Downvoting for disagreement is okay? That is new to me. Well okay then. I will proceed accordingly.

Thanks.

From somewhere in the warren of links to dang posts I traced out from there:

> I think people have the wrong idea about HN downvotes because they think Reddit rules apply to HN.

That one gets me, because I had previously assumed that downvoting for disagreement would be a standard Reddit thing, but that HN's community is supposed to strive toward a more "we're all mature adults here" kind of community standard.

That's not true. Downvoting for disagreement has been ok on HN from the beginning.

I realize people have strong feelings about it, but that's because of the psychological intensity of the mechanism. There's no way HN would survive without downvotes, despite the fact that not all downvotes are fair. It's a critical part of the immune system.

https://news.ycombinator.com/item?id=16131314

I have noticed the same thing over the last four or five months. I’ve been curious if a voting ring has sprung up on HN or maybe a COVID impacted demographic suddenly has a lot of time to patrol the site, or if it’s something else. It would be really interesting to have some anonymized voting data to analyze and see how downvotes map to accounts, geography, and economic trends over time. Though voting data must be relatively huge compared to content and I bet it gets discarded after items close to replies.

> advertising has had a fair amount of safe-harbor carve-outs for over a 100 years or so that are not available to other industries.

What safe-harbor carve-out existed 30 years ago for advertising that another industry might have wanted something similar on?

Lying to consumers. Puffery is the legal term of art.

The "reasoning" allowing advertising/advertisers to lie is that it is assumed consumers do not believe it.

It is not fraud to distract from evidence that your product might not work. It is fraud to claim your product works if it doesn't.

Therefore ad companies are not going to try to prove that their product doesn't work.

Common law fraud is hard to prove in the US.

All of the elements listed below must be proven. However, some of the comments here describe behavior that comes pretty close.

(1) a representation of fact; (2) its falsity; (3) its materiality; (4) the representer’s knowledge of its falsity or ignorance of its truth; (5) the representer’s intent that it should be acted upon by the person in the manner reasonably contemplated; (6) the injured party’s ignorance of its falsity; (7) the injured party’s reliance on its truth; (8) the injured party’s right to rely thereon; and (9) the injured party’s consequent and proximate injury.

Please don't break the site guidelines by going on about downvotes. I know they suck, but bilious addenda about downvotes also suck, which is why users here are asked to restrain themselves from posting them.

https://news.ycombinator.com/newsguidelines.html

Apart from taking threads badly off topic, such complaints usually end up as uncollected garbage, since often the most unfairly downvoted comments end up getting corrective upvotes from users.

https://hn.algolia.com/?query=corrective%20upvote&dateRange=...

Does advertising create an increase in buying your product. That’s the most important question.

At least with ecommerce one can track users that have seen and/or clicked on ads with what they have purchased and return on ad spend can be calculated.

But yes, the question still is: does it raise intent or did the user already want the thing, googled it, and clicked on the first result (which is your ad, above the organic results that also lead to your online store)?

The fundamental problem with this is that if I bought your product, it's because I was looking for your product after doing a decent amount of research which meant I was on a bunch of retargeting lists and you've wasted money on me. I have never to my recollection purchased anything because of an ad. If I have clicked through an ad, it was simply because it was the quickest way to get to the website/product.

There is a role for advertising because Google polluted their organic search results with Ads making it harder for you to find what you want and forcing you to compete for the top Ad spot. I realize that I can only speak for myself, but I have never seen any evidence that advertising works and I've worked in ad tech.

It's pretty clear to see direct causation when using ads to promote your product. Promote your product using ads for 1 month, then turn off ads and see how many sales you get. For a majority of e-commerce, you'll see a substantial drop.

You've researched every item you buy in your weekly shop? You've never bought anything on a whim, or a gift for someone else?

If you are sinking $100M in, I sure hope it’s having some effect.

You're also bidding for eyeballs. An established player might want to scoop up the most relevant keywords forcing competitors to at least price match, effectively starving them either of views or cash

There's still value in getting people to buy now instead of maybe buying later.

The thing about the "advertising doesn't matter" argument is that there are situations where advertising absolutely matters - when a company is just starting out and there's no way they can get organic referrals, when the public is unaware of their existence, etc.

Moreover, most companies started small at some point so advertising and maintaining advertising made sense up to a point.

And if we look at those companies where maybe advertising actually doesn't help, companies that have reached a level of success where organic referrals and public awareness drive most of their business. And they're ongoing businesses that probably reached that level through advertising and have money now to use for that.

But not only are they already advertising but they don't if their word-of-mouth/public-knowledge presence will last indefinitely, they don't know if just word-of-mouth would let them control their image, would stand-up against future competitors ads and so-forth. So, even supposing you could show advertising didn't offer any immediate increase in customers, continuing to spend on it doesn't seem to me as irrational as it sounds.

This is the point that a lot of people are missing

Uber shows up in the news and in conversation every day. Most people have heard about it by now. Very few people are going to install it because they saw an ad.

Not the case for your neighborhood "Bob's Burgers" or "Uber Competitor" or some other new company. Or for specific marketing actions of known companies (promotions, new services, etc)

Furthermore, many of the people who say they hate advertising also hate PR and other sort of marketing campaigns. So presumably their theory is that people should build something and just hope people will discover them somehow.

It's a subset of the misunderstanding of "meritocracy" that defines it as "how well you can do at a narrowly-defined task" as opposed to the practical definition of "How many people know how well you do at a narrowly-defined task."

Our industry is shot-through with idealists who believe meritocracy should be the former, when in reality it's always been the latter (and always will be, unless someone can invent omniscient humans).

Well, and arguably, "How many people know how well you do at a set of tasks that are broadly relevant to them."

> Furthermore, many of the people who say they hate advertising also hate PR and other sort of marketing campaigns. So presumably their theory is that people should build something and just hope people will discover them somehow.

Could it be that people simply don't like being interrupted and having their time wasted, whether it's by ads, marketing or PR? Just because you have a business to grow shouldn't give you the right to waste my time.

Also Uber would benefit from a higher price of advertising for upstart taxi companies.

That still doesn't address the GP point, advertising doesn't matter if you can get customers coming to you organically. Business size changes the odds of that, irrespective of that if the customers would have come to you anyway (and there are ways to know this!) then the ad spending was largely a waste.

You misunderstand what advertisers are trying to achieve. It's about Brand Awareness. They just want their brand out there and seen. Even click-throughs aren't as important as they once were. Coca-Cola still advertise when they probably don't need to.

I understand the distinction between advertising and building brand awareness. It doesn't sound like Uber was doing the latter. It also doesn't take away that if you're advertising to an audience who'd have come to you anyway, you're wasting your money.

Brand awareness must also be maintained, just like human relationships! How often do you tell a loved-one that you love them? Is once enough, because going by your stance, since you've told them already once before you shouldn't need to do it again!

I don't think there's a large "advertising doesn't matter" camp. For most the real question is around the efficacy of certain types of ad buys, ad targeting, SEO efficacy and the amount of fraud in the system.

The real problem is that for large companies with brand awareness they're starting to find a lot of their spend may not be generating revenue. Ebay in the Freakonomics podcast turned of about $100M in ad spend with no loss of revenue. But smaller, companies usually don't have the staff/knowledge necessary to know how and where to use their ad dollars.

>when a company is just starting out and there's no way they can get organic referrals, when the public is unaware of their existence, etc.

I don't think I ever saw a Whatsapp or Telegram advertisement in the first few years of their existence. There's a lot of software in particular that gets around by word of mouth alone.

Necessity for advertisement seems correlated to how unclear it is what advantage your product has for the consumer. If you make something that's like X but twice as fast or twice as cheap you don't need advertisement.

Ad spend by incumbents will also drive up the price of advertising in general since it's a limited resource, making an uphill battle for the smaller competitors where it matters. It's another moat.

Depends on the timeline. If for financial (or other) reasons, they need to "reach the audience ASAP", then sure, advertising is likely to be a good or even required move.

But if they want to exploit a more organic growth curve, and are in a financial position to do so, then advertising isn't so useful.

Ardour is a cross-platform DAW that has been around for 21 years. Every 2 minutes somewhere in the world, someone starts a version they got from us. We've never advertised.

Maybe the person is referencing how difficult it is to determine the value of the counterfactual? If it were possible to determine I imagine people would really want to know.

I think the implication here is "that honestly doesn't seem that important" because the expectation is that the number of people who would find you anyway would be a small number compared to a well-run ad campaign, at least over timeframes startups care about.

Having worked with a decent number of small teams without a lot of brand recognition, ads bring in a lot of customers: it was pretty obvious in our metrics when we ran ad campaigns vs when we didn't. Sure, eventually a few of those people may have found their way to us somehow anyway, but there was a step change in volume from ads. At least in my experience, the idea that ads have no significant impact is not really supported by the evidence. I assume that's what the OP meant by it not seeming important: they're working under the assumption that "do any ads do anything" is clear and you don't need to prove the worth there — instead you need to prove which ads work better than others, and cull the ones that don't perform well (which will also help save you from scammers).

One could argue that advertising for large brands with name recognition could be less valuable. I don't really have data or experience to know that one way or another, but in a competitive market that seems a little hard to believe. Tesla is an interesting counterexample, but I think it may be in part due to the EV market in the US not being particularly competitive; no one makes attractive, mid-priced cars with long range other than Tesla, let alone having an established charging network for road trips. And Elon's antics are, tbh, a form of marketing in and of themselves; "all press is good press."

Edit: I wrote the above paragraph badly. I can definitely believe ads are less valuable if you're a big brand vs a small one. But I'd be surprised if they're useless, at least in a competitive market. People don't have infinite money, so if they're looking to buy X and an ad for X from your competitor pops up, it seems reasonable to me that some percentage of the time your competitor would make the sale and thus you wouldn't, even if organically that sale might've gone to your company instead.

Also, I wouldn't argue that even small companies need ads — just that they do seem to be effective at increasing the volume of people interested in using your product at a given time. That may or may not be necessary/useful depending on your situation. I have been on teams where we intentionally turned off ad campaigns because we learned what we needed to learn from the cohort of new users, and now wanted to improve the product based on their feedback before spending money on more ads.

>One could argue that advertising for large brands with name recognition could be less valuable.

It is without a doubt, and this is what the Freakonomics advertising episodes are mostly about.

Yeah, I wrote that badly. It's easy for me to believe they're less valuable. I really meant "One could argue that they're useless, but that seems hard to believe in a competitive market."

Did you actually read the linked tweets?!

Uber turns off a shitload of ad spending, nothing bad happens to new user acquisitions.

I'd say there are about a million better uses of a million dollars than just pissing them away on a scam.

There’s an interesting question there about when this form of advertising becomes obsolete.

Uber is in a very different position than many other companies. Anyone who browses the internet with regularity is already aware of their existence and probably just needs the right set of circumstances to come together to make Uber useful to them.

I suspect the results would be different if Uber were earlier in their adoption curve, but maybe that’s not true either. Maybe they’d be ignored for different reasons at that time.

Likewise, I probably haven’t seen an ad for coca cola or kleenex in a while. Once a brand is ubiquitous to the point where soda becomes coke and tissues become kleenex in the lexicon, it feels like ad spend is wasted.

Brand advertising is different. It is there to remind people about the brand. Companies do it because they have the analytics that show that it works.

They mostly do it from inertia, how well it works is a separate question. Historically you can find many examples of hugely successful advertising campaigns, it’s much harder to quantify the negative.

I think both Uber and CocaCola has a lot of work to do until I like any of them. I like CocaCola as a product though. Never tried Uber. For example I really like some Nestle products, but try to avoid them because of their bad company/brand reputation, even if their product is both cheaper and tastier I buy from a competitor instead. Whatever a company stands for and how they act will affect their brand.

Coca Cola spends billions on ads.

8 million views: https://www.youtube.com/watch?v=yg4Mq5EAEzw

>8 million views

So, less than a viral cat video?

Is it really christmas time, if there are no cocacola-trucks-driving-to-town ads on tv?

Paying for crappy app installs on shady sites is not really advertising.

What they should have done was a proper hyperlocal SEO campaign like Firestone Gieco, and Mc Donalds do.

There's a huge difference between a company that is widely known for being the app for ride hailing and a small company trying to get eyes on their new product.

Well, Uber is known to everybody by now. To the effect that "to Uber" is even kind of a verb/noun.

Also people either need a hired transportation or not. If they don't have a car or don't want to mess with the traffic and need to go somewhere, it's either Uber or Taxi usually.

It's not like Coca Cola, which is well known, but people could do without it (unless addicted), so needs to constantly nag people.

And it's not like some new product, which without advertising nobody would even know it existed.

In fact most of Uber's existance its operation has been 100% advertising (spending VS money to offer cheap rides and expand and gather "eyeballs" and "customers" without a profit). In my book, customer acquisition without profit is another name for advertising.

So it doesn't sound strange that it could do without advertising today.

But what if there were 3-4 strong players in the same, each eating in Uber's market share? You'll see how fast they'd found advertising indispensable again...

Literally lighting them on fire in a cold office to help provide some BTUs would’ve been a better use of that money, since at least that would offset spending somewhere else.

> All big companies do this.

Except Uber, apparently? Or would the method you're talking about not have discovered that something fishy was going on? (I don't work with ads so I don't know the limitations of the type of experiment that you're talking about)

It would work for some of Uber's ads. From the article, it looks like they were frauded mostly by in-app ads. And they were paying for installs - not actual trips.

So, no, Uber's advertising here is a little different than (I think) the majority of companies. They are mostly paying for installs rather than sales / conversions. A lot of newer "app" companies could be in similar situations.

Though, honestly, this seems like a massive fail on their analytics team for not figuring this out earlier. They should have been able to see that all of these "installs" from certain advertisers were not leading to trips.

In fact, it says they turned off 66% of ads. They didn't randomly turn of 66% of ALL ads. They turned off this TYPE of ad, which they failed to earlier recognize was ineffective.

Step 1) assume your ads won't work.

Step 2) have enough analytics / logging in place to convince yourself the ads do work.

Step 3) if they don't work, turn them off.

Looks like they skipped step 2 - which honestly, is not uncommon for a fast growing business - even if they are huge and already make a lot of money.

What they found isn't even what people are discussing. They found that certain networks they were buying ads from were almost 100% fraud (which is pretty well known).

Instead, people here seem to be discussing that most online advertising is fraud, and/or that there's no way to prove it's effective. That is absurd.

Hang on though, in the linked thread they said that the fraud was incurred at the point of real users signing up for uber -- when they type uber into the app store search (e.g. an organic install), the ad network fraudulently takes credit for it at that point. So "enough analytics/logging" would not do the trick here - I think they would have noticed if a certain type of ad wasn't leading to trips, as you are pointing at.

> You can A/B test ads pretty easily

A/B testing ads is a complex matter. you can A/B traffic and conversion easily, but a lot of established companies with fierce competition fight for mind share, not direct conversion; for that, you have both awareness effects (user won't forget about coca cola if they don't run ads for a month, and an ad that doesn't directly convert but increase awareness still has value) and coverage synergies (the number of repetitions in a day will increase coverage non linearly and the amount of channel repetitions will increase awareness more than a single channel view, even if it doesn't convert immediately)

> Sure, you're never going to know exactly how many people you advertised to would have organically, eventually found your product and bought it.

This is typically an order of magnitude less than the attribution figures stated by digital marketing experts. A/B test is the right method to use, but the crucial thing is you need an earmarked population to see zero adds over your attribution window, since what you care about is impact on incremental sales, rather than incremental click likelihood.

There is a long econometrics literature on this and it is not a fussy technicality, the figures typically differ by 10x +.

Instagram definitely does this. I'm part of the magic cohort who never see any ads :)

It can happen that an almost certainly bogus scientific field persists for decades using state of the art research methods and no bad faith on the part of the researchers (any bias they introduce is probably not conscious). https://slatestarcodex.com/2014/04/28/the-control-group-is-o...

A/B testing is... not a state of the art scientific research technique. Moreover the companies that provide the tools to do A/B testing are the same companies that sell you access to advertising space. I'm not saying that it's a common practice to defraud A/B tests, I'm saying that the fact that adtech companies have chosen to enable that research methodology out of the set of all methodologies they could offer suggests that we should expect, before seeing the results of any A/B trial, that the results will tend to favor the adtech narrative.

I don't think that people in the adtech space believe they're selling a bogus product, but I do think they wouldn't want to know if they were — they have a good thing going.

If you're employed in adtech you're mostly fine, skills transfer. If you're invested in adtech, do what you can to diversify away. Advertising is overvalued to some extent and that bubble will burst at some point or other. The question is just how much actual value advertising provides, how much will remain when the bubble bursts.

>but you can also measure against SEO traffic with a high degree of certainty.

None of these benchmarks distinguish between the selection effect (clicks, purchases and downloads that are happening anyway) and the advertising effect (clicks, purchases and downloads that would not have happened without ads).

You can fix this by dividing the target group into two random cohorts in advance: one group sees the ad, the other does not. Designing the experiment thus excludes the effects of selection.

When you do this experiment correctly, you find out that ads have low effect or are not cost effective (as eBay discovered).

So how much ad spending is going to 'dubious ad networks'?

Are they easy to categorize? Or is it a big secret what's dubious and what isn't?

This is all discussed in the podcast.

There’s no revenue change even after testing (when you tack on costs of ad delivery).

"You can A/B test ads pretty easily, and this is quite common. With some degree of statistical certainty, you can tell how one ad performs to another."

Nope.

You need a unique id for an AB test AND apple wont give this to you. No company is going to give you a click/tap id. Android might. This is so inaccurate I don't know where to start. Apple has turned off all unique ids that work across an AB test [1] cross platform. You would need on the ground mobile ad experience to know this.

https://www.invoca.com/blog/what-is-idfa-and-why-apple-kille...

For native Android you 100% have access to an Android "device id". I set this up on an Android app we delivered. There's even companies that pay other Android apps to host their code inside the app, co-related that ID to other data available to the web (e.g. IP address, installed fonts, etc) to provide a very good guess of a user's device id via the web.

Apple had something similar, but you're right they're starting to turn it off. In my past life in Adtech we noticed the click through rate of users with Android user agents were a lot higher on certain campaigns than iOS users for this reason.

Across this and the other[0] highly active digital ads post, there's 3 interesting forces at play:

a) technologists screaming "ads are literally the worst societal cost, like ever" who don't want to understand the industry

b) advertising folks taking up arms to defend their shamanistic, money-printing machines without statistics (Reason No. 7 will SHOCK you)

c) A tiny, tiny group of Mandalorian-like voices who have a necessary statistics AND industry understanding who are being drowned out

(this is in jest, but I double-dare you to say it isn't at least directionally accurate)

Other commenters have mentioned it throughout this whole comment thread as "incrementality tests". Amongst other approaches, this is the way.

Freakonomics severely lack the industry understanding. Listening to the podcast was like hearing how HTML is a programming language from the kid in week 2 at code camp. Then too the article, all the issues with Uber was just doing a bad job of managing their ad spend and they can fall into group B, noted above.

The baseline of this work is a control group who see no ads and then you build your tests from there that factor in channels, cohorting, and other components to get a statistically significant outcome. Yes, this will get more challenging with upcoming privacy changes (IDFA removal, et al). However, the last 10 years this wasn't a problem and I'm sure the corporations in the identity resolution business will hand-shake on a bunch of 2nd party data deals that just move the deals done in broad daylight around identity tracking and audience creation to the alley. Further, any advertising that is tied to an already known customer is able to be backed into at an audience level with login and cookie data. Even Pi-hole users may not be exempt here.

To finish with some constructive advice:

1) Advertising is not a synonym for marketing. We're only talking about advertising in both HN threads.

2) Every industry has high and low quality. Pareto's principle should be aggressively applied to where one spends their budget in the cesspool that is the Internet.

3) If you're ever seeking an agency to provide advertising services and they don't have a qualified data science or statistics leader (10+ years work experience, degree in Stats/Math/Econ, an MBA, or similar), run. Run from those shiny-shoes gurus. Channel your inner Usain Bolt and run.

0. https://news.ycombinator.com/item?id=25620707

You can definitely measure 'lift' - though it's far easier with online only sales.

For sure it's harder for say Gucci to measure incremental cross-platform lift but any good large brand spends big money trying to parse it out

One example way to get some value measurement would be Facebook's powerful tools. Beyond a basic 1:1 audience 50/50 exposure test, Facebook also lets you upload offline sales (or use their pretty effective pixel for real time online sales) and then feed into their system. Using the data your purchase data FB matches to ads delivered and you (with PII but also purchase value and frequency). Then the advertiser can define lookback/conversion definitions to get a measure of incremental lift. E.g. those who clicked ad bought __ right away, those who viewed ad within 1, 7 or 28 days bought __.

For what it's worth the adtech forums I participate on view Uber as incompetent idiots on this issue (which has been reported on like last year) - all they had to do was measure beyond the initial install for say did these installs actually pay to ride...

Install fraud from SSPs and resellers is huge, doing this basic measurement and quality control is the reason Facebook and Apple have huge and growing app install business. Though TBD on Apple fucking over Facebook and keeping the attribution for themselves alone with their new privacy features

Back in the early 2010s I worked in ad tech on a data science team, and one of the things we were pushing for was causal A/B testing; basically turn off a campaign's advertising to a % of people and correlate it with sales to measure ROI.

As we were kicking this off I was at a conference chatting with an executive at another ad tech company. His response: "oh yeah I know a guy who tried that, he's not in the industry anymore."

We almost immediately came to realize our launch clients were getting negative ROI, sometimes severely so. AFAIK our efforts fizzled out, and I believe none of the people on my team are in the industry anymore.

I might be misunderstanding... but, negarive ROI? I.e, the more they spent on ads, the less sales they got?

Feels like a missed something, as that sounds... counterintuivie.

(Not in any way related to the ad industry so pardon my ignorance)

As I read it, the amount they were spending on the ads themselves was more than the converted revenue from ad clicks. While the decline was probably (guessing) not linear, spending more on ads led to less than proportionally more revenue. If that had happened I can imagine calling that negative ROI.

That makes more sense!

Thanks

No. A negative ROI just implies that the ratio of benefit/cost is less than one. If I incur a cost/investment of $100 but it creates value of $200 then I have an ROI of 100%. If this same expenditure instead only produced $80 value then I would have -20% ROI as in the value I’m realizing from my investment is 20% less than the cost of the investment.

Ratio can't be negative, they can either be above of below 1. What you are talking about is "$benefit - $cost", not "$benefit / cost".

ROI is (net benefit)/(cost), where net benefit is (gross benefit - cost). Net benefit can be negative, making ROI negative if cost exceeds benefit. GP was using benefit as a shorthand for net benefit, which can certainly be negative.

Even gross benefit could conceivably be negative, if the ads are bad enough! (I.e., a terrible ad might prevent a sale.)

We tend to call most rations below 1 'negative'. Its technically wrong but most people get what you mean when you say it.

Positive and negative are the right words on a log scale :D

Ratio can't be negative, but ROI can. Positive ROI is the percentage of the ratio being 1 or higher. Negative ROI is the percentage of the ratio being less than 1.

Or, from a "common sense" perspective, if I spent $1000 on advertising, and earned $100 in revenue from that investment, my ROI is definitely negative.

I would expect:

  ROI > 1 = profitable ad spend
  0 < ROI < 1 = ad spend does not cover its cost
  ROI < 0 = losing money because of the ad spend (the ad is damaging to your brand)

What do you mean by the two references to not being in the ad industry anymore? Quit? Fired? Why?

Did they leave the industry because they realized their product wasn't providing a positive ROI, or were they fired for pointing that out to other people? I think the former is probably what you mean, I just want to be sure.

Good question. In the first case it was left vague but I understood it as being fired and then not trying to find a new ad tech job. In the case of my team it was finding better industries to work in.

Yeah, it could be that many startups try to do advertising ethically, realize that it doesn't work, and drop out. That's necessarily going to leave all the firms willing to sell snake oil behind.

You can get firms selling stuff that doesn't work that are "trusted" simply because they've been around for years; there are plenty of distinguished brands selling homeopathic remedies, audiophile speaker cables, timeshares and MLM schemes.

Similar time frame for me, worked at an ad-tech startup, when retargeting was first becoming a big thing and we were pivoting the startup from the bad business idea it started to to doing retargeting as a demand"-side-platform" (DSP) on ad various exchanges. Tried myself to confirm whether any of it had any positive effect, I couldn't really discern any increase in the click-through-rate (CTR) for various approaches, but I'm not a stats expert, etc. so talked to the founder about how we should employ someone with a stats background. That convo went nowhere, and for that and other reasons I was out the door within a couple months.

That was the era of the ad-exchange DSP bubble. After that I went to work at another company that was on the other side of the exchange pipeline and I could see all these DSPs just plugging away doing their thing and none of it looked (to me) like it was accomplishing much. It was all bottomfeeding off of lower quality inventory but I suspect making big promises to investors.

That startup eventually pivoted a couple more times and sold to a bigger player a few years later, making some money for the founder but I suspect no value to the buyer.

Reading about startups that pivot dramatically multiple times make me cringe so hard.

The goal of pivoting isn’t throwing spaghetti at the wall until you hit something before running out of VC or angel money. That just shows terrible product/marketing leadership.

Your prior story is unsurprising combined with that.

As a person with just shy of a stats graduate degree, the early 2010's were a nightmare of trying to get ad tech companies to grasp their incompetency. I met a dizzying number of slick talking frauds and became quite jaded.

Ok good, it wasn't just me being overly negative (because I usually am). I am not a stats grad, my math is weak, but it didn't look good to me, and I tried to get my employer to bring a stats specialist in to analyze. That got a cold response.

>Stopped [some] Spending on Digital Ads, Nothing Happened. Why?

[Added "some" for clarity.]

Let me try to restate the author's article because he presents it in a confusing way.

The issue is that both Google and Facebook have "core ad tech" that works decently with proven ROI -- and they both have "additional ad tech inventory" (a.k.a the partners/affiliates) that's much lower quality:

Facebook "quality" ad placements on their core platforms with decent ROI:

- ads in Facebook Newsfeed

- ads in Instagram feed

The "questionable" ad placements with much lower (possibly zero) ROI:

- ads in Facebook Audience Network

(Here's an example screenshot in Facebook's Ads Manager to visualize the options above: https://storage.googleapis.com/website-production/uploads/20...)

Same concept applies to Google AdWords. The adwords clicks on "google.com" perform better than the ones coming from partner websites. The lower quality ad tech in partner networks has more bots, more scam websites, more fraud, more negatives, etc that reduce ROI.

Bottom line is... if you're buying digital ads, you need to understand exactly what type of clicks you're paying for and how it actually performs.

great summary

Source?

This all true. Source: My 11 years running ads on these platforms.

Can you please elaborate? Your source is just an appeal to authority, but I would really appreciate more evidence to help explain your position.

It is true. Partner sites make revenue for the partner through revshare and partners attempt to create fake clicks and even conversions to inflate their revenue. Cobra effect in play.

This risk is zero (except for things like ad neuseam) on first party / owned and operated placements like search/ig/fb.

As with anything you should verify your roi through third party measurement providers, just like you would do for others.

Disclaimer: current fb ads engineer, former google ads engineer.

By that logic any lottery winner could argue that lotteries are a great pension plan.

If they consistently win lotteries year after year with a good ROI, sure. Not sure how that comparison works.

Ever heard of survivor bias? Let me make an example: I choose 2048 random addresses. I choose I random company on the stock market. Half of the addresses I send a email stating the stock will rise in the next month, the other half the stock will fall. After a month I check the stock and disgard the addresses where my prediction was wrong. Than I repeat the previuos steps until only one address is left. Now ask him about my expertise on the stock market.

Really? Downvoted because I wanted to know the source for the proven ROI?

i agreed with u

That's more or less accepted. Ads that appear in Google search results have some value, since they appear when someone is looking for something specific. That's measurable.

Ads that appear on irrelevant web pages have less value. That's well known. Apparently so little value that it's near zero for known brands, this article says. In the end, they're like useless banner ads. Mostly clicked on by bots.

It's not about clicks or traffic. The real issue is selection effect vs. advertising effect

SEO traffic benchmarks don't measure the right ting or show real ROI. FB or Google don't want you to measure the right thing.

Traffic to the website has licks, purchases and downloads that are happening anyway (selection effect) and clicks, purchases and downloads that would not have happened without ads (advertising effect) mixed. Only way to differentiate between the two is do real experiment where you divide target groups in two sections and show only the other group the ads.

When you do this experiment correctly, you find out that online ads are usually wasted money for any established brand and lower than expected value for less known brands.

These are pretty easy to measure on Google or FB with their "lift study" tools and a conversion pixel. The lift tools will let you create a holdout group of X% of people who should be targeted by your ad. They won't see the ad, but you can still measure the rate at which they convert on your site. Then, you can compare that conversion rate to the rate of people who actually saw/clicked the ad.

Conversion pixels are never 100% accurate (and are becoming increasingly inaccurate due to browser/OS changes) and shouldn't be used to say "This ad generated exactly X sales that wouldn't have happened otherwise!", but you can examine the relative rates from people who saw or didn't see your campaign.

I've also seen companies who don't trust FB/G to grade their own homework on a known lift study, so they measure the effect by running PSA ads. Their ad campaign will be something like "Donate to charity XYZ!" but will be set to measure purchases on their website. That lets them establish the baseline purchase rate of the customers in their audience.

> They won't see the ad, but you can still measure the rate at which they convert on your site.

This sounds like such an assumption. There's no guarantee that any one person will ever "convert" on your site. That's such an ugly carrot that FB/G are dangling in front of you. "If you spend enough money with us, we'll eventually show your ad to someone that will absolutely buy your product. Pinky swear, just keep putting coins in the slot to continue".

Let me explain it a little differently with a hypothetical scenario.

You set up an ad campaign with an audience of 100k people who are already on your mailing list.

You set up a conversion pixel and run the ads. Let's say that Google's tools say that 10% of people who saw the ad converted. That's incredible! But considering that they were already on your mailing list, maybe they were going to buy anyways - or maybe your TV ad is actually what convinced them to buy.

You test for this by creating a holdout group from that 100k. Let's say 5k people. These are people that could have seen your ad, but we're intentionally not going to show it to them. We'll still use the conversion pixel's measurement functionality on them, and we see that 8% of those 5k people buy your product.

Now, we have a clearer picture - the Google ads drive 2% more conversions for your audience - or people on your mailing list who see an ad on Google are 25% more likely to convert than those who don't.

Nobody can predict the future, but it's a lot better to inform your predictions based on historic performance than on no data at all.

>These are pretty easy to measure on Google or FB with their "lift study" tools and a conversion pixel.

I hate that Facebook and Google have normalized this invasion of privacy.

https://en.wikipedia.org/wiki/Conversion_tracking

This was normalized on the web long before these companies started - they're just the best to do it (though Google did acquire one of the earliest players in DoubleClick).

The term "conversion pixel" is a misnomer these days for a Javascript snippet or API call - the concept predates JS and used to refer to a 1x1 transparent GIF.

Anyways, that generation of tech is being pushed out due to browser privacy enhancements and will be replaced with privacy-protecting browser and server APIs that introduce noise to the data, such that you can still measure the directional/relative impact of advertising without being able to tell exactly who converted.

There are a few competing industry proposals involving differential privacy/cryptography/blockchain/fixed entropy that are being publicly discussed here: https://github.com/w3c/web-advertising

Just because the ad placement is low quality (eg audience network) doesn’t mean they can’t produce a positive ROI. Keep in mind, Lower quality comes with a lower price, so it makes sense for some brands, and some products. The reason people dont get a positive return on low quality ad placement is that they often use the same ad, or adverse the same product as for the higher quality ad placement. These ads and product often needs to be tailored for that specific placement.

Text-based search ads - are completely different from - network banners ads.

They are used for different reasons.

100% of ad buyers, marketing people know this, it's not an insight.

Nobody is 'accidentally spending money' on banner ads, when they thought they'd be spending money on Text ads.

Also - it's very straight forward to apply different measurements to FB ads in and out of network.

i usually get ads for things that i just bought. That means they are very good at snooping, on the other hand if I just bought a TV set then that doesn't mean that i would need another one soon.

Funny that Goole/Facebook currently don't seem to be running ads for complements of products too often; Amazon is better at that (if you bought this book, then you might also want this one).

It seems like we've been talking about this so long that a complementary product feature would be standard everywhere even if it didn't work.

There’s a useful correlary to this for consumer tech.

While it is tempting to monetize with a publisher network, these networks don’t work well enough to justify their own spend for many advertisers. As a publisher you’ll likely be stuck in a race to the bottom until you have sufficient scale for direct ad deals (read as 10s of million MAU).

I’m not sure if kicking fraud out of these platforms would raise ROI, or reveal that the real price of banner ads and interstitials is 0.

I think a lot of this was already obvious to slightly-savvy mobile app users. Look at pretty much any free-with-ads app; a majority of the ads are going to be ads for other free-with-ads apps, many of which you already have installed. The thing is, outside of the Internet most ad spend is for marketing ubiquity, not direct response. You don't really buy, say, TV advertising with the expectation of getting so many clicks or calls out of it. And, up until a few years ago, TV was the lion's share of ad spend. So who knows if we'll see an actual industry reckoning or not. The whole point of advertising is to waste money, after all, and plenty of brands were fine with getting nothing but exposure out of it.

More interesting to me is the fact that tech companies are finding it surprisingly difficult to control where their ad-spend goes. I suppose this is an inverse of the problems with supply chains, where Apple can take three years to get a connector vendor out of their supply chain even when they were using literal child/slave labor. It's a market for lemons; bad "money" (publishers, suppliers) drives out good. The question is: will questionable ad publishing actually harm corporate reputation to the point where big ad spenders go away, or will we just see periodic Adpocalpse-style waves of spending being decreased and then brought back?

The point of traditional advertising IS pretty much to waste money, similar to how the point of a peacock's tail feathers are to waste resources... it is a signal to potential partners (business or romantic) that they are so capable they can waste resources. It is a signal of bonafides.

It's also an important signal to consumers.

That the products are overpriced?

That you're paying for these ads.

Yes, that’s the point.

No, that the product is good enough that it makes enough money to afford expensive advertising.

There is a really good essay about this that I re-read quite often: https://zgp.org/targeted-advertising-considered-harmful/

I can't believe I had to scroll down this far for someone to point this out.

It only says that the margin on the sold products are high enough so they can afford expensive marketing. Does anyone seriously believe that the massive marketing for a product like Coca Cola is a sign that it is a high quality product?

Yeah, that was who I meant by 'business partners'

The problem is that it is no longer true with online ads. The barrier to entry in online ads is non-existent and thus seeing an ad for a product should not be considered a trust signal (if anything, it should be the opposite) unlike on TV or print where the price of such ads works as an effective filter.

I'm genuinely shocked at how many people think advertising doesn't work.

Just because it doesn't make a person consciously stand up and say, "I want a Snickers" does not mean the ad didn't work.

Like, honestly, are there people here suggesting viral ads don't work? Or don't understand the point of Coke or Mercedes Benz ads are for? Coke establishes themselves as the defacto soda. There is no other soda, or if they are, wish they were coke.

MB ads among other things reinforce to MB owners the wisdom and luxury of their brand. To own an MB is to be part of a club, one that is advertised across the media spectrum. If MB didn't advertise at all, they would either become another commodity brand, or have to be so luxury that only word of mouth is necessary (Bently, etc).

Now, maybe that's what cortesoft is saying "get you to waste money" but if it got you to waste money on their product, it worked.

Sorry, I think you misunderstood my point a bit... I think I am agreeing with you.

My point was that advertising's purpose (from a consumer point of view) is for the ADVERTISER to waste money, to signal that they have confidence that their product is good enough to recover the cost of wasting money on advertising.

They weren't disagreeing with you in their reply, just emphasizing the same point.

I understand the point but the MB example could be off the mark. According to their site [1] they sold more than 2 M cars in 2019. Luxury yes, at least a bit, but a pretty large club. Maybe Ferrari and similar brands? But they do a different kind of advertising.

[1] https://www.daimler.com/investors/key-figures/markets/

I don't think they meant that it's a private club.

Personally, I think it's less about luxury and more about value signalling. Having a Mercedes says that you're comfortable financially, and they're "dignified". I'm not saying it's true, but it's what their marketing generally implies. Luxury is generally just a dog whistle for value signalling merchandise.

Ferrari does different value signalling. They imply wealth because of the cars performance, and they're generally a much more obvious signal. They're usually a bold color like red in the ads, and the body styles are distinctive. You could miss a Mercedes in a group of sedans; you're not going to miss the Ferrari.

The private club brand, to me, is Rolls Royce. I would guess because I never see ads (and rarely see them in person), so my only references are pop culture where they're associated with the ultra wealthy.

I'm not entirely sure about that, at least not in all cases.

I was recently in the market for a new mattress, so went to a mattress store and tried some out. Even though Tempur Pedic is all the rage with their insane ad spending, I was not all that impressed, and found a considerably cheaper mattress that felt much higher quality.

Now, this may all be sales mumbo-jumbo, but the sales rep noted that a lot of people come there only interested in Tempur Pedic mattresses, which he regards as "literally a piece of foam, total crap mattresses." They sell at such a high price point because they advertise so much, while the mattress I found came from a no-name brand that doesn't advertise at all, and was therefore cheaper.

So I guess in some markets, advertising does work. I've never heard of someone telling me how much they loved a mattress unless it's a Sleep Number or a Tempur Pedic, and those advertise by far the most.

I guess that's the thing; for a lot of industries, people aren't generally going to make a purchase without doing some amount of research. Maybe Tempur-Pedic was on their mind when they entered the store, but once they're in the store they're generally going to end up trying out several mattresses. I /love/ my mattress (and have said as much to friends), but I honestly couldn't tell you the brand off the top of my head; I just went to a store and picked whatever felt the best.

One could argue that Tempur-Pedic has to sell at a higher price point to make up for the money they spend on advertising.

This was true in the past, but nowdays it’s much easier to try out a product and see reviews.

With Uber it would be so easy to penalize drivers who cancel because they don’t like where I’m going, but I’m still wasting 30 minutes quite often, because Uber spends that money on ads and UI rewrites instead of improving the core experience.

I suppose this is true in the same sense that the purpose of life is to waste oxygen.

this confirms the previous post if you subscribe to the idea that the purpose of advertising is to waste oxygen

I work at a fortune 50 company, a top 10 advertizer. We have a universal holdout poplation that NEVER recieves any directed/targeted marketing that serves as a control group for many marketing programs. Further in more mass areas there are direct A/B tests that go on. There is a very strong awareness of campaigns that are measurable with ROI's and other programs that are general branding that you can see the impacts in gneral customer awareness surveys', favorability ratings etc.

Measuring as spend ROI is standard practice on any large scale ad campaign. The trick about having a holdout population is a neat twist.

It boggles my mind when HN commenters start claiming that measuring ad campaigns is impossible or that ads are universally ineffective. Anyone who has spent time working with large ad campaigns should know the tools and methods used to measure these things.

I suspect the HN sentiment comes from a common feeling among techies that they are somehow immune to influence from advertising, combined with a high adoption rate of ad blockers. HN commenters tend to assume that other consumers are just like themselves, which is far from the truth. In the real world, advertising (when done right) is not only very effective but not that difficult to measure using modern technology.

> I suspect the HN sentiment comes from a common feeling among techies that they are somehow immune to influence from advertising, combined with a high adoption rate of ad blockers.

I don't believe this at all. The reason I am skeptical of the effectiveness of advertising is despite engaging with dozens of people over the years, I have never had a single person provide me with convincing evidence that it works. Instead, they simply repeat "but of course it works!" similar to your comment.

So, where is the evidence?

Google's yearly revenue

I'm asking for evidence that advertising is an effective use of money. All you're saying is that people spend money.

By your logic, Scientology and Gwyneth Paltrow are also selling the real deal.

I manage advertising spend for a small technology company. We spend in the 10's of thousands per month on Google and Facebook.

We can measure customer acquisition cost and we do using normal tools like Google Analytics.

We've had days where we pause our campaigns, and we get fewer new customers that day. We've had platforms pause for a period of time because of some issue, and we get fewer new customers over that period of time.

It's one data point. I'm aware of plenty of other companies with similar results to what we see.

Thank you. Even if it's only one data point, I appreciate it.

Every campaign has TWO holdouts. First, the universal holdout who gets nothing from any campaign, and a second additional holdout who doesn't get a specific campaign but gets other campaigns. We have an entire group of people designing the holdout matrix to make sure they are sound and don't cross over. That's the elementry school version of things.

The graduate level stuff we run is using decioning engines with adaptive models down to EACH email, switching up not just offers but also ad copy, images, time of delivery etc. that drive VERY complex holdout matrix's which can limit the number of things we can test at once.

One of the executive decisions is when or if the drop customers out of holdout populations. Usually keep the holdout going for X period of time as a standard practice, but if a campaign is showing very strong response we might drop half the holdout if we can still make a read on campaign. We have a formal notice of decision that is published when this happens. Holdouts have alot of opportunity cost, but finance doesn't see those numbers.

Some of the digital exhaust from our primrary product can be used to solve marketing attribution issues for major brands in an area that has until now lacked attribution. Our product beta is just hitting the market.

Your Fortune 50 company owns a heap of brands (at least 16+ active) that span multiple demographics, geographies and maybe more importantly self-identities.

Does the "universal holdout population" span across all the brands, or do you have a "universal holdout population" for each brand?

There is the core product, accessories, repair parts, digital services. Each area has it's own holdout because the sales process and channel are different. Channel partners also run their own advertizing which gets looked at.

For example: The digital services are an ingredient brand and can only really be sold to core product owners so they'll have a differnt holdout as their target population is different. Also that area has a short measurable path and thus different metrics and measurements.

There are some B2B areas that may not have holdouts, they functionally don't do any targeted marketing.

This a flawed way to measure ad spend and roi.

eBay tried what you did and noticed that revenue increase didn’t warrant the ad spending costs required to do so.

This is pretty interesting study at Ebay.

https://www.nber.org/system/files/working_papers/w20171/w201...

They tested the counterintuitive claims by performance marketers that 'brand' keywords (containing Ebay) had the highest ROI. They did an experiment and found that, contrary to attribution model, these Ebay keywords resulted in ZERO incremental sales, although they were clicked on by many people that purchased.

This resulted in cut of Ebay marketing budged by $100 mn.

There is now a big literature in economics looking at experiments and natural experiments, generally finding much smaller sales impact of advertising than claimed by industry participants, and genearlly -ive ROI.

https://www.nber.org/system/files/working_papers/w20171/w201...

This Freakonomics episode is a nice overview.

https://freakonomics.com/podcast/advertising-part-2/

Obviously 'brand keywords' has the highest apparent ROI. It is users already looking for your site that click those. It is probably some internal bonus scam that fuel such nonsense - a industry wide marketing department conspiracy.

Yeah, I remember this one. Worked in top 10 brand performance marketing team as analyst/data scientist and brought this up to our marketing director.

I have never been shot down so quickly in my career. Very toxic environment to work in.

I have this suspicion that mobile ads have way weaker effect than claimed. No-one in adtech has any interest in finding out. Marketers don't want to lose budgets or importance either. So a lot of money is wasted on annoying people. How sad.

I'm certain they do want to find out. Adtech still wants to know where paying users are even if the total is less than they'd like to claim.

I think the problem is that the overwhelming majority of digital ad formats just don’t work. Does anyone who works in digital ad tech genuinely believe that these formats work? I have been here for the last ten years and have come to believe they do not, certainly not comparable like traditional linear TV or print in their heydays.

There are exceptions. I think Google search ad works so well it’s eating the product. I think YouTube’s TrueView is great for modern brand advertisers (safety issues aside). But by and large, banner ads, forced video pre-rolls, etc don’t work. The negative message of the format offsets any message you try to convey.

Yes, fraud is a part of this. But I think a lot of this evidence looks the same as if none of their targeted, performance-based advertising was doing anything at all, beside really weak branding.

Genuinely interested to hear from people who disagree.

> overwhelming majority of digital ad formats just don’t work

Doesn't it? It may have miserable conversion rate, limited target audience but it apparently is commercially viable. People would hardly spend money on sending e-mail spam in 2020 if it didn't work.

I thought that at first. But like the original post points out, Uber was spending a lot of money, not just in absolute terms but in percentage terms, and it wasn't working for them.

There's an allure to making something measurable and setting goals against it. Often, those goals take on a life of their own. That's what I am suggesting happened here, I guess.

if you read the twitter thread we see that uber was spending 10x more money than they should have on ads simply because they weren't auditing where their money was going towards.

so there's evidence that people (incl uber, a market leader) do indeed spend money on ads even when it does nothing

I got 0.66x, as in $100M out of $150M. I likely missed something. Where did you see 10x?

$100M was labelled "fraud" in this example, but there was more waste uncovered in the remaining $50M that didn't get that label.

Mobile games run on this. They have a target user acquisition cost and spend based on that. It seems to work for some very profitable companies. People shouldn’t try to over read into this. Businesses can’t run on irrationality for very long

Indeed. Mobile games. Perhaps the people who play mobile games probably would click an ad of another alike game or otherwise related content. They usually aren't mindful enough to prefer finding and choosing exactly what they need over just clicking an ad.

How do we not know that the spam is sent out by one sucker after the next that go until they realize it is not worth it?

Facebook and IG ads work. I agree that display network ads do not.

Ya for sure. I was a Googler so my examples were biased, these aren't the only ad formats that work but there are fewer than most people (even in ad tech) realize.

Yep, even FB "audience network" ads suck and are fraud ridden. Newsfeed is a fundamentally great format for ads.

However I have not logged into FB in six months because the actual content in my newsfeed is so toxic. I wonder if this will cause things to change, although it seems unlikely.

I last logged into FB in 2017... never going back.

Mobile devices tend not to have adblockers, so exposure might be a lot higher.

Also, the only ads I've clicked are mobile ads. I've even installed apps via ad before - I've never once purchased an item from an ad online (but I've also run an adblocker for a decade).

Ads certainly "work" by some metric, and some definition of "work", but I think the metric and definition is unclear.

I think CTD ads (click-to-download) are one of the exceptions. It's a great format for some cases, eg games. For game devs, trust that yours is addicting enough that you can safely show your users a competitor's game knowing they'll come back to you. For players, you were looking to play a game anyway and maybe you want to try something new. Click, download and play that instead.

The only ads I click on are mobile ads, and that's because many times the X to close them is too small. Other times the ad is jumping around my screen and it steals my fingerpress.

Instagram ads for millenials are like flies to honey in my anecdotal experience. So many of my peers buy their furniture, dress shoes, clothing, water bottles, watches, dog leashes, pretty much everything in their lives from seeing some like-aged person use that thing on an instagram ad (and the product usually being pretty cheap). It blows my mind how successful these dime a dozen drop shipping companies based on instagram are with millenials and gen z, and buying like this will only be more normalized in the future as more people know someone who got something nice from that instagram ad.

Yeah I've noticed I've even been drawn to something because of the mindless scroll to that's pretty neat, the button to buy it is right in front of me.

I suspect this really only works for more novelty things like clothing that are cheap enough to enough to impulse buy and of interest to image conscious people.

>It blows my mind how successful these dime a dozen drop shipping companies based on instagram are with millenials and gen z, and buying like this will only be more normalized in the future as more people know someone who got something nice from that instagram ad.

Especially with stay-at-home orders due to the pandemic, everyone (well, not everyone... but hyperbole) now has their own drop shipping brands operated out of their basements, all selling the same cheap products off of AliExpress. Will consumers catch on to what is going on? Do they even care?

Digital advertising is super effective for mobile ftp games. It is not a question if they work, it’s a question if the game can earn enough to justify the cost. Companies in that space have a super smart team of economists and data scientist constantly checking when a mobile app is worth advertising.

Someone in this thread said no one has proved to them that digital ads are ever worth it. My experience is that when they are you don’t want to advertise it to potential competitors or the people who carry your ads.

Interestingly mobile ads are typically cheaper to buy than desktop ones.

I've been playing a silly phone game for about a month now. Only last week I realized it has ads: I'd become so ad-blind that I completely ignored the banner ad they keep at the bottom of the window at all times. I only noticed it because it flickered when transitioning to another ad.

Makes sense to me.

CEO: "We need to do something about growth!"

Marketer: "Mobile ads are something!"

CEO: "I read those are hot. Here's a bucket of money! Do lots of something so we have lots of growth!"

[spending ensues]

As the Uber story shows, a lot of money gets spent without anybody making sure that it actually works. The actual goal is not to have an effect. It's to be seen as taking bold action. If the business gets better, bonuses and promotions ensue, no matter what the actual cause. Cargo cult management.

It sounds insane, but I'm sure it's rife. Years ago I was coaching at a Lean Startup weekend class. One team really killed it, rapidly testing product hypotheses through real-world testing. Their initial idea was quickly proven worthless, but they listened to users and came up with an idea that would sell.

I kept in touch with one of those team members. He went back to his prominent, well-funded startup, excited to improve his company. But nobody wanted improvement. Execs wanted to sit in a big room, think big thoughts, and produce specs. The engineers were to implement those specs. But nobody would measure the effects of anything. Proof that a grand poobah's idea didn't actually pan out was deeply unwelcome. It slowly drove my pal mad and he quit. The company bumped along for a few years to a modest acquisition, but it never really lived up to the hopes.

Engaging random marketing consultants is a good way to waste money on ads.

However, anyone engineering for advertising from the start of your distribution plans should be able to measure these things. It’s not hard to track which signups came from ad clicks versus organic traffic. The problem is that most companies start out focused on their apps and simply assume advertising is a totally independent function that happens elsewhere in the company. Invest some time into including advertising as part of your signup process and it’s really not that difficult to track LTV of customers who signed up via ads.

In fact, that’s how Uber confirmed this fraud. They noticed that some of these fake customers were clicking on the ads and getting signed up within seconds, which is something only bots can pull off. If the customers you’re getting from ad signups aren’t actually spending money in the app, you should know about this from basic analytics.

Even if you do this the ad-people will claim that showing people ads repeatedly will make them buy your product eventually. Maybe when they actually need such a product. At this point you cannot connect the events anymore. This effect is probably weak but not falsifiable.

Wow, that’s quite a revelation. Uber turns off 2/3 of ads and metrics are the same. Eventually, with optimizations, they got down to 10% of previous spend without loss of revenue. I wonder if the Uber story is unique or it cuts across all advertisers. How would this affect Ad Tech industry?

Impending crash - time to short the entire ad tech industry.

I just don't want to lose the web along with the ad tech industry though.

The web existed before the adtech industry. It will live after it.

As a smoking crater maybe. The web was orders of magnitude smaller before the adtech.

yep, but it is mostly those parts I want back. I can almldt live fine with:

- programming blogs that doesn't have ads

- Wikipedia

- three or four online newspapers/websites that I pay for anyway.

- HN

- all the old stuff: web rings and enthusiast web sites

- I'd probably miss stackoverflow but I use that site less and less these days anyways.

Most of the rest can burn and we'd maybe be better off. At least search results would be cleaner :-)

Wikipedia is constantly running ads. They're just "charity" donation ads so you don't think of them as such, but turn on the TV and you'll notice that charities advertise themselves to get donations all the time. It doesn't make it not advertising.

As for HN, how long have you been using it? Are you aware it's ad supported? Some stories are ads, usually for hiring.

> Are you aware it's ad supported?

This is 100% wrong. HN is paid for by YCombinator. Yes, some of the posts are hiring posts, but no one is paying for those.

What pays for HN is the success of the startups that YC invested in.

> Wikipedia is constantly running ads.

This is wrong too. They only run ads in November. The rest of the time the site is 100% ad free.

We were talking about adtech.

You are allowed to think differently, but personally I'm fine with the banners on Wikipedia (except IIRC and AFAIK they pretended they were short on money while in reality they were expanding).

I'm also fine with YC running job ads on YC.

I'm however not fine with 100 - 1000 [1] ad and tracking networks pretending they have reason to log my browsing habits on a typical site.

[1]: based on copying the list from "cookie banners" to Libre Office Calc and checking how many lines I got.

The web was small because it wasn't filled with ads, blogspam, clickbait, etc. So much stuff out there is just pure garbage.

Yes, a smoking crater full of exotic diamonds.

We don’t need more internet content, we need a higher signal to noise ratio.

I don't want to lose the web of the 90s and early 00s. But that one is mostly already gone.

This version we're currently in, no it can die off for all I care.

Bring on the disruption!

The ad industry has always claimed that fraud doesn't matter because it is priced in. It's been over a decade since I worked in click fraud detection but I believe Google's numbers were around < 2-5% of clicks were fraud (someone here might have up to date numbers). So this is a huge revelation from my perspective.

Working in iPhone games, our biz dev people made us integrate with the shadiest of ad publishers. But we required attribution and could track how much the users spent, so could stop spending on them quickly, which was the normal result. Most of the companies were surprised we wanted attribution and had to build something custom, which to me means at the time nobody else was tracking it so lots of customers must have been getting fleeced. From the tweet, it’s crazy that they were spending 100m without tracking the quality of the users.

Also worked in ad tech. There were some types of customers that cared about performance, but the largest were typically agencies and all they cared about was spending the budget they were given. Everybody knows this is happening, you can differentiate yourself slightly by doing better detecting it. But at the end of the day people really care about customer acquisition cost which includes the fraud, so if you had 0% fraud everyone would just raise their prices or margins

> Also worked in ad tech. There were some types of customers that cared about performance, but the largest were typically agencies and all they cared about was spending the budget they were given.

Large companies call it "brand" spending. Which is to say they're spending to maintain brand awareness. Measuring ROI would call the practice into question.

> From the tweet, it’s crazy that they were spending 100m without tracking the quality of the users.

They were high quality users: Uber didn't pay for installs, but for installs + first ride. What's being alleged is that the ad networks were detecting a user installing the app, and fraudulently creating an ad display event to make it look like the install happened due to the ad.

Depending on the content of the ads, I could easily believe that anyone who is remotely the market for even the occasional Uber ride already has the app on their phone and, if they want a ride will either use Uber, whichever ride-share is cheapest, Lyft if it's available because they don't like Uber, or a taxi/limo in situations where that's more convenient.

I guess you can offer promotions but then you're giving money away.

I’m not sure if this is equivalent to fraud. But I’ve recently discovered that 99.9% of the website visitors to my new startup’s website are just random @gmail addresses from India and China - most of which are coming from Google ads. We don’t spend much, about $100/m mostly to keep our #1 ranking in Google (which I think helps), but it does seem fraudulent. If I turn the ads off, the traffic from those IPs drops off a cliff.

With Google ads you can select the countries where you want the ads to run in. If you go worldwide and chose to maximize clicks, Google Ads will mainly run them in India and other emerging countries because it's much cheaper to get clicks from there.

Yeah so that’s the weird part that I didn’t mention. These ads only run in the US.

Maybe this is due to chinese and indian click farms running through VPN’s?

Click farms click on everything in order to dodge fraud detection, as far as I know.

I've been suspicious of this in my own campaigns as well. Google Ads lets you drill down to specific cities where your ads were clicked. Very often, Ashburn Virginia shows up very high on the list. Ashburn is where AWS's us-east region lives, and presumably also where many VPNs operate out of.

I simply ended up excluding Ashburn from our campaigns for this reason.

Is it possible to create an IP blacklist for your ads? Seems like blocking India, China and AWS/Azure/Digital Ocean would be step one.

This might help you fix the problem:

Google Ads default settings for location targeting is based on "Presence or interest" in a location, not whether the user is actually in the targeted country. That means even if you have "United States" selected as your target, your ads will still show mostly to people outside the US who "show interest in" the US.

To fix it: Campaign Settings > Location > Location Options, and under "Target" select "Presence". Never use the default "Presence or interest" option, as it will result in exactly the scenario you're describing.

This is my #1 pet peeve with Google Ads. Using the default setting is an incredibly common (and expensive) mistake people make setting up new campaigns.

The default Adwords settings are garbage (unless you are Google).

By default Google will show ads to people in or "intersted in" the selected location. You can change this to only show ads to people actually present in the specified location, from campaign level location targeting. I spend low 5 figures per month on ads and changing this parameter has significantly reduced poor quality clicks.

I also had a small Google ad spend ($350 over 3 weeks), that I setup to run exclusively in the USA, and had the same experience where IP addresses recorded by hotjar were all from India, all new signups from the ad spend were '.gmail' email addresses and although I had about 100 signups not a single one entered a credit card.

Yeah, same here. We have done some spam mitigation, but we still get so many bogus sign ups and form submissions. In this instance, who is committing the fraud? Is it Google to ensure you spend your entire budget? I don’t see anyone else benefiting from these bots and click farms. We’re so small and new that it can’t be a competitor.

Some "bots" perform searches and click on ads to build a "user profile" for the browser.

...so that they can bypass Google’s captcha.

IIRC sometimes bots will click on random ads to try to obscure the nefarious other things they are doing. You are just collateral.

Physical location only or location of interest included?

Almost the same here. Had an Google Ads budget of $200 a month. All I got from this were dubious sign ups, which all together never looked around in the application and never came back. As my ROI turned to be negative, I turned off ads completely. My organic sign-ups even increased since that. And those people are staying!

I wish more companies would try to find out the actual value of advertising and targeting. And it probably won't be as good as FB and GOOG would like us to believe.

BTW, there was a recent episode of Freakonomics Radio which talked about this eBay experiment

https://freakonomics.com/podcast/advertising-part-2/

this is a really good episode, worth a listen

^agreed

You can use third party measurement providers to validate the roi.

They go into that in the podcast. ebay had one third party prove that the other third party was just making things up to validate the ROI.

Something for the HN crowd that goes beyond articles: A HBR article[0] which talks about an experiment that eBay did testing if digital ads are effective and the paper that the actual study[1] produced.

[0] https://hbr.org/2013/03/did-ebay-just-prove-that-paid

[1] http://conference.nber.org/confer/2013/EoDs13/Tadelis.pdf

https://corgilytics.com/content/do-online-video-ads-impact-l...

This is a look at what a $200k Youtube ad buy will generate for a fictitious brand with 0 prior marketing efforts. Display advertising is so hard (if not impossible for many verticals) to get right.

Because most advertising is not effective on a 1:1 basis. The clean, direct conversion funnel is in most industries and sectors a complete myth.

Advertising is about awareness. All good marketing is long-term. Digital advertising has been sold (by overzealous marketers) as magical 1:1 sales machines. It is almost always not that.

This is the hardest thing to convey to clients. Clients want a promise and a guarantee on their investment. The truth is there are no gurantees. But that is what clients want so they go with whoever promises the most the most confidently. The most confident ones are usually either lying or don’t know what they are talking about.

I believe there is some overall culture-rotting efficacy, but it's such a depressing conspiracy: complete coordination failure with real problems, and yet a cartel of marginally negative-ROI ad spending to prop up consumerism?

I worked in wealth management up until recently and can testify this exact behavioral bias was to be found in almost every single client, and sometimes in professionals as well, especially during market downturns. It's likely to be a mix of several factors, such as the fact that losses have a much stronger emotional impact than gains of similar entity, or the fact that most people aren't really trained to reason probabilistically in an intuitive manner. It is easy to be lured in by someone promising the moon when most people won't settle for anything less.

That sounds like a religion: You don't know exactly what you get, there are no guarantees but if you really believe in it (and pay your marketing church tax) you'll probably end up in CTR heaven.

If you meet ad agency people, you will definitely get that cult-like vibe from them. There's a definite slimey feeling to any of their conversations regarding work. I overheard 2 people from different agencies trying to one-up each other with their stories of how much of their client's money they spent on ridiculousness. It wasn't about how they could save their client money and then spend it in other areas. One person I met that worked in ad tech and I told him the efforts with extensions/private mode/vpn/etc to block as much of ad tech as possible. He was totally aghast that people would go to that much effort to block their "harmless" tech. Yet another Kool-aid drinker. I have a feeling chatting with any G/FB employee would be similar.

Yeah I think Google and other advertisers are great at predicting what people will buy and then shows them ads for it. That’s different than showing a person an ad and changing their behavior, but to advertisers they can’t know unless they experiment.

But that would still make their ad spend useful. Even if not in the traditional marketing sense.

But if you can't see any change in business from 2/3 of ad spend at all it must be fraudulent.

> Yeah I think Google and other advertisers are great at predicting what people will buy and then shows them ads for it.

Seems to be entirely subjective.

I bought a new blade for a bread cutting machine on amazon, the last one lasted ~25 years. My current recommendations: blades for bread cutting machines.

I also checked some Christmas decorations on Amazon, but didn't buy any. So next to the blades I get Christmas decorations recommended. January is a bit late for that, isn't it?

I also bought a single server rack on eBay. Now I regularly get spam emails from eBay reminding me of new offers for server racks. Even the company I work for doesn't have that many, so why?

In short I am not really convinced that these platforms are good at predicting what I want. Rather it looks as if they are good at showing me what I already have.

I would guess it's fairly hard to encode a lot of rules that are obvious to most people algorithmically. Amazon does encode a likelihood of repeat purchases to some degree; they offer a coffee subscription but not a weed whacker subscription.

But besides doing something like "purchases >$100 are not likely to be repeat purchases," I would think it would be difficult.

The usual reply to this is that Amazon shows you that because you might return it and then buy a new one, gift it to someone, etc.

They show it because it "works" for them, not because it's wrong and they just can't somehow make sure they don't show something.

Because even if 99% of people don't buy anything twice, 1% probably does for some reason, and that signal just trumps all the noise. (And that's why most of ad spending is bullshit, but since Amazon's cost [even with opportunity cost] is basically 0 on their own site, they'll continue to do this.)

The answer to your confusion is that un sophisticated remarketing (showing ads for searched or purchased items) has in nearly all cases a measurable improvement on ROI. Improvements to filter out useless ad views are much more complicated and expensive. Advertisers actually don’t care about any individual case, but buy on aggregate results.

Oh I don't think anyone is confused about why this happens or why it's still worth it. The next step we need to take is to agree that it is unethical to steal many people's time for a little reward.

Agreed. Companies often don't understand the "directionality" of purchasing behavior.

For instance, I recently bought a new iPhone directly from Apple, and I bought a case for it on Amazon. Now two months later, Amazon is recommending that I buy an iPhone (...which I bought two months ago from Apple).

Buying a case an then waiting two months to buy the actual phone would be a very strange thing to do, but Amazon seems to think I should do this.

Remember this is the same Amazon that files C# and Javascript books under "Law books" etc, I have documented it: https://erik.itland.no/fun-with-amazons-ai-machine-learning

And Amazon is definitely not alone: Google has been troubled with "AI" running rampant in search results for years, seemingly with no or very little QA.

>In short I am not really convinced that these platforms are good at predicting what I want. Rather it looks as if they are good at showing me what I already have.

While that's definitely true (I get ads from Amazon for stuff I've already purchased all the time too), in that case Amazon is leveraging their own marketing channels to support their own business -- which has, effectively, zero marginal cost to them.

However, when advertisers buy space from Google or FB, that's not the case. There is a definite external cost to such ad buys.

As such, being able to identify the "value" of such expenditures is (or can/should be) important to the advertisers.

That said, much of advertising is based upon the idea of "top-of-mind awareness."[0]

The idea being that if you are considering a purchase, the brand that comes to mind without prompting when considering that purchase will be preferred over brands of which you aren't immediately aware.

Purchase decisions made on a whim or without any research are usually those which are low cost. Which is why companies like P&G, Coca Cola and like companies focus on "top-of-mind awareness."

If you're going to buy a washing machine or a riding lawn mower, you're much more likely to do research than if you're going to buy a soft drink or a bag of chips.

That said, the same "top-of-mind awareness" can be helpful even for big ticket items, as those brands may well be the first ones about which research may be done.

Identifying how such advertising may impact purchase behavior is a complex topic, and unless you can quantify specific ad-views to actual purchases, doing so is generally speculative and is the subject of a great deal of quantitative market research[1].

[0] https://en.wikipedia.org/wiki/Top-of-mind_awareness

[1] https://en.wikipedia.org/wiki/Quantitative_marketing_researc...

The probability that a random person would want to buy a blade for a bread cutting machine is nearly 0%. The probability that you will buy a blade for a bread cutting machine again is non-zero because you may be dissatisfied with your new one.

Some time back I kept hearing how effective remarketing (aka retargetting / cyberstalking) was. So I did my own (small scale) A/B test and it didn't work for my small business. Fair enough. But the weird thing was that no-one else seemed to have done the same simple A/B test. Or, if they had, I couldn't find it.

https://successfulsoftware.net/2014/12/23/remarketing-does-i...

It is also widely quoted that you "have to see an ad 7 times before you buy", or words to that effect. I tried to find out if this was true. Turns out there is no real evidence for it:

https://successfulsoftware.net/2010/06/03/do-customers-need-...

It is hard to avoid the conclusion that a lot of marketing is based on bullshit.

I also looked into Twitter demographics back in 2014 and they were clearly bullshit as well:

https://successfulsoftware.net/2014/04/03/twitter-demographi...

As someone who believes that ad tech is the primary force driving the ever growing consumer distrust of technology, I can’t help but feel a little giddy reading this story.

I'm not surprised by this a all. I do PPC for a living and I find that a lot of ad spend is totally wasted on paying for clicks that you would have gotten anyway, especially on Google and on Mobile devices.

A lot of big brands bid on, and pay for, their brand terms because someone at some point told then that they should. So when a person uses search to find Uber, they get an ad first, then a regular listing second. And almost everyone clicks the first link. Take away the ad and the free click is first.

In fairness, I've worked at several places that engaged in this practice, and they're perfectly clear that they're paying for clicks that they'd likely get for free.

The theory is that if they don't do this, their competitors buy this ad space and steal some of their organic traffic, and/or that they're just bidding up the cost of that space to make their competitors less efficient.

Now, that theory may not be cost-efficient in practice. But it's not like the marketers didn't understand that someone searching for the name of our app were going to see it in the list anyway.

That theory can easily be tested by turning off the ads in question and seeing what actually happens. Can always turn them on again later if needed.

Ebay did this 7 years ago, not sure how it turned out though https://hbr.org/2013/03/did-ebay-just-prove-that-paid

Would you be willing to make that decision knowing full well that it could reduce revenue that is 100% traceable to that decision? I mean, obviously there are ways to make this politically acceptable in your organization, but you better be completely confident that it won’t bite you in the ass.

You can just turn the ads off in one small geography and/or for a short time. Seems to me the potential benefits far outweigh the risks here. If your organization can't see that, then you have bigger problems.

Google Ads also has a built in report for seeing uplift of ads + organic and organic only

I'd take that report with a big grain of salt and run my own tests independently. Google is more than slightly biased here.

You’ve got a point. Plus attribution modeling isn’t for the faint of heart.

One popular method is a geolocation test. Turn off the test campaign (usually remarketing) in certain geos and see what happens.

Exactly what I suggested earlier... see my other comment reply.

The solution to this would be to have a very authoritative site for a query show above any ads.

If the company ranking results and selling these ad spaces are the same every decision will not be in case of advertisers/users. Google has long lost it‘s spirit and is only optimizing for money.

DHH of Basecamp had a long twitter thread about this. Their competitors bought “basecamp” as a keyword for google ads, effectively forcing basecamp to pay to stay #1 on searches for basecamp.

Google has/had no policies restricting this, unless of course, you guessed it, the keyword you are squatting is “google”.

This being allowed shows the sad state Google is in - you search for a brand, get ads for a competitor? Search engine, my ass. You now effectively need to pay Google to keep your product relevant through ads, even when the user is specifically looking for you.

If I search "Kleenex", am I specifically looking for the Kimberly Clark product or should Procter & Gamble be allowed to bid on ads for that search?

Certain types of product have become directly linked to a product name, yes. Google is smart enough to know this, though. If I google for my non VC funded product name, which is unique, why should the first result be for a direct competitor with deep pockets in advertising?

This would be fine if Google embraced their dependance on ads, but Google still labels itself as a “search engine”.

Isn't this more of a protection for established players? ie if your product is a GitHub alternative, should you be allowed to buy ads targeting ppl who search for Github to build awareness?

If I search "Kleenex" on Google, I'm asking Google to find me Kleenex, not some ads. If Google is going to start listing things in highest-to-lowest bid, then it stops being a search engine and starts becoming an auction house of links.

It already is an auction house of links. I think you have to go down at least 6 or 7 links nowadays for actual search results.

With google specifically, I've always wondered if the placement of the listing itself is related to the ad spend.

Isn't it "take away the ad" and Google will put first a ad link to a competitor instead?

I really can't feel bad for someone who wants to pay 100M to annoy people and waste their time. The "fraudsters" here have done a service to society: Uber wasn't satisfied with the free, organic growth they were getting and wanted to be more greedy and pay 100M to litter the web with ads; the fraudsters took the 100M and cleaned up their littering by "consuming" these ads with fraudulent clicks while giving Uber the outcome they wanted, albeit one they would've gotten anyway had they not been greedy.

What's important to highlight here is that marketing teams at companies such as Uber are incentivized to pus vendors to drive fraud. Specific example: Uber works with vendors A, B, C. Marketing team is incentivized to reach a cost per sign-up of $X (Let's say $50 for the sake of this example). Vendor A, running fraud, delivers sign-ups @ 45$ each. Vendor B, clean, delivers them at $55 each. Vendor C @ $65 each. The new baseline from an exec standpoint is $45 each, and every vendor that doesn't deliver at that level is cut. Repeat with multiple vendors.

The solution to this problem is incrementality measurement at the channel level. Every time you scale with a recently onboarded vendor, measure baseline of ALL conversions happening on your app. If this baseline doesn't move, cut the vendor. I say scale and not launch because upon launh, there won't be a visible impact on the global conversions. To be able to spot this spike from baseline, pick a small market than "worldwide". For ex, pick "California", let the new vendor scale in California, and measure spike in California.

I'll also mention: this is a problem that goes up every ladder. Marketing individual contributor wants to show good performance to their manager, so delivering rides for cheap is good. Marketing manager is in the same boat with the CMO. CMO -> Board -> VC (VC will be happy to see great efficiency on the customer acquisition side). VC -> LPs (LPs will be excited about customer acquisition efficiency). A limited number of people are actually deeply concerned with this, and that's why it's taking so long for the top KPIs to change from "cost per action" to "incremental impact and incremental ROI".

Having worked on the publisher side for most of my web dev career, I can say with confidence that most ad spend attribution is a load of crock. Sure people along the line claim to have attribution figured out but, coincidentally, they all attribute your spend to their "value add."

The incentives of the current ad system and ad spend attribution for everyone along the chain from publisher => advertiser are all misaligned and almost no one is doing it properly. It's so bad because everyone gets paid to turn a blind eye.

For example, lack of good bot detection. Publishers benefit by higher CPC/CPM. Ad networks, ad agencies, and creative agencies all benefit because they take a vig off of total # of ads served and/or total ad spend. Average individual paid ad buyers benefit from the vagaries of the system because it allows them to justify spend by correlation, not causation (e.g. I bought this KW and traffic went up! Too bad conversions and revenue didn't.....)

Like organic SEO, the gulf between the average and the good is huge. If you find a good paid ad manager, hold on to them because they can be worth their weight in gold.

Most companies either use a neutral third party attribution product or build their own system. They're not simply trusting the attribution stated by the companies along the line. I agree that those are all biased. I also believe many of the neutral and home grown attribution systems don't work very well.

I recently read Subprime Attention Crisis, [1] which talks about the ways in which advertising is opaque and may be leading to a bubble. In particular, the author draws parallels to the mortgage crisis of 2008. He doesn't say a catastrophic event is necessarily imminent or even inevitable (it could slowly deflate, either by design or by chance). But it has made me think that at some point there will be a tipping point and there will be a mass exodus from online advertising.

This will have big impacts on businesses like FB and Google, whose businesses have been designed around ad revenue. It would be interesting to envision what a social network would look like if they weren't incentivized to gather data about you for the purpose of advertising.

1: https://www.amazon.com/dp/0374538654?tag=macmillan-20

2 [video about same]: https://www.youtube.com/watch?v=X9dlJ4sHfSk

I like that this story references the #deleteUber tag on Twitter because it helps organize the timeline of events going on at Uber.

There was another twitter thread on here recently about the chaotic and disastrous (though successful) Swift rewrite they did which happened during the same time frame. https://twitter.com/StanTwinB/status/1336890442768547845

During those time frames is also when all of the execs were quitting or being fired, along with their internal harassment problems.

It feels like Uber has really succeeded in spite of itself.

Tim Hwang wrote an excellent short book that deals with this, Subprime Attention Crisis: https://www.goodreads.com/book/show/50403486-subprime-attent.... He addresses that most interactive advertising is never seen and prices are massively inflated. The attention is subprime, like subprime mortgages. One companies realize most of their ad money is being wasted, we might see massive recognizing in Google/Facebook valuation and in the rest of the economy as a result.

This book is the laughing stock of ad tech right now, it's got some good ideas, but it's 90% sensationalized and half-baked. The mortgage analogy is weak as well. This book is a money grab more than anything. It's an attempt at stirring the pot.

What 10% of the book is good?

I think calling attention to issues with ad tech is good, that's where my 1/10 score comes from.

do you know if anyone has written a detailed critical response?

people in the industry it is about laughing it off i think is to be expected. i doubt most traders in mortgage securitization in 2006 would have agreed the sky was about to fall.

I saw a nice synopsis from a finance person on reddit on why the analogy to the mortgage issue is way off the mark, but I doubt I can find it again. My point is that there's a whole burgeoning industry in whining about how ad companies are evil, but it's not really quite so black and white.

There is also one agent problem : manager status is linked to headcount or/and budget. As a head of marketing, you have no interested at all at reducing your ad budget, quite the opposite !

Right, if you do science and it turns out your department has a smaller effect on revenue you just lost power/status. So these experiments won't be done at most orgs.

That’s a rather naive way of looking at how corporations work. If you’re the head of marketing and go to the CFO and say “hey, we found a way to get the exact same results we currently get but at half the cost” you can bet you will be rewarded for it.

Depends. Are you sticking with that company or do you want to shop your resume around?

And given the terrible career advancement options in our industry it makes total sense to optimise for "resume value" as opposed to "business value" since your resume will compensate you much more (in the next company you're going to end up at) than contributing to your current company where you'll end up with a pitiful statutory raise.

No one is going to half the cost, and executives don't in general do any sort of cost reduction activities for two reasons. First, they would be fighting diminishing returns since there is only so much reduce-able cost. Second, they might in reality only be able to do a small cost reduction, then they are doubly screwed as they will reach diminishing returns faster. If the company mandates cost reduction executives will just fire a few hundred or thousand employees because that is deterministic, easy, and instant savings. They will just restaff when the cost reduction campaign is over.

Headcount is a sad driver for promotions, so managers are not really incentivize to do it. Only the good ones would try to keep the team lean and fast.

A smart manager won't just let their budget be cut, because that won't be good for the company. If they find that marketing approach X doesn't work as well as Y or Z, they will simply move funds to do more of Y and Z.

By doing so they will increase revenue and profits all the while keeping ad spending at the same level.

I learnt an old adage while doing my MBA: "Half the money I spend on advertising is wasted; the trouble is I don't know which half." - attributed to John Wanamaker.

Online advertising was supposed to fix this. You could never tell when a potential customer read a billboard, but you could absolutely determine when someone clicked on an ad.

I don't know what happened. I suspect everyone got obsessed by tracking user behaviour and remarketing and all the bells and whistles.

Got a call from our Google Ads account manager to better match conversion analytics with Google Ads, after 90 days we found that the ads didn't convert at all, no change in sales. End result was to suspend our Google Ads budget. The best phone call I got from Google staff.

As a publisher and ad network vet, I can vouch that Dr. Fou knows his stuff. He’s been one of the clearest voices about the mechanics of ad fraud. Any engineer would be enlightened at what wild escapades have been hacked into DOMs around the world on the backs of starving publishers. Any business person would see how much ad fraud has shaped the world we live it. He’s a fellow that’s definitely worth a follow!

thank you!

Friends don't let friends do programmatic direct response advertising. Do it directly on the platforms yourself.

I run an ad agency with relatively well known clients and I consider the word "programmatic" in a resume a negative correlation.

Care to elaborate more? What exactly is "programmatic direct response advertising"?

So programmatic means using a 3rd party ad tech tool to place your ads across all internet ad placements.

Doing it directly means placing ads yourself on the biggest platforms (Facebook, google).

Direct response just means "if you actually want results/ROI", as opposed to for nebulous brand goals.

Playing devils advocate, couldn’t this just be reflective of the fact that Ubers brand reach us strong enough that there is very little advertising that gives them a marginal benefit?

Uber owns taxi services as a brand as much as Google does search.

I guess there are 3 reasons why reducing ad spend might not show a negative effect:

(1) Due to fraud, the money wasn't actually going to ads.

(2) You didn't need the ads. You were over some saturation threshold and the law of diminishing returns kicked in hard.

(3) The ads were valuable to you but not in an easily measured way. Maybe you could coast on brand awareness inertia for 6 months or a year before it affects sales.

This case seems to have decent evidence for at least part of it being #1, but not necessarily all of it.

you are right; some of the biggest brands need to do the least advertising

Related article posted earlier today: https://news.ycombinator.com/item?id=25620707

It mentions that other big companies, such as P&G and Chase, also noticed that cutting ad spending had no result on their business outcomes.

Yup, the problem is advertisers want big numbers more than they want ad performance.

Honestly though why tf did these companies never try lowering spend incrementally to see if it made a difference? Is this why engineers are so valued because we understand the simple concept of AB testing?

For the same reason that you never lower your margin. It's very easy to lower your margin, and you might get growth as a result of your lower margins, but it's a massive gamble with the most likely outcome being that you are now selling the same stuff for less money. And do you know what happens when you cut your marigns and don't see growth? You get fired.

Same thing with advertising. You drop your ad spend, you lose market share, and you're fired. There is no step 4. If you're right and the ad spend was unnecessary, you've made a tiny difference (no one cares about cost, only growth), if you're wrong, you're fired.

Disclaimer: I’m not a marketing guy.

It’s sold as hand wavy. I once dropped in on a consulting client (at the request of an investor on the board) and did a quick assessment. The assessment was holistic and marketing was one of many enterprise value levers, so did a rough check. The marketing firm, very similar to the Uber commentary in the underlying podcast above where he found 20,000 installs coming from an app with 30,000 MAUs, I calculated claimed performance which back of the envelope would imply you could capture the entire US vertical for that industry for sub $X0M of annual spend. I knew that space very well, and well that just was not possible.

My quick question kicked off a large tap dance in which some finger pointing occurred and a “new attribution formula” was introduced.

The vendor got fired.

This was sort of a reasonably sized entity, with some sophistication. But think about it, unless you’re tech native, your VP of marketing has no granular data abilities in 2020, and unless you have a pool of SQL capable analysts lying around, AND you’re going to insist to validate on your own, it simply doesn’t work.

So you trust the vendor. But wait the vendor gets paid X% of what you spend. Hmmm, would they realllllly work super hard to make sure the results are real if they are not easily found out to be falsified? This is a paranoid stance, obviously. But intrinsically structurally flawed.

The Uber guy on the podcast straight up said he wouldn’t buy a dollar of programmatic ads without a big data team internally and a policy of (loosely quoted) “we need that file to check and if you don’t send it to us we’re cutting you off”.

That’s just a lot of work.

Marketers do tons of A/B testing and most measure conversions based on ad copy and placements. The problem often is they don’t look beyond aggregates ( as Google/Facebook flood marketers with dimensions, this one is easy to overlook). So it the overall ROI may be passable, but specific segments are over-analyzed (did YOU place the right message?) and others ignored. The platforms have never ever in any amount of years been willing to hand over raw clickstream logs of ad data, so marketers are forced to rely on misleading aggregates.

I’m sick of engineer elitism. Try harder to see that brilliant folks exist in every department. You’ll be a happier person and learn a lot more. Marketing analytics is fascinating and full of puzzles. Marketers were driving engineers to make multivariate testing scale 20 years ago.

A lot of comments are rushing to dismiss the entire concept of advertising as ineffective, but they’re missing the point of these Tweets.

One of Uber’s advertising partners created numerous apps that did things like auto-click on their ads and install apps in the background, collecting the commission for themselves on customers who didn’t actually click the ad or intentionally install the app.

They discovered it when they turned off those ads and didn’t see a drop in signups, which is something that normally happens when they turn of legitimate ads.

Right, it seems like the point isn’t “Targeted ads don’t work,” it’s “There are ways to fake ad targeting that fooled a major company.” Uber could benefit from legitimate ad targeting (assuming they haven’t hit market saturation), we just don’t know.

I think Uber is to blame here, reading the story it is obvious that they bought ads from very shady sources/networks, probably they went for the cheapest CPC. You get what you pay for.

"You get what you pay for" doesn't mean you deserve to be defrauded for choosing the cheap option.

On the other hand, if you pay someone $20 to buy a brand new ApplePhone 12 XS Max, you should expect to not get the phone. The Twitter didn't mention anything about price; and it is possible that the fraudsters were actually charging a normal market rate; and nobody deserves to be defrauded for trying to get something for cheap; but if Uber was taking the cheapest option then it's hard to feel sympathetic towards them - they're a big company with lots of marketing people and they should know better.

In reality, you get AT MOST what you pay for. There are plenty of cases where you get much less, or nothing at all, for what you pay for.

This story goes to show that.

This looks like a very significant disclosure on the businesses' part. Before COVID-19, there was no reason really to turn-off the ad spending just to see what happens (with some exceptions, such as Uber in the other article). I was surprised to see that the small businesses have experienced the same outcome as the large ones. It also looks like at least partially this seems to be coming from the mobile advertising. I wonder, how should we change our Ad placement strategy as a result of this data?

I think all of us know deep down that something about online advertising doesn’t seem right. I’m still surprised at how big of an industry it is given how ineffective it can be. To me it seems overvalued.

I’m not saying Facebook, or online advertising is perfect, I have not run Google ads because most of my audience hangs out on Facebook and Instagram.

I have been running Facebook and Instagram ads for 2 years now for my small business, and what we do is we hyper target the audience, from their age, sex (the business appeals more towards female demographic), to their schools and job (people who work at an office or get a higher income is more likely to purchase)

From this hyper targeting we are able to generate a good amount of sales, sometimes more sales than we can handle.

And since we didn’t do any advertising during the pandemic, we can really see the difference our advertising makes.

I believe these companies maybe targeting a broader audience which maybe diluting the quality to a point where there is no difference between the organic reach and paid advertising reach

(A broader audience may expose your ad to fake accounts, bot accounts) etc.

I expect Unilever has some data from their Facebook and Twitter ad boycott.

They're restarting their advertising.

https://www.campaignasia.com/article/unilever-to-end-faceboo...

The use of the "fraud" here is curious, as I don't think it's what's most advertisers would use. "Waste" would be a better word.

Fraud generally indicates a bad actor or something the advertiser was unaware of. Claiming to run my ads on the New York Times and instead running them small blogs would fraud.

On the other hand, if I chose to run my ads for wedding dresses on ESPN.com, they would be very effective (they'd have a lot of waste), but that's not really fraud. I made a bad decision.

I read the Uber comments as 2/3rds of their spend was waste, not fraud. That still seems a surprisingly high number, although I wonder how much of it was upper- or mid-funnel, which is notoriously harder to measure.

That's what happen when you have too much money to spend and you don't take time to evaluate if money is well spent.

That's what happens when your own job is to waste oxygen, pull numbers out of thin air and take credit for "conversions" that were organic in reality; in this case you have no incentive to optimize things because you'd inevitably show the company your own position and maybe entire department is irrelevant and a waste of money.

No, it's what happens when the objective is something different. The objective is often to get big ad views & audiences, not to drive performance.

... or deliver value to customers ...

The customers value the big numbers, so...

I am surprised that the concept of incrementality is not discussed more in this context.

At a high level, marketers measure the effectiveness of ads using metrics like ROI, ROAS, etc. The way those metrics are measured is by choosing a certain attribution model (eg: last click, first click, etc) and attribution window (eg: 1 day view, 28 day click). As you can already tell, it's a bit like accounting - you can get different results depending on which rules you follow. So clearly, there's a lot to criticize.

But, a lot of that criticism goes away as soon as you introduce incrementality and start adjusting for it (randomly split the population, only show ads to one group, then observe the results between the two groups). If the two groups exhibit similar behavior, your product is already getting a good amount of WOM and other sources of distribution, so no need to push that paid channel quite as hard (or at all). Most young companies are not in that position, and the amount of adjustment needed is in the sub 20% range. Whatever the range, my point is that the concept of needing to make this adjustment is well known and understood, and the fact that this was never mentioned in the original article is really strange - it almost sounds like if the involved people didn't know about it.

Here's more info on incrementality: https://www.adroll.com/blog/marketing-analytics/beginners-gu...

Tell this to the poor scammy-dating-site-slash-mail-order-bride-site operator who has been sold 10+ years of impressions on me:

- an extremely happily married man

- with multiple small children

- in a rather conservative church

Sometime during the last two years they finally stopped advertising to me. I've always blamed their marketers - or Google for being so utterly incompetent as to not pick up any of the available, extremely strong signals sent out - including me reporting the ads as irrelevant on a number of occasions, but maybe I should blame it on incompetent middlemen?

If you take a look at people who have been caught in extra-marital affairs I think you'll find those three qualities are not immediate disqualifiers.

Maybe the "happily" part, but how does Google know that?

> Maybe the "happily" part, but how does Google know that?

3 hints for Google:

1. I newer showed any interest in those ads.

2. I never sought out anything related.

3. I repeatedly put in the effort to mark these ads as irrelevant.

Maybe Google knows something about married, church-going men with children the rest of us assume to not be true.

Not that it applies to you, but more generally to a similar demographic.

The podcast episode where Kevin Frisch (former Uber head of US/CA performance marketing) talked about this issue: https://listennotes.com/podcasts/marketing-today/historic-ad...

All podcast interviews of Kevin Frisch: https://lnns.co/5UZnQ_FopCb

Why is no one discussing that the term "defrauded" is not substantiated by the tweet?

I've been running on site attribution surveys[1] on a handful of brands that I work with and the results have been pretty surprising. Word of mouth tends to be overrepresented and FB underrepresented though there is probably some sample bias so it's far from a sure thing.

[1] https://www.zigpoll.com/examples/attribution-survey

A lot of ad spend isn't about positive ROI but "being in the game," i.e. if you don't advertise then customers, suppliers and partners all perceive you as a market laggard. That perception has major consequences. For example, the top technical and managerial talent goes to your competitors.

But there are companies that do little advertising - they have other ways of building and sustaining their brands.

For anyone super interested in this topic (and the potential ramifications for Big Tech and society), Tim Hwang (former global public policy lead for artificial intelligence and machine learning at Google) wrote a wonderfully in-depth book about this. See the link below: https://us.macmillan.com/books/9780374538651

I know someone who worked at a marketing "machine learning" company. The ML really needs to be in quotes, because if you looked under the hood you'd see the people behind the curtain (literally -- Amazon Turk workers). But VCs loved them, customers loved them. The blind selling to the blind... Marketers had money to spend, ML is the hotness, everyone makes money! (My friend no longer works there)

I don't know now, but some time ago Google consultants told me that I should create an ad that displays when someone searches for my brand name. It was very odd because without it, my page ranked obviously first for the brand name. It may be the case it looks good in ads reports to see paid conversion.

What definitely made me realize what ads are about was when I have installed user interaction tracking on my website. I was able to see what paid ads users was doing.

90% of the traffic was generating users that behaved like bots.

But there is a big marketing and a lot of money for everyone. No one really cares about results. You can find tons of articles how to optimise campaigns that are aimed at SMB.

But no one will mention most of such advice are for companies that spend tens of thousands of dollars per month. Otherwise there is too little data for ROAS or even CPC and CPM optimization. Comparison results for smaller campaigns are just not statistically signifant.

I must agree with the article it is all about perfect crime. No one , including clients, are not concerned about real results. It is like let's throw money into the fireplace and enjoy it is warmer.

Yeah, not surprised at all here. What is surprising is that they actually found this out and acknowledged it.

Used to work in the performance marketing team for one of the top 10 brands as an analyst/data scientist, and the reality is that the company politics and individuals' career ambitions top the science. Most of the time the work that gets visibility is spent on "creating stories around data for the leadership": marketers do not understand at all the data they are fed to by the advertising platforms. Also, running controlled A/B studies tended to show between 0 and 10% (this was rare, closer to 0% more common) causal impact on campaings making none of the marketing campaigns profitable in the sense that was used for performance evaluation.

There is one key visicious loop keeping the game going:

1. The marketers want to show great numbers back to the leadership (internal storytelling). Their employment/bonuses depend on the number reported by the ad platform.

2. The ad platforms want more of your ad spend which can be achieved by showing better ROI in their platform.

3. Both the marketers and the ad platforms want to show as good numbers as possible, leading to a situation that nobody is incentiviced to measure true performance / correctly measuring performance.

None of the A/B tests nor analyses about ad fraud went beyond the marketing director. One cool case was analyzing Google's DoubleClick ad logs for a bunch of campaigns that showed there were few cookies that received a significant % portion of the total ad spend - meaning that few "insividuals" received tens or hundreds of thousands of ad impressions.

Nobody cared and this still goes on. It is such a toxic environment to work in I ended up changing career a bit.

There is this nice book on the subject called Subprime Attention Crisis by Tim Hwang, on why the whole ad industry might be a bubble.

And this Freakonomics podcast episode on digital advertising https://freakonomics.com/podcast/advertising-part-2/

The only sane way to do advertising is to do what’s done on TV: you buy a time slot based on estimated impressions and then a third party auditor (Nielsen) uses surveys to get a better after the fact number. Doing sales based on server traffic is just pointless. You get what you pay for (a server moving bits around), and not a damn thing more.

Are Nielsen surveys still a thing? Every single device that displays a video signal to you has been tracking what you watch since way before ad tech became a thing. The companies collecting that data then profit from that collection.

As an anecdote, I have AT&T Gigaverse. I primarily only watch recorded material on the DVR, and only watch sporting events live. However, once I'm done watching a signal from my "cable box", I switch it over to my external media device on a different input on my TV. The cable box stays on whatever that channel it was on for however long it takes before the cable box goes into screen saver. Someone is getting skewed ad view counts by 1 as my cable box displayed it but nobody saw it.

This is probably a waste of electricity?

Yes. It is. However, if you can tell me how to turn it off in a manner that actually turns it off without unplugging it, I'm all ears.

Two related articles I read this year: [1] is an account of a deeper analysis of this problem, [2] is AirBnB's experience with cutting off spending on online advertising during the pandemic. The conclusion is the same, but what is important is the analysis of the reasons for continuing ad funding in orgs which has more to do with the marketing executive's psychology and need for feeling important than any objective analysis of the effectiveness of the campaigns.

[1] https://thecorrespondent.com/100/the-new-dot-com-bubble-is-h...

[2] https://www.theinformation.com/articles/airbnbs-new-strategy...

There is a thread in r/adops in response to this HN thread for some good context.

https://www.reddit.com/r/adops/comments/kprrcd/uber_discover...

This has already 303 comments so probably no one will see mine, but I was planning this year to start doing some online advertisement to get more visitors to my website and convert them into Patreons. But I think this will not a good idea after all. Any other recommendations in how to be more visible to more people on the Internet?

He who cracks the code of the call center assisted lead gen in the online world will win the day. This is why "influencer" marketing is such a draw. There is NO more attributable conversion statistic than "I talked to THIS person, and had them talk to YOU in order to make a purchase". I've run agent-assisted call center lead gen for education industry and over the past 10 years, nothing has been more profitable or effective. The call center conversion rates subsidized the HUGE losses on pure digital and other ads.

I am enjoying seeing this coming to light more and more. Paid Ads with FB and Google used to actually be really good, they worked really well but since the influx of Drop shippers, algo updates and everything else in between they have both got really expensive and drive far fewer results.

I mentioned in another post on HN about Googles Quality Score of your ad based on 0-10 scale. There is no clear way to fix the issue, you just have to read Google's Docs and figure it out. In the mean time, you will be charged extra of the low-quality ad but it will be shown still. Now you are spending money, changing a headline here, changing a bit of text there and still you get 4/10 score.

PPC has become nothing more than gambling with it almost feeling like 'If I just change one more headline I might get rich' - this is what it has felt like to me for a long time now.

I've had numerous experiences working closely with big companies and small companies' marketing/media buying teams. From my experience, these big companies lack the diligence on ROAS. Maybe it's the lack of ownership, whereas for smaller companies, usually the CEO/COO/Founder(s) on top of their spending and other performance metrics. I see much less of a waste on the ad spend when it comes to customer acquisition.

For example, I had a company that was spending a million dollars a month, they are a big brand, a tech company, their team kept on asking why the spend wasn't up even when their CAC was way over their target. The team wanted to keep the spend up to meet their metrics at any cost, in reality for every customer they were acquiring they were losing 50% on ROAS.

Good to see this type of fraud being called out.

I once worked for a company where we were tasked to create "fake users" to make it look like a website was getting way more traffic than it actually did on the premise of doing an "extended load test".

Come to find out years later the person commissioning the fake users was trying to sell the website and wanted to validate their asking price -- with fake users and fake traffic.

Long story short it didn't work out that well because eventually the algorithm running the fake users got entirely blocked out by whoever was the host at the time and the guy wasted his money.

I just wonder how many times things like this have happened and people were very successful from it.

Fake likes and fake posts by fake users who then are clicking real ads seems like the landscape nowadays =/

The Uber incident has been known for almost a year. There are other instances, mostly undisclosed audits which have shown similar results. One audit revealed $220m in ad spend for a global brand with a +$500m budget had no impact on their results, it was vapor. On the finding of these results, they didn't want public embarrassment or the hit to their share price.

Not everyone in the industry is fraud, not everyone in the industry is a huckster. But most of the industry is people talking up their game and creating a facade, from Brand CMOs to Pubs gaming their performance numbers. Fraud can be reduced, but few want to engage in the rigor and even less want to recognize actual/validated performance when the current fiction suits their purposes more easily.

When Google replaced mobile app install campaigns with machine learning based "universal app campaigns" (https://support.google.com/google-ads/answer/6167162?hl=en) that give you very little control over the placement of the ads I always wondered if they were simply showing ads to Play Store users who were searching for the name of our app and charging us for installs we would have gotten for free anyway. Does anyone have here have experience with UAC or know how to prevent this type of thing from happening?

This is really just Uber failing to effectively manage their ad spend. Somebody at Uber is not doing a core function of their job if this can happen. A person at Uber is responsible for ad spend, moving it between vendors, checking CPAs, etc.

Yeah, first thing I thought as well when I read this.

Dude has $150MM annual ad budget and has never audited his vendors?

Either total incompetence, or something shady was going on.

As with almost everything in life, there are multiple factors to take into account.

We recently ran several drip campaigns, as well as a few fire-and-forget campaigns, together with corresponding ads through Google. There was a marked difference in customer uptake before and after the ads. We have also pretty regularly received feedback on our ads.

In fact, that is what we have observed over the years. Customers are conscious of what the ads say, where they get placed, and - usually - can crosscheck the quality of the ad with that of the product.

In my experience, it is rash to generalise and say that ads are not useful.

So Uber has 15B revenue and they only spend 150M globally on acquisition ad spend.

It'd seem to me that this is relatively thin, and likely wouldn't have any impact for some time.

Uber is already well known - their spend in most cases boils down to brand advertising and nothing else.

Marketing is hard, and anyone just throwing money at something isn't going to win hard.

For an established brand, user acquisition is going to obviously have to be more targeted. Putting ads on 'Breitbart' was never going to drive new customers.

In fact - I don't think 'banner ads' will drive any new customers for Uber.

> So Uber has 15B revenue and they only spend 150M globally on acquisition ad spend.

The events described here happened right at the start of 2017, so you wouldn't want to look at their current revenue. Their 2016 revenue was $4 billion, 2017 revenue was $8 billion.

It's not necessarily that the ads aren't effective but their reporting of ROI has been off in my experiences. I've been running attribution surveys on the e-commerce sites I own [1] and the results have never matched up with what FB or Google reports back. Some of it may just be bias but I can definitely how some rounding errors may start to creep in with large numbers.

[1] https://www.zigpoll.com/examples/attribution-survey

Could some of this be because the given timelines are too short? They gave the example of Chase, which moved from 400,000 digital ads to only 5,000, but they declared there was no significant difference in business after "only a few days."

As a person who has never used Chase, I'm not going to suddenly want to switch to them just because I saw a banner ad. That would be ridiculous. It's the sum of all the Chase advertisements I've seen in the past decades that have made me believe they're a popular and trustworthy bank.

It would be the downfall of the Google empire (and probably half of the Internet) if the majority of companies realized that most of their online/mobile ad budget is absolutely wasted. I don't know of any other industry where people put so much money into it only to have so little tangible evidence at the end of the day of what they really paid for ("look, X real people clicked on your ad.... at least that's what we tell you and whatever that means for you.").

Here's a nice article the correspondent did about the online advertising bubble. Not a lot of evidence of the efficacy of advertising, and few dare to run the one experiment that counts (pull all ads for a quarter and see what happens):

https://thecorrespondent.com/100/the-new-dot-com-bubble-is-h...

For research we made ads which were grey, flat field, and basically said "please don't click on me" and we got astronomical numbers of clicks sometimes. Hmm...

But... reverse psychology works? Wouldn't people be curious what's behind the "curtain"?

Yea, there's some of that.

A better test ad would be something that looks like an obscure website failure/error message, like "404 Ad Not Found 18afe4481".

Or even just a copy of the placeholder image $POPULAR_BROWSER uses when an <img> link is broken.

This related article is excellent – "The new dot com bubble is here: it’s called online advertising" by Jesse Frederik and Maurits Martijn. https://thecorrespondent.com/100/the-new-dot-com-bubble-is-h...

> Marketers are often most successful at marketing their own marketing.

I remember reading about ad-fraud on HN. I can't find the post, but they were talking about "ad attribution" and how the ad-world has items that may have some statistical meaning, but it gets lost in the sales pitch. They ended their post by talking about how the bottom will eventually fall out of from under the ad-world in a matter of time. Uber finding this out the hard way doesn't surprise me.

Does the idiocy of these companies surprise anyone? I had people tell me I can't stream videos... here it is... thousands of paying customers every month. The difference between Uber and I? I'm profitable. I'm coming after Udemy and Pluralsight. https://www.codehawke.com/all_access.html

In the middle of the night I visited a dutch but and sell website searching for something specific. The search results, page after page, were filled with advertisements from the same auction agency. My single visit, clicking though 20 pages produced well over a thousand ad impression. I had to click though them to find other peoples ads. The next day they were all gone. Apparently the budget ran out.

"For ex, one ad network launched 'battery saver' style apps in Google Play, giving them root access to your phone.

When you type the word 'Uber' into your Google Play, it auto-fires a click to make it look like you clicked on an Uber ad and attribute the install to themselves "

The things you can do on Android devices are insane. Also ads aren't necessarily for conversion, they're for permeating brand.

Another related article skeptical of the effectiveness of online ads (by the Correspondent, which is sadly shutting down this year): https://thecorrespondent.com/100/the-new-dot-com-bubble-is-h...

A root cause of this is an overreliance on Multi-touch Attribution (MTA) models, instead of true incrementality experiments: https://www.adweek.com/programmatic/lower-ad-fraud-will-be-a...

I'm reminded of the famous quote by George Best: “I spent a lot of money on booze, birds, and fast cars. The rest I just squandered.”

Uber instructs ad buys to block all ads on breitbart, but ads still appear regularly. Uber identifies agencies that are responsible and stops all advertising buys with them. It amounts to 10% of their ad budget, no change in new users is detected despite 10% drop in ad spend, about 15M.

Edit: oops didn't finish the whole thread.

Next uber turns of 2/3 of the ads. 100M. Still no effect.

Well people are not going to magically drink less coca because there's no ad. Best case is you get the effect 2 years letter.

Old joke: Advertising exec says: "I know I'm wasting 50% of my ad budget. I just don't know which half."

Ad tech is a bubble and full of sketchy players. The advertisers always get fleeced, the only question is for how much.

Run surveillance on masses and defraud businesses: the advertising industry.

Of course it could just be a parallel construct for funneling money to surveillance tech companies such as Google by compliant companies that know they are not getting anything for their ad monies, to serve the policy goals of unseen but powerful interests.

I was VP Growth for a fintech for a while. And the one thing I learned that Paid Ads are a Trap. Here my open the read medium article about this https://medium.com/@franz.enzenhofer/ads-are-a-trap-80df01d2...

Tl;dr: money is at best a halfway decent accelerator. And as long as you don't massive positive retention just a sinkhole of fake growth. And if you have this massive retention there are better ways to invest your money.

Depends how much of that is attributed to fraud. There should be equivalent of cyber security to ad “security”.

I like how some people in this thread are calling some transparency in adtech space "take down".

I remember reading a while back that if you pay for likes from a like farm VS genuine, you get about the same quality of likes because the like farms have to like EVERYTHING to cover their tracks. How is anyone supposed to know they're really getting what they paid for?

It's amazing how large brands like Uber aren't tracking actual performance. There's "metrics" and then there's actual results. I worked at an ad agency on the HBO account and was shocked at how basic their digital team and our agency team was.

Can someone explain the tweet for me please? I am really confused on what she's trying to say.

Uber were running ads, and some of them were appearing on Breitbart.

There's a campaign called "sleeping giants" that looks at a company's corporate values, and looks at where that company is advertising, and asks whether that ad placement is compatible with the values. Breitbart has freedom to say what they like, but customers of Uber don't have to buy the megaphone.

Uber tried to pull the ads from Breitbart.

But the ads kept appearing.

So Uber looked closer, and they realised some of the data reporting was fraudulent. One ad company had a "battery saver" ap on Google Play. That ap had strong permissions, and it was scanning for people typing "uber" into the Google Play store. When they did that the ap made it look like the user had clicked on an ad and then installed the Uber app when they'd just organically found the uber app themself.

Uber closed off a lot more of their ad spend and they found no difference in user signup.

ah, so the investigation into the Brietbart campaign made them discover other things completely unrelated to Brietbart itself. Thanks!

I read a news article yesterday that had the same ad repeated roughly 20 times. Would the metrics show a separate impression for every one of those ads I scrolled past?

Also, despite the repetition, I don't remember what it was for. I think it had a photo of a woman outdoors.

> The biggest story in tech no one’s talking about

Is this not an old (3 years), well reported story?

https://www.reuters.com/article/us-uber-fetch-idUSKBN1ES1ZI

I had consulted for a gentleman that owns a one-word domain. He was using AdWords to direct traffic to his site, without any analytics.

I added analytics and began tracking conversion rates (sales).

I was able to reduce his ad-spend by 75% and maintain the conversion rates.

90% of email traffic is spam. 90% of ad clicks are fraudulent. 90% of visitors? 90% of tweets? 90% of likes? Some day, 90% of YouTube videos. As human-mimicking technology improves, all human interactions that can be faked, will be.

I actually did find a few products which I used for at least a short while, on Facebook.

If I had to guess, hyper targeted digital ad spend is very effective, but you'd have to sell very high margin products to come out ahead.

I believe the article is mainly around third-party websites displaying ads, not FB. I agree that Insta, FB, and Google ads are effective, but only on their own properties. I ran Google ads for an ecomm project and quickly disabled the affiliate site ads since they generated garbage traffic.

Ya anecdotal for sure but a ton of my friends bought their significant others some new fangled thing they only heard about because they saw it on Instagram. HN loves to bash marketing (and FB/google) but it has never been easier to spin up your own shoe company and easily advertise and get customers (e.g, allbirds).

I've also seen and personally know people who bought crap off Facebook ads and ended up either being scammed or with a product below what the ad promised.

For me I've been very happy with my Facebook purchases

A solid organic marketing strategy is worth investing in, rather than pushing all your funds in traditional digital marketing. Traditional ads often feel forced, and internet users are keen to ignoring them.

This smells like BS... Uber can easily track user behaviour with their own metrics, and see how many users are actually real users.. No way this would get unnoticed...

OR there is something really unhealthy going on at Uber.

Something unhealthy going on at Uber? That's never happened before...

https://arstechnica.com/information-technology/2015/02/50000...

https://www.wsj.com/articles/uber-to-pay-148-million-penalty...

https://www.washingtonpost.com/blogs/the-switch/wp/2014/12/0...

True story, when I joined Uber, this was my team's first project. Our goal was to find unattributed spending and bring transparency in ad spend with automation and better tooling.Ask me anything.

True story, when I joined Uber, this was my team's first project. Our goal was to find unattributed spending and bring transparency in ad spend with automation and better tooling. AMA.

I’m continually haunted by a fear that online advertising is a bubble propped up by fraud and willful blindness, and that when it eventually pops it’s going to destroy all our careers.

So think about how many ads you've clicked through in your lifetime on the internet.

Honestly, I'm with you. I have no idea who's using the ads other than logrolling robots.

Digital ad spend feels like an 'emperor has no clothes' situation. No one wants to admit it, but it is highly inefficient and fraught with fraudulent clicks / views.

Tangentially related, but how is Uber Caller still expanding (even to various countries) on WeChat? Aren't they essentially using stolen cards and accounts?

I hate that we use twitter to report on stuff...it feels like the worst medium to talk about meaningful issues and the easiest way to get things twisted.

I continue to believe the main reason Facebook is scared of new iOS tracking controls is people will figure out the CTR is the same without the tracking.

Off topic but why do people make these twitter threads? It is the most uncomfortable way of reading more than 140 tokens. any other platform is better.

I feel like a company like Uber could of been justified put the large majority of there ad budget into well placed physical ads in cities and airports.

Does this mean that all of the so-called "small business needs it" arguments against durable federal privacy legislation are totally moot?

ebay did the same years ago and found out that ads were relevant depending on the new/repeat cohort of visitors.

the ad market is looking a lot like the 2008 subprime market, lots of opacity, lack of accountability across all actors (Agency, platforms, customers). And if we consider how relevant is the ad money in making the internet able to run nowadays, this starting to be pretty concerning.

Wow so this means if some companies optimized their ad spends to stop spending on fraudulent ads for growth they could probably turn a profit.

So, somehow blackmailing a company to stop buying advertisement at another company you disagree with turned out to be a good thing.

I always saw ads as a zero sum game. We all have limited dollars and attention. When you advertise, you might slightly change the allocation of attention and dollar to your product.

When everyone advertises, it is back to the same status quo. And when everyone stops advertising it ends up being the same.

I really hope more companies would stop advertising or at least question the value they are getting out of those money pits. Ads created some of most horrible companies the world ever saw (Google, Facebook).

I always saw ads as a zero sum game. We all have limited dollars and attention. When you advertise, you might slightly change the allocation of attention and dollar to your product.

That's worth thinking hard about. Is advertising worth it? With most US consumers spent out to the limit of their income, all advertising does is try to move demand around. It's thus zero-sum and a waste of resources for the country.

I'd argue that ads should not be a tax-deductible business expense.

If ads don't work, how did the advertising industry convince people to spend on advertising in the first place?

Many (most?) people high up enough in an org chart to control a significant budget in online advertising are so abstracted away from the actual mechanics of that spending and its impact, coupled with communication chains that get increasingly political (you don't want to say your expensive proposal did nothing, after all!) and you get a recipe for shady ad networks doing basically whatever they want.

They work on the margin, so if your competitor spends on advertising, you have to spend as much yourself just to cancel out the effects of their ad spend.

As for big brands and brand advertising - the same applies, there's just more inertia.

Welcome to the Advertising Tech Bubble.

Maybe. A lot of people have been saying this for over 10 years. If it's a bubble that's real though, the effects are going to be pretty widespread. Not only do two of the biggest (and best-paying) tech employers get slammed, but so do all the inter-related companies. And, also BTW there will also be a lot fewer fat exits for startups if Google and Facebook acquisitions get turned off.

First signs of bubbles are fraud and wrong valuation.

First sign to me that the dot bomb was imminent was a two inch article in the WSJ summarizing a finding that something like 2/3rds of startups had board members or upper managers that had been sanctioned or investigated previously for securities fraud.

It's telling that all the FAANG companies are basically in open violation of a number of laws.

I love it that it was the Sleeping Giants campaign against neonazi website Breitbart that started this.

When I'm reading this from my phone, 50% of the screen is taken by ads, so please, Forbes, stfu.

Online advertising is fundamentally broken because it pegs dollar amounts to HTTP requests. Ad fraud steals billions from advertisers because it's so easy to fake being a publisher, impression, or click. Not to mention that you have to take the word of the upstream partner (ie Google) that the requests are valid without them providing you any actual request logs.

Wow there are actually orgs such as sleeping giants? They actually make living off harassment?

Can't wait for the Enron-like moment in adtech. The numbers are all fraudulent.

You need proper conversion tracking or you're gonna have a bad time

But without unlimited tracking, how will small businesses survive?

Looking forward to Google ad Facebook doing some massive refunds.

Couldn't have happened to a more deserving company.

big brands have the least to gain from online ads

Apparently because they're being defrauded.

How were the ads "slipping thru"?

The podcast[1] does a lot better job of explaining what is going on than Twitter.

[1] https://www.marketingtodaypodcast.com/194-historic-ad-fraud-...

thanks

Some vendors weren't respecting the Breitbart ban.

adapp solutions, hasoffers, tatto media, tune have been defrauding their customers for quite some time now

Wow, Twitter is such a horrible way of telling an important story.

Augustine fou is basically a troll in r/adops.

To some extent yes, but I think he's better than a troll, because IMO there is value in what he's doing, despite the hyperbole.

the adops community helps me to understand the mechanics and when i don't know something I usually turn to that community to ask questions.

most are helpful, but a few anonymous ones spend their time making polls that make fun of me; but that's OK, I know it comes with the territory

So, there's some bits of this story that hint at the problem here. The big headline is that as they removed Breitbart, their spend was reduced. If you're really doing "performance marketing" in the sense that people think of it, that doesn't make a lot of sense. Of course, it makes total sense in the industry.

If you are properly doing performance marketing, that means you're going after the best opportunities to get customers, and you're paying as little as you can for those opportunities. If you cut out a whole set of opportunities from consideration, that should not change your spend. If those were really some of your best opportunities, that means now you'll have to show ads where you have worst opportunities. Either you'll have a lower chance of getting a conversion or you'll have to spend more for each ad... either way, your opportunity cost should go up. Basically you've constrained your supply without constraining your demand, so the price might go up, but demand still gets fulfilled.

Except, this isn't what really happens in the industry. Instead, you're given an ad budget and a campaign window, and you're told to burn through the budget within the campaign window. Most ad networks will try to find the best inventory to give you out of that campaign window... except often the "performance" they're chasing isn't "performance". It's "impressions" or "unique users" or "clicks" or "conversions" (where the goal post on what a conversion means is moved to "goes to the landing page" instead of "buys the product"). So what they end up doing is finding the cheapest (i.e. crummiest and least likely to actually perform) of whatever your "performance" goal is. So you'll get impressions that are least likely to click, or clicks that are least likely to convert, or conversions that are least likely to actually buy your product.

This happens, because there is actually a fixed constraint on the quality side of the supply.

You're trying to spend $X in a tight time window, and the dirty industry secret is the need to spend that money so quickly is the real challenge. Ad networks don't get paid the part of the budget they don't drain. The ad spend is a big deal. More important than performance. Ad execs' comp are tied to that spend.

But when you cut a big chunk of the supply, the expectation is you're not going to hit the same spend, so... you cut your budget too. Well now that changes everything. Now you don't have to spend so much money over so much time. Imagine if you cut your budget in half, but because you are performance focused, you want to lose the worst performing half. You should expect better performance. Way better. Because now some other idiot "performance" marketer is going to buy up all those crummy bits of inventory, and you're going to pay the big money to outbid them for the bits of inventory they were going to get that would have performed well.

Lay down with dogs, get fleeced.

The adtech industry is a huge sham. It's an open secret that ads don't work and that most of the data is fraudulent. There are thousands of companies with deliberately complicated relationships to pull a giant shell game with advertiser's money. All of the data that supports the lie is built out of massive internet surveillance dragnets. Ultimately, even casting all of this in the best possible light, it's designed to psychologically manipulate people. No one wants to see an ad. Adtech is despicable.

amazing

Together with https://news.ycombinator.com/item?id=25623858 it might appear like there's a coordinated takedown of ad tech companies going on.

Google derives 80% of its revenue from ads, Facebook 99%. Imagine these giants coming down if it turns out that online ads aren't effective?

EDIT: Tweaked the wording regarding the alleged "coordinated takedown" as a reaction to a comment.

Good. Advertising is a terrible zero-sum industry that just distorts market activity. It has little place in capitalism or any other system.

Exceptions:

- New products deserve some advertising. I would handle that like the patent system. Demonstrate your product is sufficiently different, and get access to some attention economy time slice.

Some may say, well what about competition with existing products to bring down cost? Simple, get rid of stupid https://en.wikipedia.org/wiki/Monopolistic_competition and start selling things in unmarked tubs with a price tag. Cheapest one wins. Of course, there has to be quality control and avoiding regulatory capture with that, but stupid brands are not a good solution to that problem.

Centrally planned economy that also controls speech, based on the patent system. This seems like quite the recipe for success.

Start with https://en.wikipedia.org/wiki/Cidade_Limpa. I don't think this will slippery slope if well done:

- I choose the TV channel, website, etc. to visit, but public space I might have to walk through because it's the only option - In that public space it may be impossible not to see the adds - sight lines in public spaces are public property

Now, one may ask "what about paying someone to verbally advertise in public space". I think that should remain legal, because that does seem like a slippery slope. I'm willing to risk that society won't collapse so far that that highly inefficient form of advertising became cheap enough with destitute surplus labor.

What makes you say advertising is zero-sum? This doesn't seem to be the case at all to me.

Take diamonds for example. Prior to the mid 1900s, diamonds weren't nearly as sought after as they are currently. It took the marketing and advertising efforts (among other efforts, like monopolizing the mining process) of De Beers to convince the public that diamonds should be highly coveted. It was advertising that pushed the public to think spending X months' salary on a diamond ring should be normal. De Beers is obviously highly derided, but this a pretty clear cut case that advertising is NOT zero sum. The value of diamonds went up because of advertising, and nothing else depreciated in value as a result.

I won't even touch on your other points about unbranded goods, except to say that consumers definitely show preference for branded items, and I don't really see why that is a problem.

> De Beers is obviously highly derided, but this a pretty clear cut case that advertising is NOT zero sum

If you take into consideration the money wasted on overpriced stones, it's zero sum. How did people ever get engaged before De Beers?

That is not at all what zero sum means. If you're going to use a widely-used term at the core of an argument, you should be sure you're doing so correctly.

So, advertising created value here by convincing people to give their money for shiny stones? Instead of, you know, investing in health, education or a vacation? Overall, was it a net win? On the one hand, one company makes money, on the other hand, many people waste significant resources for a symbolic gesture.

> diamonds

I mean zero-sum in that advertising shifts spending from one area to another. Perhaps all the diamond ring money would have gone into other things.

It's a hard point to prove or disprove, but even if advertising overall deceases the consumer saving rate, sure the vast majority of advertising cancels out. (I'm going to buy tide or downy, but not both.)

> except to say that consumers definitely show preference for branded items

Well these meaningless competing brands convey no information and waste tons of resources. And it's rather an embarrassment to Capitalism for the primary non-price information exchanged by market actors to be bullshit noise, even if it isn't a "real problem" compared to global warming or something.

I don't think that's the case. If you browse HN somewhat regularly, you'll see that a lot of times when a post makes it to the top (Or close enough) it's very likely that a related post will make the rounds pretty soon.

I think it's more related to people trying to ride the karma wave than "A coordinated takedown of ad tech companies". And it is effective, because it happens way too often.

I agree, I remember reading an article or a comment explaining this phenomenon on HN, with examples.

Nevertheless it would be interesting if the idea of digital ads being ineffective snowballed to the point where it became like a self-fulfilling prophecy, and led to the demise of large companies that we think of as "giants" or "monopolies".

Or a coordinated "strike" by major ad spenders.

I'll bet that a lot of ad spend is zero sum. They'd probably rather share profits than boost google's.

Yes, this was just a standard follow-up [1] and we did the standard counter-follow-up, which is to merge the threads.

[1] https://hn.algolia.com/?query=follow-up%20by%3Adang&dateRang...

Hi dang, I'm wondering is there a list of HN phenomena that you've noticed over the years?

Examples would be:

1. follow-up submissions

2. an initial wave of critical comments followed by a wave of apologist comments

3. first page of comments occupied by a single top level comment with lots of descendants (you often post to draw attention to other pages of comments in these cases)

There's no systematic list but I use links to HN Search (like in the GP) to refer to such patterns. There must be a hundred such categories by now. Eventually I may try to compile them into something more standardized.

>it might appear like there's a coordinated takedown of ad tech companies going on.

Or, you know, somebody read the first article on HN, then found the other somewhere, and thought of posting it, as it relates to the same disucssion...

Happens all the time with posting clusters (and the Baader-Mainhoff phenomenon).

Privately knowledgeable parties (not me) have been talking about this for ages. A close friend quoted the same %age as the podcast underlying the tweet did as being fake.

By the way how impressive is the scammer level of sophistication. App level man-in—middle-esque (probably imprecise terminology) forging of attribution based on massive monitoring and siphoning of what otherwise would have been organic conversions. Super cool. And those were the ones they caught imagine what they didn’t.

No need for a conspiracy: one popular article sparks the submission of something similar, and given there was enough interest to make the first one popular, it's not uncommon for another to become popular.

I see lot of anxious people from the ad industry commenting on these threads trying to convince the rest of us we are wrong.

It's funny how quickly the word conspiracy is used by the party that has most to lose.

It's more likely that people have spend the past 10 years buying into the online ad industry, got burned and finding out it wasn't because they weren't spending enough it was that they were defrauded.

We should see class action lawsuits over the next 10 year horizon that will put a massive hit to the shady ad tech industry.

Millions of small to big businesses were defrauded out of their ad spend and now the industry will simply make way to a whole new scam.

I don't think there is a coordinated takedown; after all, ranting about it on HN wouldn't affect anything if advertising actually worked as designed and had the intended effect.

The problem is that first off, people rightfully dislike advertising so it's expected for them to rant about it, second, regulation is being worked on (whether the GDPR, CCPA, or similar) that would thwart targeted advertising, so advertising is now also facing pressure from the government and not just from people ranting about it, and third, these articles talk about brands themselves having doubts about the effectiveness of their advertising efforts and the media rightfully reports on it.

Ranting about ads by itself won't change much, but in this case it seems some advertisers themselves are having doubts as to how effective it really is and both the media and HN is reacting to that.

> Google derives 80% of its revenue from ads, Facebook 99%. Imagine these giants coming down if it turns out that online ads aren't effective?

Just because Google and Facebook are able to make a profit off their ads doesn't mean that their ads are actually good value and not overpriced, nor that the people in charge of assessing the ROI of these ads are incentivized to look into it (which is the point I'm arguing in the thread you linked). You can very well agree that online ads are effective but still aren't worth what they are currently priced at.

The issue is, most online ads don't work EXCEPT for G and FB. That's why they're so dominant.

The average third-party ad networks are junk, rife with fraud (ads that are never actually seen or clicked by a human), poor targeting, poor measurement, limited brand safety, and opaque billing.

First-party ad tech provides much better control over these factors and can generally be offered cheaper because they can more effectively allocate their clients' budgets. In what other industries are the superior products cheaper than the junk? (not rhetorical - I'm curious if this exists in other industries)

G and FB are the biggest first-party ad sellers, given that their total addressable audience covers most of the free world, with Amazon rapidly gaining ground. TikTok, Pinterest, and Snapchat are building clones of G/FB's ad tech by poaching a lot of their engineers (karma for G/FB cloning their consumer products...) and are also showing strong growth as first-party sellers.

If you're a website/app that monetizes through a third-party network (including Google Adsense and Facebook Audience Network), you're going to see your revenues plummet in the coming months.

> it might appear like there's a coordinated takedown of ad tech companies going on.

Oh please let it be so, even though it's just a wild ass guess. Ad "tech" has ruined the internet (only slightly hyperbolic). Oh to be able to recover the "innocence" of the late 90s internet. Maybe I should just stop browsing so I can spend more time yelling at the kids to get of my lawn

I am at a loss to this '90s innocent internet. It scared me into using adblockers for over 2 decades (which coincidentally makes the article and this whole thread completely alien to me).

You had to use an adblocker because advertisers went crazy with tracking and a lot of shady stuff.

If you've been on various internet marketing forums, you will see that this is just a group of people who weren't smart enough to land a wall street jobs but morally unhinged enough to defraud businesses.

I am already hearing about law firms sending out letters and emails to people who bought Google or Facebook ads in the past 10 years.

With the new USGOV administration hawkish towards social media/tech (essentially ad revenue dependent cmopanies) and now seeing class action lawsuit against ad tech companies, my popcorn is being prepared in the microwave.

People have no idea how lucrative ad click fraud is in the botnet market. It used to be that botnet operators DDOS for hire or install Dutch spyware toolbars, now with headless browsers and fingerprinting evasion it has become impossible to distinguish real ad impressions from fake ones.

Your only tool to fight these fraudsters are statistics but over the past 10 years the "bad guys" (I don't know who is more evil, the people defrauding businesses legally or illegally) finally figured it out. They know if they are too greedy they will be caught so they now play the long milk game.

It is incredibly lucrative. A single botnet victim can yield upwards of $2~4 in ad revenues from a variety of ad networks.

It is IMPOSSIBLE to distinguish real from fraud ad clicks/impressions since the introduction of headless browsers and fingerprint evasion

Somehow web scrapers/botnet operators have found the sweet spot. This is just like how the American Cosa Nostra working with Russian criminal groups were able to siphon off crude oil from the USGOV in the 80s.

I miss the 90s when everything was more simpler and secure as a result. You can't hack an HTML document.

People tend to upvote things related to what they've seen recently.

It is in fact much more likely that Google decreases the visibility of articles that in any way tarnish Alphabet's reputation.

Google is by far the most problematic. You are paying ridiculous amount of money for a click you don't even know came from a legitimate source.

There are many many botnets out there undetected running headless chromium browser instances that is indistinguishable from a real one.

So if you can't tell and then you will never know if you are paying for legitimate prospects or not.

However, advertising for established brands direct response isn't what they are going for rather brand awareness

Ads don't work

I've said for a long time that Martech is due for a correction due to lack of understanding in attribution. Could this finally be the bubble bursting?

Might be a possibility to disrupt the ad business. FB and Google might be paid tons of money for providing little value or maybe even toxic value.

Reply on news.ycombinator.com