Hacker News Reader

Comment by thinkcomp | Hacker News Reader

Not creating fake accounts. Allowing them.

Imagine you're Coca-Cola. You have an on-line ad budget for this quarter. Where should you spend it? Probably the places that have the most reach: Google and Facebook. Do you know how many cans of Coke or other drink brands you will sell based on these ads this quarter? No. Is there any way for you to figure it out? No. All you know are the numbers Facebook tells you. And those numbers look pretty good. This is how "sophisticated" ad buyers work.

It's actually far easier for a tiny business to measure the conversation rate between clicks and actual purchases. Big ad buyers throw stuff at the wall to see what sticks, and they rarely bother truly measuring because that's often impossible.

The entire ad industry is based on fuzzy metrics. Within that fuzz is an awful lot of room for fraud.

View on HN

Is this how you think ads work? This is horribly naive. If an ad exec at a large corporation thinks this way they should be fired immediately.

To answer directly : of course you can measure how effective your FB ads are. Pick two markets with similar demographics and a timeframe. Run FB ads in one market as an experiment and traditional media ads in another market as a control. If sales increase in the experiment market vs the control that is strong evidence that FB ads are more effective than traditional marketing. For higher confidence repeat the test with multiple markets and multiple controls.

That experiment took me 30 seconds to think of. A sophisticated marketer can probably come up with one that gives even more solid evidence.

>Pick two markets with similar demographics and a timeframe. Run FB ads in one market as an experiment and traditional media ads in another market as a control. If sales increase in the experiment market vs the control that is strong evidence that FB ads are more effective than traditional marketing.

This sounds like an incredibly naive and expensive experiment, filled to the brim with noise.

If you have historical sales data and advertising data by geogrpahy, you can control.

It's not perfect, but if you do it frequently you can estimate (some of) the impact.

Yeah, I also worked in ads and your experiment for a brand of any reasonable size is effectively impossible. You will have a challenge:

- Finding a time where there isn't some global event (e.g. oscars, olympics, etc)

- Weather effects (it was hot/cold so they went out/didn't go out)

- Local effects (Warriors just won the championship)

- SUPER local effects (it was prom at this high school)

- Brand history (Coke does better in ATL than Pepsi)

- etc.

And then, even if you did, ACTUALLY being able to measure impact on anything but ultra-transactional things (e.g. buying a movie ticket) is super hard/expensive too. You're going to run a survey after each experiment? And THEN, even if you DID run a survey, maybe it worked for one brand and one brand only! Maybe it doesn't even cross over to other brands in a category, let alone other categories.

Look - scientifically speaking, sure, you should be able to do what you describe. Almost none of the time does that work in the real world.

Are you saying a good ad manager shouldn’t try to do measurements though? That seems like a cop out. Why would you be making the big bucks if you can’t control for all this noise?

Ad campaigns are super expensive. If I were an exec at a company managing a campaign I’d want to see solid metrics and impact on sales, otherwise what’s all this money for?

In other words, if you can’t tell the difference between spending the money and not, don’t spend it.

Yeah, it's very very difficult for brands which are bought offline.

That's why Google are working with mastercard to get hash credit card transactions.

Additionally, Amazon are probably in a really good place to do marketing for CPG brands, as people actually purchase those on Amazon and thus they can track conversions (whereas FB and Google aren't great for these kinds of advertisers).

> Big ad buyers throw stuff at the wall to see what sticks, and they rarely bother truly measuring because that's often impossible.

> The entire ad industry is based on fuzzy metrics. Within that fuzz is an awful lot of room for fraud.

Citation needed.

Do you have any hard data besides anecdotes and contrived examples?

Reply on news.ycombinator.com