Facebook managers trash their own ad targeting in unsealed remarks

When your predicate influences the next version's antecedent, it's easy to show a big ROI, lift, or whatever other jargon you might care to use.

FTR: https://en.wikipedia.org/wiki/Dragon_NaturallySpeaking

Dragon Systems released NaturallySpeaking 1.0 as their first continuous dictation product in 1997. <...> As of 2012 LG Smart TVs include voice recognition feature powered by the same speech engine as Dragon NaturallySpeaking.

I’m not saying there hasn’t been innovation in the past 5 years — absolutely there has.

In that industry, 2013-2016 were peak “ML” and “Cloud” where the CIOs at my company and competitors were fully bought into the ML and cloud hype, how it was going to solve all kinds of problems, without understanding what those problems were, and without realizing the complexity of getting meaningful, applicable data for those problems.

In the past few years, it feels like more people realized that ML is less mathematical magic and more of different “kinds” of curve fitting.

In five years we will without a doubt have systems that play more games, solve more puzzles, fold more proteins and so on, but we'd have more progress on intelligence if we could build a machine that could reliably plumb a toilet or catch a mouse as well as a cat.

Voice recognition is neat, and it is extremely useful in certain niches, but it is generally far inferior to mouse+keyboard or touch or dedicated buttons in all cases where those are practical. It is almost unusable in public spaces. Text-to-speech is mostly in the same boat - extremely useful in some niches, not a game-changer in any way I expect.

Text generation is only "realistic" in shape, but for now remains a novelty. Picture generation is the same. They are at best at the level where it already costs pennies to obtain similar texts through human labor. Having conversations in a meaningful way is even farther.

Playing Go better than humans is a fun gimmick - an awesome achievement in some sense, but with 0 practical applications outside of Go.

Overall, there has been good progress in AI/ML, but I don't expect we'll see any major changes too soon from this. Especially since the idea of actually seeing commercially available self-driving cars has been pushed back a decade or three compared to the initial optimism.

I'd also add that most of the progress in ML/AI in the last few years has essentially been of a technological/engineering kind. There haven't really been that many deep scientific insights, we don't necessarily understand any of these problem spaces better. We've just been able to match certain neural net architectures with certain problem spaces, and massively advanced in realistically-available computing power to actually train these nets on enough data.

But we haven't gotten any new insights into stuff like language from GPT-3, for example. We haven't even gotten any new insights into Go from AlphaGo!

In that case, non-targeted advertising gives you a <1% chance of reaching your audience per impression. FB is much better.

I'd be curious to see how blogs do at targeting certain demographics. Presumably if you had a blog aimed at high-earning college educated people you could get a pretty high ratio of (target demographic/all viewers).

But I assume advertisers generally trust metrics more than intuition so you need something like Facebook or Google's ad network to identify someone as belonging to a certain demographic before the advertiser will believe it.

FB advertisers are paying for results and they will pay, or not, for those results, not really the targetting.

If ABC corp knows for sure their target is top 50% Households, and FB can only roughly provide that, then the value will work itself out in the numbers.

If FB can provide that target more precisely, those ads will convert better becoming even more valuable.

It's easy to see how the statistical problems compound.

FB does not have exact earning levels, so you have to infer that from, say, likes. Let's say you can build a model salary(likeBMW, likeTravel, ...) = %likeBMW + %likeTravel +...

This gives you an estimate which is (70-80)% accurate, so you predict >£250k/yr 70pc of the time. In c. 25% you mispredict.

Now it seems to me that this 25% is going to compound across several categories: when you say "College AND HighEarner AND ..." you get more than 50% of your target group not matching this exact criteria (all you need to fail is one condition to fail to match this conjunction).

And according to FB comms, it looks like >50% didn't match client's chosen criteria.

I think this is the right analysis of the issue. ML systems of this kind are very bad at making targeted judgements. It's really little more accurate than guessing the mean of something (eg., salary) for your group.

All ML has to do, for FB/Google/etc. is improve targeting a few percent to have a significant value proposition.

However, the propaganda has it that ML systems can "target" you, etc. Only in the way a nuclear bomb "targets" a house.

Talking about (inadvertent) misquotes: the quote was attributed to a "February 2016 internal memorandum sent from an unnamed Facebook manager to Andrew Bosworth, a Zuckerberg confidant and powerful company executive who oversaw ad efforts at the time [...]". (Italic emphasis is mine)

Our general rule is if the product doesn't have broad appeal, you don't run it on FB.

My running theory on all ad networks is pretty simple. There are a very small subset of users, 25%-30%, of people who are regular purchasers and these companies know that based on conversion data. They generally just throw your ad in front of these people and let it ride.

While sure it sounds like you were ripped-off, I would expect that just about any mass-market medium offering ad-targeting to "senior-level engineers at technology companies", "senior managers", "wealthy yachting enthusiasts" or similar super-high-value clients be a ripped-off if it was offered. I'd imagine anyone in the position would avoid anything that could target them with ads (blocking ads, going incognito, avoiding social media, etc)because otherwise they'd be continuously barraged by companies.

The theory behind micro-targeting was that your audience's conversion rate varies considerably by certain key targeting attributes. And, if your audience has varying conversion rates, your CPC bids ought to reflect this.

For example, let's assume your LTV is $100. And, let's assume your average conversion rate is 1%, but actually, males convert at 0.5% and females convert at 1.5%. If you do NOT split your audience by gender, you will be forced to bid $1 per click for every user. However, if you split your audience by gender, you can bid $0.5 for men and $1.5 for females. By splicing your audience, you gain considerable efficiency. The theory behind micro-targeting was to find the permutations of targeting attributes that split your audience into segments with the most variance in conversion rates.

However, what's happened over the past few years is that Facebook's AI has become vastly better at doing micro-targeting than even the best individual marketers. This is partly because they have more data than platform participants, but also because they have much smarter engineers. As a result, it's now become better to hand over targeting responsibilities to Facebook's AI. For the most part, you just tell them what you want the average CPA to be, and they do the targeting for you behind the scenes.

I started out with low budgets too, and I can see how it would be easy to blame the platform when the real problem is poor campaign design and creative. Don't get me wrong, big advertisers have a huge advantage bc they can iterate on creative and campaign design faster. And, this advantage often explains why most small advertisers fail. But, at the actual auction level, I've never seen any evidence of favoritism nor do I think it would make sense from Facebook's perspective.

Surely FB employees aren't managing those bot accounts themselves. Do they generally determine accounts that engage with ads to be "human/real", thereby incentivizing bad actors who spin up fake FB accounts to click/"like" ads?

I've done some FB advertising for news type content (much easier to target properly) and was getting incredible value for the money. Usually it's pretty obvious real fast if you're getting real action or not and you can adjust accordingly.