Hacker News Reader

This is exactly why so much suspect ad inventory continues to be sold online. Back in the old days an advertiser could watch TV and say definitively if their ad played in the purchased slot or not.

The larger problem here is, depending on what is being optimized for - such as CPM, a big advertiser can begin to allocate an increasing chunk of their ad spend towards the worst performing inventory. I've noticed even Google has more and more very questionable ad products being sold, complete with nearly useless "conversions" being mixed together with real on-site conversions.

For all the heat Snapchat got in their IPO story, the sponsored face/AR filters are the best thing I've seen for brand advertising online in a really long time.

As someone whose clients are the companies you listed and a lead on data and insights, I disagree with a part of this.

While it is true that many agencies plan based off share-of-voice, this is simply done because of laziness, and more prevalent, the lack of clear data that matters (we have X dollars, we can reach X number of people, post-campaign, we reached X people, campaign was a success!). Clicks don't matter, I agree, but conversions do. Specifically sales. That is what is paying for the advertising. As ad-tech advances, planning for sales will replace planning for reach. Think of an upside down triangle where digital and audience targeting fill the smallest part. This will be the first place where ad investment will go because you are able to see the direct results on sales. After you saturate this bucket, you can start filling things like lookalikes, contextual, behavioral, to TV and OOH at the top.

How do you measure the brand lift? I ask because my company, Survata, runs brand lift studies for display ad buys of the size it sounds like your company would be running using retargeted surveys, but as there is no place to self serve dropping a view tag on Facebook unless it's from a whitelisted provider (which we'd like to become, but haven't been able to find the right contact at Facebook yet), so we can't run similar studies on Facebook ads at the time of this writing. Interested in how you're measuring brand lift from these ads, outside of reach metrics.

Agreed on your point. The total dollars spent by brand advertisers simply dwarfs direct response ads, and the industry standards for "success" often seem useless to direct response advertisers. Remember, these are brands who had been buying magazine and TV ads prior to the internet, so any quantifiable numbers is a major step forward for the industry.

Getting white-listed for view tags on FB usually requires a FB rep (certain spend and other variables). Curious, why does Survata run brand lift studies vs optimizing for lower cost per leads/sales? Since this is a digital business, I would imagine the data is there to measure this?

Sorry, I should have been clear. We don't run brand lift studies on Facebook. We run direct response and lead generation ads and optimize for conversions. We run brand lift studies for clients who spend millions on display advertising. As these advertisers run these ads to raise brand awareness, they're similar to an advertiser running Facebook ads to spread awareness in terms of goals.

And thanks, I knew about that, but without any kind of information about the spend level required to get a Facebook rep, it's hard to justify increasing spend for the purpose of getting our view tags whitelisted.

Ah apologies, I misread! Good luck, hopefully this article cracked the wall a bit at Facebook and you can get your tags whitelisted. I'm not sure spend would be a factor anymore- I would imagine they would want to see enough advertiser requests for your tags, where they will feel justified to dedicate engineers to vet your tags.

Thanks I'll look up your firm.

We typically these days prefer market mix models and the like for building definite guidelines on absolute money allocation.

We do run lift studies for operational comparisons and optimizations, but find these techniques unreliable and generally overestimates for any absolute conclusions. We do technically evaluate most new techniques that come to market.

You are using Facebook ads for exposure, the same way consumer companies have been treating TV. You are looking for mindshare. How is that working out so far? Are you able to measure exposure to impact on future purchases?

Of course there's no way to measure it. Rather, there was, and it was called clicks. When it turns out click numbers weren't as impressive as they should've been, now suddenly everyone is back to being comfortable with their ads being tantamount to digital billboards and pitching their service as "brand lift."

Actually for digital ads, there is a way to measure offline sales lift (albeit a few shortcomings). It requires a lot of money (i.e. ~50K for a study, and media spend 250K+). We are able to tie a digital impression to traceable tender (credit card, loyalty card, etc). There will be an exposed group and unexposed group who match each other audience-wise. Lift is then calculated and vetted for significance.

For more info, I recently did an interview on an ad-tech site about this: http://www.thalamus.co/blog/lenny-chase-interview/ Happy to talk more separately if interested.

Any insight into options for advertisers without that sort of budget to put towards just a single brand lift test?

Brand lift or Sales Lift? Brand lift is a bit different because there are a lot more companies offering this due to more simplicity. The shortcoming of Brand Lift is every vendor has their own methodology when it comes to deciding what exactly is a marginal lift in brand awareness. I think the easiest option for a lower budget is using a FB/IG brand awareness campaign and looking at ad recall numbers (https://www.facebook.com/business/help/1029827880390718). However, we don't really know how they calculate this number and it would only be useful to benchmark between campaigns (if you trust the number). The best option right now is to find a vendor that polls the user who saw the ad, like Vizu (Nielsen). Unfortunately, to get a stat. significant read, you probably need a lot of results, and thus would need higher media spends.

Both actually.

I don't trust the black box of the ad recall stats they provide, so that's not really an option. Is Nielsen's data any less of a black box for this sort of data? And is it actually a decent sample? The older way Nielsen did things was biased because people had to sign up to be a part of it.

And yes, the challenge is that to get a statistically significant read, we need to throw a lot into a specific test, and that is hard to justify without benchmarks. Kind of a chicken and egg problem if your brand is large enough where it takes considerable display dollars to make a blip against your other traffic.

I've been curious if things like Adobe's Attribution econometric modeling tools help much here, but guessing that lack of data will be an issue there as well.

I agree with you 100%. No options are ideal and statistically correct which sucks... a lot. As you mentioned there will always be selection bias and a black box, just hopefully less, with Nielsen for example. Not many people in our space even understand these problems exist, so I think you already lead the pack in this regard. For the sales lift studies, the methodology seems less "iffy" since its one to one and measurable results. One can argue there's selection bias due to no cash data. I haven't seen any cheaper options to do this while minimizing error also.

My client is in the same chicken-egg problem and the way we approached it is: Going on blind is worse than the risk of a bad study. It's completely up to us to do due diligence and ask the right questions to find holes in the vendor's methodology and minimize the risk. We felt that the data gained will be accurate directionally, albeit skewed.

I've never had first hand experience with Adobe's attribution models but from what I gather, lack of data and similar to media mix modeling.

IMO, advancements in this space will be battling privacy concerns.

Yeah, privacy concerns are what will decide a lot of this. I'm very torn because as an end-user I definitely have gotten more nervous with the current state of tracking, but as an advertiser, these are arguably the toughest problems in the industry and what keep me up at night.

Have you found any good off-the-shelf tools to help with the incremental lift analysis that might be better suited for advertisers who are not as massive? Our current plan is to get DCM in place, pipe everything we can through there, and then let our Data Science team go nuts at trying to model it. Seems like that's our best bet since tools like Adometry or other dedicated attributions platforms don't make sense for us yet.

Not operating anywhere near the scale of grandparent, but spending a decent amount on FB ads results in a noticeable lift in search volume. It's hard to back this into value but it's at least independent verification that your money isn't just being dumped in a hole.

On your first question, we don't really have a choice but to begin moving money from TV to Digital because TV viewership IS declining and people increasingly spend even their TV time fiddling with their phones :-)

Yes there are a bunch of regression modeling methods and some complex setup lift and attribution studies that let you do this. Some of the techniques are under experimentation. They are generally expensive.

> The primary business objective of advertising for companies such as mine is to build brand awareness, product benefit awareness and brand identity (eg. are you partial to Coke or Pepsi? Nike or Addidas? etc). Doing this will influence your purchase when you next shop for our categories (eg. your next shopping trip to Walmart or Amazon). Note that we are looking to influence a future shopping trip, any "digital conversion" we may get (eg. click thorough to our product page or an online retailer) really is pure gravy from our point of view.

How do you calculate ROI for the ad spend? How do you figure out if you targeted correctly? For instance, I'm highly unlikely to buy either nike or Adidas, so I'd be a poor target.

For absolute spend ROIs and the like, we typically prefer building market mix models these days though we keep experimenting with others.

On our scale, quality of targeting is something we trade off vs reach. The relationship is often non linear - say you get 10 million reach at an expensive CPM but very high accuracy vs. a 100 million reach that is cheap CPM but has half the accuracy, we'd might be better off putting money in the second bucket.

I don't operate at the levels you do. Which also means we are intensely concerned about wasting ad dollars. And, I have to say, Facebook is a total waste of money. Building brand awareness is extremely expensive. Most small to mid size advertisers need conversions. They need real people interacting with their site or product, signing-up, purchasing. Facebook, over the years, has proven to be a great place to set fire to a big pile of cash when it comes to this. Sure, it can work here and there, but, for the most part, nah, waste of time and money.

Any good sources to read and learn more about this form of advertising? Thanks

Go work at a company with a $100M+ online advertising budget :)

Sorry for the trite answer, but this kind of knowledge is so new and ever changing, and there is so little incentive for it to be documented externally, that if you're really interested in it, the best way is to become one of them.

I think this is rather old-school, actually. They are using FB in the same way they use TV and newspapers, etc. Positioning and maintaining status at top of mind in the consumers' consciousness.

I believe reading ad-tech blogs would be a good start as well as industry sites such AdAge, Digiday, and the blog on Thalamus.co. (I do not work for any of these sites, but have contributed thoughts)

>Hope this helps explain how big advertisers look at media.

And that is the real crux of the matter. FB is a media company. They are currently denying it in the wake of the political discussion (please don't go there) but the real reason is valuation of media vs. tech companies.

It sounds like you have an extremely vague and unreliable means of measuring a return on your hundreds of millions of dollars investment.