Works best in online direct to consumer brands.
Take an audience of X people. Divide them in two. Show ads to your test group, don't show to control. Watch your business grow and gauge the lift between the two audiences.
The companies that know how to advertise at scale do this constantly and can gauge the real effect of their ad dollars. Facebook, Google and others make these tests possible in their platforms, while other software suites such as Impact Altitude and VisualIQ allow you to do this kind of analysis and testing as well.
In the end, most of it proves out to be incremental. There are notable exceptions of course, but when are there not?
The flip side is that not every campaign, targeting strategy or ad will be incremental. You have to test and retest your way to it. A blanket assumption won’t work.
The fact that Web and AdTech allow the appearance of individual-level targeting does not necessarily make that a Good Thing.
Which is the wrong experiment to run. The old adage is that 50% of advertising works, you just don't know which part. You need an experiment to prove that ad spending scales with ROI, as ad agencies claim. That may well be completely false.
Incrementality testing isn’t limited to your advertising as a whole - you can get down to very fine detail as long as your experiment is correctly designed and you wait for stat-sig results.
Specify a set of zones. Target different levels of advertising at these. Look at sales trends. Rinse, wash, repeat.
Note that major corporations often designate specific areas for product testing. The Chicago area, and Australia, see trial runs of McDonalds products and store concepts, as an example.
The fundamental concept of testing advertising effectiveness is not new. Here's an account from 1909 (from one of my faviourite sources on advertising and media, Hamilton Holt's Commercialism and Journalism):
https://archive.org/details/commercialismjou00holtuoft/page/...
There is a limited audience that can be influenced. Just like house ads on a site (where you advertise to sell your own goods on your own site), there's only so much that is affected because you have finite traffic. There is a curve for every permutation of context (demographic, geographic, etc).
It is something that could work for retargeting existing users but doesn't work for acquiring new visitors.
You define an audience. You split it. You test and measure the result.
What component of that strategy doesn't work for new visitors? Many eyeball vendors (Google, Facebook, heck traditional TV and other media!) are set up precisely to allow for such cases.
One tool with FB lift tests is to make a lookalike audience. FB will then split that audience in two for you so you have a test and control group.
The original test required two ad accounts and keeping your overall advertising static (not changing anything) for the duration. Today it is much easier and has a lot more features.
Google does a search lift test with YouTube as well. You can run a large set of ads against an audience in YT and google will either suppress them for a subset. Then you can see your overall organic lift based on your video ads.
YouTube brand lift is an indirect measure that is hard to translate to actual sales to see if the ads were cost effective.
The math is fuzzy when measuring marketing ads and it is real hard to be truly scientific.
See, e.g., ADVT5440 and ADVT5501 at Webster University, for example:
This course emphasizes the importance of critical thinking in the planning and development of message strategy for advertising and other marketing communications tools. Class discussions explore the decision making process and development of criteria for evaluation of alternative message strategies.
http://www.webster.edu/catalog/current/graduate-catalog/cour...
Or in related textbooks, e.g.,
So advertising industries use people as guinea pigs to study the "real effect of their ad dollars". No ethics committee, no informed consent, nothing.
Yet another reason to block all ads.
Internet companies run A/B tests all the time, and it's not controversial.
It is important for a brand to have their own attribution platform that measures incrementality as well as the relative contribution each ad exposure makes toward a given sale.
Both of these things are possible to measure. The problem is that the band of users that display positive ROI (when incrementality is measured) is so small that people don't believe the data.
Additionally, brand exposure is hard to measure and almost completely ignored in digital advertising. Over time I imagine that there will be more research in this area (beyond just measuring an in-view ad impression)
Bob Hoffmann, very influential figure in advertising (http://adcontrarian.blogspot.com/) likes to point out that there were not that many companies which built their strong brand with online advertising only.
OK, he is old and grumpy dude and he dislikes online ads, but this is a fair point. Once company becomes big enough, it strives to buy a Super Bowl ad spot. And we are well into 10 years of internet being mainstream media, but how many brands were built online only? Not that much, and I'm saying it as "online ads work", kind of guy.
Now big company marketers are used to thinking of US as a homogeneous market for efficiency reasons and so TV still works in US if your target audience claims a broad demographic. However, they're beginning to learn that you'll simply not reach enough consumers if your target demographic is say young adults in the West coast if you do a TV heavy plan. US in general lags China in media landscape shift but the media behavior of the more valuable demographics are actually much closer to China than averages tell you.
I might be at a grocery store and I need to purchase something like dishwashing liquid. If I'm staring at two brands on the shelf priced identically (or near enough) I'm more likely to purchase the brand I recognize over a brand I've never heard of. How do I recognize the brand - from advertising.
That is not something you can measure with tracking metrics embedded into a banner ad. I'm sure there must be studies on impact this has.
The other thing I suspect online advertising works very well for is service based businesses, the kinds of things you used to look up in a phonebook. The last time I needed to find a plumber I typed my suburb name + plumber into search engine I imagine a lot of people do similar.
Can you name one?
I find it very hard to believe there are any companies that have scaled up only using online adverts. Why would the founders of a business do that when they can grow in their local market by talking to people as well as buying online ads for wider reach?