Ad Blocking Wars: Ad Blockers vs. Ad-Tech (2015)
whitehatsec.com
whitehatsec.com
If your website doesn't render for a significant number of people, it will get delisted at Google. Then, your website vanishes from the earth along with any ad revenue.
Google will NOT allow web pages that don't render to clog the first page of search results as a competitor would quickly take advantage of "Google recommends broken web pages".
Ad Tech has lost already. The issue is that many people have a lot of money to lose if the people paying for ads realize that they're being suckered. So, they keep playing Weekend at Bernie's with the zombified corpse of Ad Tech in the hope that the people writing checks won't notice.
Donations don't do it because nobody donates (tragedy of the commons). And most content creators are terrible at marketing and have no idea how to create their own products and sales funnels (and would be hard-pressed to expertly monetize them if they did).
We're left with ads. Without those, the web goes back to being abandoned by professional content creators, and instead becomes the domain of Tumblr/YouTube/Imgur, the new Usenet/Angelfire/Geocities.
The solution I suspect is most likely if ad blocking becomes too ubiquitous and publishing companies are finding it too hard to make money online is that publishing companies all get behind a locked (non-plug-in-able) browser that users cannot block the ads from. If you want to surf the New York Times or Wall Street Journal or the Economist or read the links on HN, Reddit, or Google News, you fire up your Media Browser and click the links and suck up the ads as a necessary evil. Click those links in a non-approved browser and you see a message saying, "Please install Media Browser to view this article." If you don't feel like looking at ads, you stick to your regular browser and surf Tumblr. But almost everyone will install Media Browser and watch the ads.
That said, I don't think the arms race ever ends. There will always be people who need to find a way to get paid to create content, and there will always be people who don't want to pay. That race goes on forever.
Hobbyist blogs, on the other hand, typically have great content, because the people writing it created it out of a desire to share their knowledge, not to drive ad clicks. There can be some overlap, and I've definitely seen what I would consider good content with ads on the side, but I largely believe that content created to drive ad revenue is a business which pollutes the web. Tumblr, GeoCities, etc had / have a ton of awesome content, and it's not typically the creators who are getting paid to put it there. YouTube does share ad revenue above a certain threshold, and I think that works pretty well.
Direct advertising still works. When the ads are sourced from the same domain as the content, blocking them is essentially impossible. This means a bit more work on the part of content providers, they have to solicit advertisement revenue directly, but many of those companies have been doing that for decades (newspapers). Shouldn't be too difficult.
That sort of advertising is less offensive as well, because it's more likely to be relevant and safe.
User always downloads everything to a proxy browser (maybe in the cloud), renders everything, runs all JavaScript, and lets it send back tracking info etc. But the User doesn't see that.
What happens next is basically AI-powered filtering: all ads and tracking and such is filtered out using classification (something like how spam filters work today, i.e. machine learning). The filtered version is what gets sent to the User.
This isn't a new idea. Amazon's Silk browser/system [1] is in the same ballpark. No doubt there are others.
The ironic aspect of this approach is that some of the leaders in classification (machine learning) are Google and Facebook, the two companies profiting the most from ads.
* http://adnauseam.io/ clicks on the ads but doesn't display them
* https://cs.nyu.edu/trackmenot/ runs random web searches to obfuscate yours
troymc already said it: Use a proxy browser. Let the ads and tracking do their work inside a sandboxed virtual system. Software outside the sandbox analyses and filters the experience then relays it to the user.
GAME OVER.
This reminds me of embedded secrets in a gadget. They are ultimately hackable to a determined enough possessor. The long series of iOS jailbreaks show that. At an extreme the hacker measures single electrons flowing in a die to extract the secret. The rule is: Don't give gadgets with embedded secrets. They will be eventually found out if the interest is high enough.
So what can Ad Tech then do?
Use malware to break out of the sandbox. However this is so blatantly illegal that it's not practicable for respectable companies.
GAME OVER.
It amazes me the degree that publishers will go to make sure I see things I don't want to see. Perhaps if they worked more on creating things I wanted to see, they wouldn't have to work so hard at forcing me to see things that I don't want to.
It's fine for the people who aren't on metered connections or have ever more powerful computers to keep up with the pace of the web - but in some respects, the ad people have won in making it terribly inconvenient.
Example Yahoo Mail. Let's say they block web access to the emails if you don't load their ads. You can still read them on your phone, in Thunderbird or mutt, and you can still choose to switch to GMail. Emails from Yahoo Mail can also be redirected to GMail so you don't even have to tell people that you are using GMail now. Then Yahoo Mail loses their user base and dies. The empty space will very likely be filled by an email start-up with an idea how to make money without spamming ads. Life goes on.
So, dear ad-blockers. I love you. You allow me to fight for my right to not be spammed. But if the service provider decides to not serve me for that wish of mine, then don't stop them. Don't take away their rights about their service. Just focus on fighting for the users' rights. Thanks. Maybe if you detect that this is happening you can provide an in-html guide how to switch to alternatives.
They don't seem to understand the lifecycle of page rendering. They assume some strange situation where the server maintains state forever and thus has permanent access to the DOM. That's hardly how the web works. Once the page renders it's game over for any kind of ad-tech. We own the DOM. Period.
More and more people find online ads to be annoying, invasive,
dangerous, insulting, distracting, expensive, and just
understandable, and have decided to install an ad blocker.
If you look at it closely, the Ad-Tech industry behaves quite
similarly to the malware industry.
I didn't know that because I use an ad blocker :-)
Congratulations, you now have a reputation as a poorly done site which doesn't work in a lot of browsers for reasons seemingly unknown.
And then there's the effects on SEO, or heck, various thousands of services that need to get some content from the sites mentioned to work properly.
The game here isn't technical, it's financial. Advertising (and advertising countermeasures to ad-blocking) are based on revenue potential. Advertisers are creating ever-more complicated systems (technical debt) for serving ads users increasingly don't want to see.
And don't forget what the ads are there for: to provide revenues to publishers.
I've already been compiling lists of domains I block for having screwed up the payload (e.g., content) badly enough that I'm not willing to look at that. If you've turned me off the content, your ads aren't getting through, regardless. Even without other adblocking.
We've been watching a 7 or 8 year advertising-based, easy-money-fueled, opportunistic, liquidity-extracting Internet bubble. Some companies have gotten big. It's based on a $500 billion annual global advertising market, of which $100 billion is online. And of which the lion's share is itself FIRE industries: finance, insurance, and real estate, all of which feed off the same fundamental monetary policy that's been fueling tech.
What's not money-chasing-money is gadgets-and-apps-chasing-money-chasing-money (e.g., Samsung and Apple, etc., advertising their latest shinay). Or Amazon (Google's largest current single advertiser, ironically).
I'm quite disappointed that a company calling themselves "whitehat sec" is so blatantly stupid. Much of this is already solved using reek's anti-adblock killer, which they're apparently unaware of. Fuck, if it comes down to it I'm sure we can render the DOM entirely and use some machine learning to recognize ads.
I've been building a web app lately and making an effort to avoid technology which tracks users and to be honest, it's not easy. This is largely the point of my my post, to those not in building the web, tracking is really prolific.
I understand tracking data is important data for some companies; however, it really is like having someone follow you around all day and write down everything you're doing, often without consensus or informing the person being tracked. The average person just has no idea about how the technology works.
I guess most users have never really been presented with an option to stop being tracked, until now. Given how invasive tracking has become, maybe it's really not such a bad thing?
The war is between service providers and their customers. And there's only like 3 steps. Step 3 is customers get fed up with being treated so poorly and they stop using the service. The service goes bankrupt and vanishes. Services which find ways to monetize without relying entirely on advertising revenue survive, and the problem vanishes in the long run.
I think there's an interesting psychological discussion to be had about the way people are misinterpreting this situation.
EDIT: Another perspective is this is a war between companies like doubleclick.net and their eventual and well-deserved extinction.
* Class or ID names. Even if obscured, they're different.
* Banner sizes. Standardisation of sales blocks means dimensions to latch onto.
* URLs. Stuff that links back through tracking systems is blockable.
* If nothing else, AI training for "does this look like an ad" and nuking it.
Everyone will be authed everywhere and advertisers will sync with vendors.
That's game over. It is very amusing how uninformed commenters are in all of these threads. Claiming advertising dead.
Or disable javascript - which I did in my other browser after one too many sites popped up with "you need to disable your ad block on our site because we are fucking special".
Blocking ads would be achieved by modifying the contents of Web pages after they leave the browser but before they're displayed on screen. Modifying data/video content after the browser has processed it means that it can be effectively isolated from the underlying code. As DOM and JavaScript will not be able to detect changes, there's huge scope for changing the way web pages are displayed without content providers being any the wiser of the fact.
Moreover, isolating changes from the browser would also permit false/random/junk data to be fed back to the browser without detection (i.e.: a dummy user seemingly responding to the ads in a random or nefarious way). Clearly, widespread deployment of such useless data would render ad statistics useless.
Let's examine some of technologies already available to Ad Blockers and users to block ads albeit if for the moment they are not being used for that purpose. They could be used individually or together or even in combination with sophisticated programs that use kernel-based drivers to further thwart browser-originating ads, threats, etc.
1. Use Secure Screen technology similar to that used by PGP decades ago to secure the viewing of email messages. Here the on-screen message is bit mapped into obfuscation to all players except the reader who views the message in the form of an image rather than it being generated from actual ASCII or Unicode characters.
2. Similarly, webpage data intercepted from the browser can be monitored using AI techniques which learn to recognize ads then block them automatically. Precision would be improved if done in conjunction with the Ad Blocker's remote blacklist servers. (Remember, your smartphone's facial recognition software is good enough now to recognize your face, thus similar code ought to be smart enough to recognize ads and to learn about changes in their content, deployment, placement, etc.)
(It seems to me that such a back-end AI scheme would be ideal to thwart the recent changes to Facebook where ads are being integrated into the content structure so as to fool ad blockers. Whilst the new ads will be indistinguishable from content to traditional ad blockers AI will recognize them easily.)
3. These techniques could or would be used in conjunction with the video Overlay mode as used by most graphics cards for movies. (You may recall that when the vector/overlay mode is used for movies then essentially moving images are isolated from the rest of the screen (effectively they cannot be seen by other programs). It's the reason why Print Screen when used to capture still frames of movie scenes end up as blank images, this isolation could be used to the Ad Blocker's advantage.
To thwart or bypass ad-removing code running externally to an effectively 'sandboxed' browser would require the complicit involvement of both the O/S suppliers and hardware manufacturers. If this were ever to happen then the ad wars will have entered a new dimension altogether, then we'll have entered an Orwellian world where ad viewing was compulsory through legislation.
[No matter how website/content providers escalate the war to force users to view ads, ultimately users will always be able to gain the upper hand with sufficient effort—after all they are in control what they see. Content providers can and do make things inconvenient but technology will always win out in favor of users (and it gets easier over time). For example HDMI was introduced to stop recording of 2k HD images but as 4k video becomes ubiquitous so do 4k telecine [camera] video recordings of screens, thus now 2k HDMI content can be recorded with almost no degradation: almost by definition the Law of Diminishing Returns ultimately favors users.]
BTW, it doesn't require much extension of thought to envisage similar techniques being used to remove TV ads. Here, TVs/PVRs would remove ads on-the-fly from recordings made for time-shifted viewings/replays: one allows for ads to be removed by delaying the time one sits down to an evening's viewing by the collective duration of all ads. Using AI would not only effectively remove all ads but also it could eliminate other annoyances such as the incessant news-breaks so frequently inserted into the middle of programs.