Secret Media wants to solve adblocking issues for publishers | Hacker News Reader

Secret Media wants to solve adblocking issues for publishers

secretmedia.com

22 pointsvmorgulis42 comments

From their white paper:

  The technology used by Secret Media makes 
  sure that each ad gets a specific URL that cannot 
  be nor found not added to EasyList by the community.

I don't quite understand why they think this would work. Surely ad-blockers can filter by domain? Are they using well camouflaged URLs from legit domains? Or do they have a never-ending supply of throwaway domain names, in order to stay one step ahead of EasyList?

Either way, I'm genuinely curious to see what they have in mind...

Bittorrent style per-to-peer botnet, where each ad includes code that downloads ads from their server and redistributes them, so ads are coming from readers' IP addresses?

Would subdomains do the trick, say, very-very-long-cryptokey.publisher.tld ?

(note: I'm hesitant to post this; any site that actually did this is a site I'm never visiting again)

The way to get around adblocking is very long crypto tokens, but not in the subdomain. All that is needed is a front-end proxy that takes each session[1] and rewrites all href/src addresses to point to the proxy. This means all URLs in the page are of the form

    https;//example.com/proxy/<crypto-token>
    # or in the no-cookie case
    https://example.com/proxy/<crypto-token>/<session-id>

Rewriting client-side generated URLs is an exercise left for the relevant Javascript framework, but only requires the addition of a simple API in the proxy to convert URLs, or some sort of bypass/whitelist mechanism.

The tokens used by the proxy can either be the cyphertext of the actual URL or a synthetic token that references the real URL stored in a DB in the proxy. Such details can are left to the implementation of the proxy.

The point is that you have to send the crypto token back to the proxy to get either 1) a redirect of the real URL, 2) the actual content served by the site in question (either from the proxy directly or as a tunnel, or 3) the advertisement/whatever, with the prox6y acting as a live proxy to real ad server, with all the stupid tracking information passed along as extra HTTP headers (in the style of "X-Forwarded-For"). The client only ever sees URLs from the single domain, each obfuscated into a crypto token. No URL would give up any distinguishing characteristic an adblocker can use as a filter.

The only costs are the cost of running the proxy, and a bit of latency on each GET request because of the extra hop through the proxy.

It might be possible to find heuristics to block in the client's DOM, which is why I have expressed concern in the past[2] about the people who will use WebAssembly and a <canvas> tag to bypass the DOM. These two techniques in combination will make adblocking nearly impossible without first either breaking crypto or solving the halting problem.

[1] As defined in the usual manner, either as a cookie or embedded in the URLs the proxy generates.

[2] https://news.ycombinator.com/item?id=10211050

But can't an adblocker just add:

    example.com/proxy/*

to its list of requests to block?

(of course, the URLs could be camouflaged to look like real ones... eg:

    http://example.com/a-very-legit-looking-article

ugh.)

Sure, you can block all of these URLS. This means you block jQuery, all other Javascript, all CSS, all images, the user-clickable URLs on the page, etc.

The point was that all URLs get encrypted/replaced at the proxy. If you load the page and block all of them, you get not only a page that doesn't work, you also lose access to all other pages on the site.

This is an extremely rude way to run a website; it completely betrays your contempt for and mistrust of the client. Given that quite a few websites already demonstrate this with their malicious and purposfully-misleading ads, I expect they won't think twice about capturing all URLs with a proxy (or similar technique).

Incidentally, this type of method could be extended very easily to prevent all deep linking. Also note that this isn't theoretical. I saw this done in... 1998/1999. It had problems and was expensive to run on the server, but I suspect Moore's law has easily had enough time to solve that problem.

If both legit and ad urls all look the same, and can only be distinguished by decrypting with a key, then adblockers may be unable to differentiate between the two.

Your can differentiate with a list of object SHA hashes you blacklist based on ad blocker user feedback. You'll still need to fetch the object, but you can dump it before rendering.

Excellent point - you could monitor the ABP database and if the hash appears, modify the content (shifting the value slightly on a single pixel) so the thieves need to block the new one.

I assume by thieves you mean ad networks, because I never agreed to retrieve their content, let alone view it.

Its an arms race, as always. And just as the media industry couldn't beat piracy, ad networks aren't going to beat blockers, even if it means content producers get their content stripped and distributed via other channels.

Ok so what are the search implications here? How does this impact Google indexing? And if it is done in such a way that Google can make a distinction, how far behind would a great adblocker be?

I don't see how it would impact Google at all. Does Google change their behavior if you include

Adding a mechanism to bypass URL-encryption for actual (non-ad) external links would be easy, if someone cares about traditional PageRank-style references.

Other than that, the client (possibly GoogleBot) still gets the same page content; it's only the URLs that change. I have no idea (and really don't care) if URL changes affect the "search engine optimization" games some people like to play. I'm describing a way people could actually defeat adblocking; if that method is annoying to use because of side effects, that's not my concern.

I doubt publishers would go for this, as it allows ad hosting websites direct control over ad traffic, and is subject to manipulation.

Are you using "publisher" to mean anything other than the author of the website? I'm not sure. It seems like you're suggesting that the ad network is the publisher, and the actual website is just "hosting", but I may be parsing your comment incorrectly.

As for any concerns that advertisers not trusting the publications they want to do business with, that's their problem. They can easily verify that a website's url-encrypting-proxy isn't modifying their ads with random anonymous checks. Contracts that require the publisher to pass through the ads unmodified (and with the appropriate HTTP headers or any other technical detail) would be the obvious next step.

There are several other obvious ways for advertisers to gain power over the actual publishers of a website, but I'm not really interested in enumerating ways to keep ads on the internet.

You wouldnt have to tamper with existing data comming from site visitors, you could just simply generate fake visits. That would be much much harder to check for. You could of course do that with the current model, but it requires a lot more resources (distributed IP addreses to come from being the largest issue).

Since this is specificly targeted at getting around adblocking, the ad networks wouldnt be able to rely on cookies etc because the same types of visitors will often block/clear them.

Thank you, much clearer now.

No, because then we can build whitelist-based ad blockers (e.g. RequestPolicy, NoScript).

Thanks! Pardon my ignorance but what if the content url itself came encoded in the same manner?

They'll lose some of their SEO juice because their URLs are no longer human-friendly. Hashing everything would be more costly than letting some people block ads.

You could have human friendly URLs for articles and use hashed URLs for image and video content.

They would have to hack into the websites we already trust to serve as a proxy for their ads to bypass a strict whitelist.

All of their papers linked to describe what addblocking is, and why it is wrong. None describe the technology they use, or how they "monotize addblocked traffic".

The closest I could find is that it is based on a polymorphic encryption algorithm, which does not inspire confidence in me. Combined with the fact that they claim to support all addblockers, and addformats, and I am calling snakeoil.

They also say it is patented, so I will see if I can dig up said patent.

EDIT: I did a search on the patent office for "adblocking AND polymorphic" (with variations of adblocking), and found no results.

EDIT2: Also, no information on how to buy their "product" beyond a generic contact us.

I also couldn't find any U.S. patents (or even just published applications) listing any of these people[1] as inventors. Nor any U.S. patent assignments to a company called "Secret Media."

Maybe they're referring to some other jurisdiction, or maybe the patent has some other inventor(s) and their entity name is something besides Secret Media. Or maybe they're full of shit.

[1] http://www.secretmedia.com/about.php

Our website contains as less information as possible on our technology, on purpose. A patent is generally public 2 years after the application. You'll have to wait at least 9 more months :) We only work with premium publishers, and we'll probably never have a SAAS solution.

We are hiring...

Malware droppers use polymorphic encryption to avoid detection ;)

http://www.secretmedia.com/manifest.php

TL;DR "If you visit a web page, you surrender the right to decide what happens to your computer."

Me thinks they will regret this line.

Well they didn't actually say that line, it's just paraphrasing.

It's more a strawman.

And yet there's not a single 'solved' advertisement anywhere on their website to demonstrate their 'technology'.

But hey, at least they let Adblock Plus users read their site ;)

Edit: Upon reflection, I get that Secret Media wants to reach potential customers, whether they use adblockers or not. Indeed, given how much adblockers improve browsing experience, maybe Secret Media and others in the anti-adblocking business also use them. Why should I expect some sort of moral stance?

Right? I shudder to think what kind of cat and mouse game they'll be trying to play with uBlock et al. JS URLs will get banned, iframes will get hidden, so on and so forth. You can't monetize ad-blocking traffic, because the ad-blockers just get better; you have to block the users who use them if you want to keep making money.

It's telling how they don't discuss at all the other things that drive people to use ad blockers. The high CPU utilization that comes with the myriad of javascript analytic links, the annoyance of having a video or audio ad play upon loading a page, or the zero days and other malicious media that have been spread through advertising networks. When ad networks state that they are working to mitigate the "potential security risks inherent in the online ad ecosystem", instead of vetting all ads, people are going to continue to block ads.

> Ad blocking breaks this harmony.

I have yet to find anyone who talks about "harmony" who actually spews anything other than concentrated bullshit. Example: pretty much everything the Chinese government says and does in the name of preserving "harmony". It may have been a useful philosophical concept in the past, but it's been misused so much by shysters lately that the mere appearance of the "H" word is an instant red flag to me.

EasyList might not be able to block their clever tricks, but isn't it about time we came up with a better way to detect anything the user doesn't want? EFF's Privacy Badger uses heuristics to determine whether a given script is trying to track the user. Perhaps adblockers need to take a similar approach. For example, they could check whether an element that is displayed by default on page load obscures the primary textual content of the page, or compare the current page with one that was shown to a well-known search engine.

"Lets's force people who have taken steps to avoid adverts to watch adverts for our products against their will.

That's bound to make them feel good about it and make them buy our products."

Advertising is going to have to move from being served directly from the web, to devices where users no longer have control, to survive. Your phone, tablet and smart tvs will do the work of inserting ads into the web and transmitting analytics, the way some ISPs do now. People will put up with it because the ads will subsidize the cost of the devices.

Whatever an advertiser thinks my eyeballs are worth to them, they're worth more to me. They're going to have to do better than subsidising the cost of my device.

Sure, but plenty of people might be willing to pay almost nothing for a new generation iPhone if it means having to deal with advertising. I'm sure the manufacturers will let you opt out for a reasonable monthly fee, though.

This is not just about whether websites deserve to be able to get ad impressions.

Will their ads not chew up my CPU and battery life?

Will their ads not track my online behaviour in addition to showing me an ad? Will their ads not load other ads that will do the same?

Because otherwise, fuck you secret media, and fuck your ads.

I won't ever hire anyone who has this company on their resume.

Having the windows firewall icon, "anti virus" and some sort of anti-malware logo in what appears to be their hitlist shows quite well where their priorities lie.

/me clicks link, sees blank page

You're right! The back button does work as an effective solution!

Reply on news.ycombinator.com