Adblockers Performance Study
whotracks.me
whotracks.me
One of the main pain points raised was the lack of any way to dynamically add rules, as well as the low maximum number of rules allowed (30k). Seems they've decided to support dynamic rule addition, as well as increasing the number of rules, though probably not by orders of magnitude by the sound of it.
Proof is in the pudding though.
[1]: https://groups.google.com/a/chromium.org/forum/#!topic/chrom...
[2]: https://groups.google.com/a/chromium.org/forum/#!topic/chrom...
[3]: https://groups.google.com/a/chromium.org/forum/#!topic/chrom...
To me this reaction from the Chromium devs is missing one of the most fundamental issues. I'm not fundamentally against the declarative API because of technical limitations; I am against it because it is a strong innovation lock. The current extension ecosystem is flexible enough to allow hundreds (maybe thousands) of people to actively work on privacy-enhancing extensions (ad-blockers, anti-tracking, etc.) and the technologies, heuristics, solutions to protect users' privacy on the Web are constantly evolving. The APIs are not used today the same way they were used 2 years ago. If Chrome decides to "freeze" the blocking capabilities of the browser into a declarative API that no one but Chrome devs can improve, they will be preventing people from finding new solutions to tracking and advertising (at least from extensions). It does not matter if they replicate 100% of the capabilities of today's ad-blockers, as long as it does not allow evolution and adaption it will become obsolete. There is precedent in this matter: Safari also has a similar API and it has been a huge pain for ad-blockers developers. The reason is simple: Apple or Google do not have the same strong intensives that we have to continuously improve the blocking capabilities of the user agent. My fear is that this declarative API will be an ok-replacement for today's content blockers, but will not allow the same kind of fast paced development we benefit from today in the space of privacy extensions.
* The blocking engine operated by either Safari or Chrome is a black-box and independent devs will have a harder time understanding it, tweaking it, improving it, debugging it.
* Chrome devs are now playing nicely and get feedback and propose some improvements to the APIs but there is no warranty this will happen again, or that they will invest time/energy in the future improving this part of the browser.
* It's harder to work with this API than a JavaScript code-base you control.
* Chrome seems a bit better here but for Safari the documentation is pretty poor.
* You also don't get feedback regarding the rules which matched on a page and this makes it harder to debug or give nice insights to users.
That's only a few points from my personal experience but I discussed multiple times with developers of other privacy-enhancing extensions/apps and we shared similar feelings.
I think this is especially true. It is somewhat similar to many other Google products like Maps and Translate. They start as a good free product, but as soon as they gain enough traction the rules change. I think once this declarative Api is the standard for ad blockers in browsers Google will start exercising its control over it for its own benefit.
I mean after they essentially blocked ways to use proxy to filter the content, next logical step is to restrict API.
(Disclosure: I work for Google, though not on Chrome)
The API change is totally unnecessary, yet is happening despite many protests.
The concern is that it was performance and privacy issue, which looks like a total BS (even according to the link we are discussing).
The extensions are installed by the user, so what not let them decide what to do with their browser? If it's really a concern, I don't think anyone would oppose if google would educate user what API given extension is using.
I mean, both engines are open-source, but yeah, I do agree that it would be nice it have this enshrined in a web standard rather than a de-facto one driven by the shins of two large corporations.
> You also don't get feedback regarding the rules which matched on a page and this makes it harder to debug or give nice insights to users.
This seems like an easy tooling problem to fix.
Currently both sides adapt, if the way it works is locked down ad blockers quickly will become obsolete.
For example a whole ago most sites were creating popups with ads, after it became bad, browsers started blocking popups, initially by only displaying then when user actively clicks. So sites started opening a popup when user made a first click anywhere on the page, eventually browsers started blocking all popups and just notifying user that popup was triggered, giving them choice whether they want to see it.
This solved the old pop-ups, but because of that a new popups were created, that use CSS to show it within browser window covering the text. In addition to that, the CSS layers were used to implement other attention grabbing mechanisms, like ad that stays in place even when you scroll, or suddenly ad appears between text etc.
This was targeted by ad blockers, which constantly adapt. Most ads are served from different domains, since typically the ad content is provided by different company, but increasingly we see ads being served from the same domain as rest of the website, or website randomizes CSS component id etc.
What chromium authors are doing is to instead provide API for ad blockers to use and list their rules and let the browser do the blocking. Supposedly that's to improve performance. The problem with it is that it will essentially fix adblockers in one place, they no longer will be able to adapt, eventually new kind of ads will start to show up, that adblockers won't be able to block. And now with other changes that Google successfully pushed, such as https everywhere, http/2 and http/3 is nearly impossible to block ads through a proxy.
This is why I personally prefer to stick with a Firefox.
I agree with everything except this. First, http/2 and http/3 absolutely do not prevent blocking. If blocking proxies don't support them, then they're the ones lagging behind.
Secondly, most blocking software working on the network layer use DNS, which still works just fine and will likely continue to work forever.
Thirdly, you can still, for the most part, MITM https connection on devices you own. You just need to install your own root ssl certificate. The only thing that prevents this from working would be HSTS preloading.
EDIT: Actually, adding your own root cert bypasses HSTS preloading.
Don't say that twice. Have you heard about DNS over HTTPS? I'm using XPrivacy on Android and have noticed that applications that use Android System WebView (based on Chrome) started making requests to 8.8.8.8, 1.1.1.1 and other public DNS services. It's still possible to block domains via hosts file, but I bet it's a matter of time when Google decides it's "in our interest" to start using their DNS instead of ISP's one.
The DNS part was already handled by someone else.
As for the MITM https you are at the mercy of the browser, they are depreciating the API they might restrict this as well if it will become the way to do filtering.
This doesn't just knock the power of your adblocker down from Type-0 to Type-4 of the chomsky hierarchy, it also limits the inputs it can act on.
As an example, if I wanted an adblocker that looks at the DOM or javascript state of a page before allowing a request to load an iframe this would have to happen asynchronously since it means communicating with the page context. You can't do this declarative style.
Or if one wanted to implement a "click to play" style tool for iframes one would have to hold the request indefinitely until the user approves. This probably isn't a good idea for technical reasons, but at least it is a possibility with current APIs.
I believe this is the reason Safari introduced content blockers. It fits in very well with the traditional computing model on iOS of preventing unbounded, arbitrary computing where possible.
And even if we suppose that it is a reasonable policy for a moment, it's still not all that relevant since we're not talking about the apple ecosystem here in the first place.
Google tried to roll this out initially without that obvious must have. It speaks to intent, and probably future prospects for the API.
And, of course, a declaritive API with pattern matching limits what you can do anyway. No heuristics, no behavior based blocking, etc. You are pretty much stuck with cataloging the patterns of a ton of websites as your only approach.
[1]: it's probably not plain old memory leaks anymore, but due to using a few long-lived content processes, pages/scripts that leak is an issue. But usually not a huge deal, once one of the processes start using 2-3GB I just kill it, and refresh the affected tabs (coughSlackcough).
I also lack the trust that even if they fixed the major issues (or even allowed fixing them yourself) I would be able to rely on things working in the mid-to-long-term.
Currently holed up on Vivaldi, where the things you expect out of the box, are in fact working out of the box (vertical tabs, mouse gestures, ...).
Ok, so then why
> In particular, there are currently no planned changes to the observational capabilities of webRequest
That's before the fact that you consider that webextensions can run arbitrary code on the page and extract whatever information they want.
Edit: Also
> Increased Ruleset Size: We will raise the rule limit from the draft 30K value. However, an upper limit is still necessary to ensure performance for users.
But, as per the article
> All content-blockers except DuckDuckGo have sub-millisecond median decision time per request.
So that doesn't make any sense as a justification either
Beyond the ability to block requests conditionally based on arbitrary logic (not just a few pre-decided qualifications like request size), one point I want to keep coming back to is that there are actually legitimate reasons why an extension might choose to slow down requests. I use Tampermonkey scripts on a couple of social networking sites deliberately to slow them down so I'll that I'll be less likely to impulsively refresh them.
I continue to believe that the manifest changes aren't written with the perspective of enabling creative, unseen uses of the API in the future. They're written from the perspective of, "let's decide up-front what extensions we want, and enable specifically them."
The feedback people have given on this is extremely broad, and is mostly ignored by this post. It's disappointing to see a response that at least somewhat suggests the Chrome team is dead set on shipping this, and is only willing to bend so far as it takes for them to enable the most popular adblockers that exist today. If it took that much feedback to get Chrome to even slightly tweak the design, then what possible feedback can people give going forward to make anything more significant happen?
Could you use Chrome's simulated network throttling instead?
I don't see an API anywhere that makes network throttling available to extensions, but let's assume Chrome adds one.
In that case, it still lacks granularity -- I only want to slow down some requests on some sites. One thing I've been thinking about doing if I turn this into its own extension is having it respond to your aggregate time on a site. So the more time you spend on a social site, the slower it gets, but if it's been closed for a while it starts to "recharge" and speed up again. In particular I'm thinking about that for sites like Twitter, where I don't mind checking it so much as browsing it.
There's a lot of interesting stuff that's possible with the current API that can't be replicated by just saying, "slow down everything across the board."
I find it telling that it was left out in the first place. I can't think of any plausible reason to have omitted it, other than to purposely hobble adblockers.
I wouldn't really call promoting their own product on their own webpage an ad.
Of course we've had our own issues with WebExtensions but since it's not politically motivated these issues will hopefully be resolved.
Enjoy the good, ignore the bad.
Beyond all that, I’m a huge fan of ad blockers and use them as an attempt to reduce tracking and targeting and it’s abundantly clear to me that they greatly speed up many webpages. The amount of junk that so many sites load, no doubt it even saves me data on my data plan.
It seems like common sense to not trust a company whose main income is based on advertising to be making decisions on ad blocking.
We would be really happy to see people running the same benchmarks and try to reproduce the results!
From a user opinion,it may not ready for everyday using.
There's a few issues with the conclusion here. First, they article measures and discusses only the time required to block a single request. Modern web pages are issuing many, many more requests than that, like the 35 that this page issues. At median timings, that would put the DuckDuckGo blocker at almost 300ms, well within what humans can notice.
The second is that this API is not used solely by the popular content blocking extensions, but by a variety of other extensions. The Chome team's performance concerns likely stem from the fact that a user won't be able to differentiate the browser slowing down and an errant extension slowing down the network requests, and there are examples of extensions that use this API to issue additional network requests or do slow things down. If you cherry-pick the good citizens of this API to show that performance isn't a problem in general, you're not showing that performance can't or shouldn't be the reason, just that it isn't the reason for the fast good citizens. What this data could be used to argue is that imposing strict deadlines on the execution time of these extensions would allow the content blockers that the community cares about to continue to function as they do today while also placing a performance cap on bad extensions.
> There's a few issues with the conclusion here. First, they article measures and discusses only the time required to block a single request. Modern web pages are issuing many, many more requests than that, like the 35 that this page issues. At median timings, that would put the DuckDuckGo blocker at almost 300ms, well within what humans can notice.
That is true, but on the other hand the DuckDuckGo blocker is the exception here and they could likely improve this performance if it becomes their focus (one way would be to use one of the faster open-source alternatives). If you consider uBlock Origin, Adblock Plus or Ghostery, we see that even blocking 100 requests would not take much time (probably around 1 ms with Ghostery).
> The second is that this API is not used solely by the popular content blocking extensions, but by a variety of other extensions. The Chome team's performance concerns likely stem from the fact that a user won't be able to differentiate the browser slowing down and an errant extension slowing down the network requests, and there are examples of extensions that use this API to issue additional network requests or do slow things down. If you cherry-pick the good citizens of this API to show that performance isn't a problem in general, you're not showing that performance can't or shouldn't be the reason, just that it isn't the reason for the fast good citizens. What this data could be used to argue is that imposing strict deadlines on the execution time of these extensions would allow the content blockers that the community cares about to continue to function as they do today while also placing a performance cap on bad extensions.
There are indeed examples of extensions doing bad things: collecting private data, etc. But we are talking about diminishing the potential privacy protection of all users to prevent some abuse. On the other hand, the manifest v3 will not prevent extensions from being slow, or doing bad things. Extensions will still be able to use content-scripts, inject arbitrary content in pages or send any private data home. WebRequest listeners will also still be accessible (only not in blocking mode) which can allow any data collection.
So yes, I think these changes could in theory prevent some cases of abuse, but I strongly believe that they will overall weaken the privacy protection of users and that this is not an acceptable trade-offs.
I can choose to not install extensions that do these things, while using ones that only have a manifest list…
Arguing this correctly is extremely important if the community wants to change Google's mind, and the miscellaneous fear-mongering and accusations of bullshit motivations that are flying around this and other threads can be easily dismissed if we aren't more careful about how we conclude analyses like these.
Not commenting on if the proposed V3 changes are a good idea or not but aren't the changes not supposed to prevent badly written extensions from slowing everything down?
It might be the case these adblockers are well written but you can't guarantee that for all extensions. If a user doesn't have an obvious way to know it's the fault of an extension then Chrome gets the blame.
By the way, I have a Chrome extension that from its own tab requests many pages from arbitrary domains to examine their content. I need to modify the request headers going out and observe the response headers coming in, but only for requests made from the extension (so it's not impacting other tabs at all). I'm guessing V3 impacts this only if the request header modifications are dynamic and can't be added via the fetch API?
It has been suggested in another comment that Chrome could give visual indications of the performance of extensions. They already do some of this to track the memory used.
I'd like to see a comparison against the WebAssembly build of uBlock Origin though. uBlock Origin uses WebAssembly in Firefox to speed up some functions. I'm not sure if Chrome allows add-ons to use WebAssembly yet.
1. Matching the $domain option using an optimized Trie data-structure
2. Parsing domains using the public suffix list, also based on a Trie data-structure
I would love someone well-acquainted with the uBlock Origin code-base to update the benchmark so that we can compare.
Edit: All the code to create the dataset (as well as the dataset used for the study itself) and to run the benchmark and analyze the results (create the plots, etc.) is available on the repository and should be reasonably easy to run locally.
I want to repeat here that this _does not_ show that the performance argument is bullshit. What we can say that it shows is that existing content blockers whose functionality the community is concerned about losing are largely performant enough that performance isn't a concern for that subset of extensions.
We can in turn conclude that there may be other ways of solving the general performance concern that the existing API poses by other clever means, such as limiting execution time and disabling extensions that violate those deadlines, or UX changes that allow the user to more readily distinguish which extensions are causing performance issues.
Continuing to push the narrative that the performance concerns are bullshit and a pretext for disabling ad-blockers is hurting the cause of allowing the existing APIs to continue to exist.
There are two ways to interpret Google's claims about performance:
A) Content blocking in general is impossible to do quickly without a declarative API.
B) The current API gives bad extensions too much power to slow down pages.
We already know that argument B is wrong, because the changes Chrome is proposing don't prevent extensions from slowing down the page. If Chrome was completely, 100% deprecating the old API, I think B would be a much, much stronger claim. But they're not.
So that leaves argument A, which is exactly what this article attacks. It is possible to build adblockers with the current APIs that are good enough that performance doesn't matter.
Now, if the Chrome team wants to argue that extensions have too much power and malicious actors can do bad things, that's a legitimate claim to make that I honestly kind of agree with. But that's so clearly not what the point of these changes are, because a set of changes that were focused on that would look very different from what we've gotten. It's the same reason why I don't take Google's privacy claims seriously for the manifest -- because they haven't actually deprecated any of the worst features that allow people to spy on me.
If someone claims that they need to buy a new car to help save the environment, and then they show you an ad for a pickup truck, I think it's reasonably safe to assume the original claim was just an excuse.
There is an argument to be made about motives with free vs non-free software, but open vs closed source is for most people much less important.
However, the parent comment is a bit misplaced since Ghostery appears to be released under the Mozilla Public License 2.0.
https://github.com/ghostery/ghostery-extension/blob/master/L...
(Disclaimer: I work for a non-Google Alphabet company.)
I don't think it's valid to debunk this claim without testing the speed of manifest/content blocker list-based blocking.
I would love to hear your thoughts about this. From our point of view, the argument is not that the native content blocker is slow or fast but instead that it's not an acceptable trade-off, for different reasons (see link below). We also show that some of the most popular content-blockers are more than fast enough in this regard. I would love Chrome to propose this declarative API as an addition to the current WebRequest APIs and incentivize developers to use it when it makes sense. But replacing the current WebRequest's blocking capability with this would prevent extensions from protecting users as efficiently as today.
I wrote a bit more about other reasons I think the declarative API as only blocking capability of the browser is not a good thing IMO: https://news.ycombinator.com/item?id=19175265
Both of these arguments are easily rebutted and have already been in this thread. As others have pointed out, the modified API still allows extensions to do things which cause performance issues, just not in that particular path. (Also, preventing ad load can improve page load performance so much that even a "slow" adblocker may make up the difference anyway.)
> If you cherry pick performant examples
I don't think these examples are cherry-picked; they're among the most popular adblockers in the landscape:
* https://www.tomsguide.com/us/pictures-story/565-best-adblock...
* https://www.digitaltrends.com/web/best-ad-blockers-for-chrom...
As others have pointed out, you can measure and break or shame poorly performing blockers without punishing the ones that work well. So:
> and there's a long tail of extensions that use this API and do create human noticeable delays.
There's a long tail of extensions that are poorly behaved in general. You can punish those ones without killing the ones that don't suck.
Maybe these results are only applicable to the desktop version of Brave?
Brave is intentionally slow on parsing and do as much work there because it doesn't parse from client code, it only use already parsed lists from memory.
"The memory usage of Brave could not be evaluated using the devtools and thus is not included in this section." That doesn't make sense, I wonder if it's maybe using a very old version based on the old muon code base? If you can get the memory from Chrome you can get it from Brave.
No information was given about versions that were tested.
Total parsed rules is too small.
Thank you for taking the time to read this study. We do not think we claimed anything that was false in this study (although the scope might not be as wide as some would expect or desire); this is not a reason to be dismissive. We have ourselves a lot of respect for the work done at Brave.
> Brave is intentionally slow on parsing and do as much work there because it doesn't parse from client code, it only use already parsed lists from memory.
That was indeed one of the things measured, but not the most important one. In fact we explicitly say that this is a one time operation and does not necessarily matter, especially if as you suggest you can perform this work backend-side and ship the serialized version to clients. What is more interesting is the time it takes for matching requests.
> "The memory usage of Brave could not be evaluated using the devtools and thus is not included in this section." That doesn't make sense, I wonder if it's maybe using a very old version based on the old muon code base? If you can get the memory from Chrome you can get it from Brave.
If we got this thing wrong we would be very happy to update the results with the correct measurements. The version we used was the latest version from `master` on the following repository: https://github.com/brave/ad-block
> No information was given about versions that were tested.
This is indeed unfortunate and we will be correcting this. The measurements were performed last week with the latest version of each projects but we should definitely indicate the exact version used.
> Total parsed rules is too small.
Too small for what exactly? Easylist is one of the most popular lists and it's pretty common to use it as a base-line for comparison. It is trivial to re-run the experiment with different lists given that all the code is open-source.
I may be incorrect, but I think their testing methodology involved loading a JavaScript version of the Brave adblocker and ran benchmarks inside the Node.js runtime, and indeed the results may be different if they had used the Brave browser itself written in C++.
Ads.
As far as I know, ublock Origin has no profit motive whatsoever. Correct me if I'm wrong.
Btw. DNS based filtering works for non tech-savvy users like my parents who would certainly fall for malicious ads which are frequent even on Google. And guess how ransom-wares spread.
https://www.zdnet.com/article/skype-served-up-malware-throug...
- It does not give you fine grained permission: Like, if you only want to allow a particular third-party on a specific site.
While DNS based blocking is fast but at the same time it is also difficult for an average user to surf the web anonymously using the same.
With the increasing adoption of DoH it will be interesting to see how DNS based blocking happens for TV Box etc.
I have not checked but was mentioned that YouTube does not show ads while watching videos via chromecast. It will be interesting to check how that happens.
However that doesn't solve the problem of the web player sucking. To solve that, your best option is mpv/youtube-dl.
They should test on average hardware and not on the top one. Take a 2013 year Celeron or Atom with 2 Gb of RAM and HDD and test on it.
Regarding privacy, I think the code that blocks network requests could run in an isolated environment so that it cannot send the information about requests outside.
Even things as simple as build times, I've seen vary by 20+%.
For the study, measurements were run with one of our personal laptops (an X1 Carbon from 2016 with an i7 U6600 CPU and 16 GB of Ram, which is indeed a pretty powerful machine). We tried very hard to limit the impact of frequency throttling due to limited thermal dissipation of the device for the long-running benchmarks. In fact, for the measurements we put the laptop outside at 0 degrees Celsius, and we could observe that the CPU temperature did not go beyond 60 degrees (which is pretty low).
Do you have any suggestions on how we could improve this setup? We welcome all contributions.
Source: https://github.com/betafish-inc/adblock-releases#intro and https://en.wikipedia.org/wiki/AdBlock
Look at that sub history..
https://news.ycombinator.com/submitted?id=kkm
They should be shamed. I'll make sure to tell anyone I see mention them to stay clear.
Article provides evidence that Google's pretext of performance doesn't hold water.