Web Request and Declarative Net Request: Explaining the impact on Extensions
blog.chromium.org
blog.chromium.org
Oh, heck off. This is Google's fallback whenever people call them out on anything -- "you just don't understand the entire picture." It's just their way of gaslighting critics.
People understand the change perfectly.
a) The privacy arguments are untrue because the Observational capabilities of the WebRequest API aren't being deprecated. There are no privacy benefits unless an extension voluntarily ignores that part of the API -- which no one will do because, as Google says:
> when extension developers are given the choice between capability and security, the vast majority of developers choose capability.
Google makes the argument that 42% of malicious extensions use this API. Well, guess what they'll still be able to do under Manifest V3?
b) The vast majority of data suggests that ad blocker performance is not a serious issue on the web right now. Ad blockers pretty much universally help more than they hurt, so it's not clear why performance is such a big concern.
c) The declarative model is a less powerful alternative, which is the exact reason why Google is deprecating the blocking capabilities in the first place -- if they didn't force it, no one would switch (again, see their "why not both?" argument).
So what don't consumers and developers understand about this? It's an objectively less powerful API that does very little to improve privacy for the sake of performance improvements that aren't necessary.
They are being severely curtailed.
https://twitter.com/justinschuh/status/1138889508512866304
> Yes, but network requests won't be observable without permission on the relevant hosts. And host permissions are changing significantly in Mv3 (e.g. they can't be granted as blanket install time permissions).
> The underlying issue with blocking Web Request is that it is effectively a UXSS, which could not be bound by origin-based host permissions like the rest of the API can.
Google's privacy argument is correct.
On the current discussion thread, Chrome devs say:
> The webRequest API's ability to observe requests is foundational for extensions that modify their behavior based on the patterns they observe at runtime.
And in this article, they say:
> when extension developers are given the choice between capability and security, the vast majority of developers choose capability.
So the argument that hiding observation requests behind a permission means that they are "severely curtailed" is just plain wrong. Most extensions (including ad blockers) will still request these permissions as wildcards across all hosts, and most users will still grant them as wildcards across all hosts.
Per-domain permissions for the blocking WebRequest API would be great (that's a change I would broadly support), and if Chrome devs honestly believed that per-domain permissions were an adequate user protection against malware, they'd look for ways to implement per-domain permissions with the blocking API, rather than deprecating the entire thing.
The Twitter post you link claims that's impossible, I don't see why. That would be a good thing for someone from Chrome to explain in either the massive developer thread that's been going on for months, or maybe in the article that complains about how much "confusion" users and developers have, instead of asking devs to blindly take their word.
> The Twitter post you link claims that's impossible, I don't see why.
Your original post claimed that people understand this issue "perfectly." It turns out there was more to it than you thought.
With respect, a lot of us have been following these changes for a long time. We know about the permission stuff already. And to be clear, the permission changes are good -- they're just not going to significantly change normal user behavior in most cases.
> It turns out there was more to it than you thought.
It turns out that a Google advocate on Twitter posted, "we can't do it the other way" with no explanation of why it would be a problem. Again, this exactly is how Google gaslights. They refuse to engage with the wider developer community, ignore proposals and objections, and then when the wider community finally gets mad enough to start complaining in mainstream channels, they start posting on Twitter and blogs that "actually, it's complicated."
If developers don't have the full picture, give it to them. The Chrome team can't complain that developers aren't informed about the technical limitations they're facing when they're posting arcane updates to Twitter instead of the official threads that everyone is following.
[0]: https://docs.google.com/document/d/1nPu6Wy4LWR66EFLeYInl3Nzz...
> The underlying issue with blocking Web Request is that it is effectively a UXSS, which could not be bound by origin-based host permissions like the rest of the API can.
The reasoning here should be obvious. Observing web requests can be useful with the cooperation/permission of the host you're observing, but it makes no sense to have an ad blocker that requires permission from the ad-blocked websites.
If you grant the core premise is that we don't want extensions to be allowed to observe all web traffic, because this permission will be abused, then the only logical conclusion is to use a rules engine. That's the only way to design an ad blocker that doesn't/can't observe all web traffic.
I don't follow what you mean by this.
Under the current V3 specification, as currently proposed, Privacy Badger (assuming they stick it out and keep maintaining a Chrome version) will need to be granted observation permissions on every single website, adblocked or not. The way Privacy Badger works is by determining at runtime based on request observation which domains it should blacklist. This is not going to be a fringe usecase.
Are you misunderstanding how the proposed permissions system works by some chance, or am I misunderstanding your argument? It's not the website you request permissions from -- it's the user you request permissions from for domains/domain patterns.
Why doesn't it make sense for a user to grant an ad blocker permission to operate on domains where they want it to block ads?
>Since the original announcement of the Declarative Net Request API, we have added significant functionality to the API as a result of these discussions. The Declarative Net Request API now allows for the registration and removal of dynamic rules - specified at runtime rather than statically in the manifest. We’ve also added the capability to remove common tracking headers, such as Referer, Cookie, and Set-Cookie.
>We are actively exploring other ways to expand this API, including adding methods to get feedback about matched rules, and support for richer redirects leveraging URL manipulation and regular expressions. Additionally, we are currently planning to change the rule limit from maximum of 30k rules per extension to a global maximum of 150k rules.
They posted a simpler post to the Security blog also addressing these issues.
https://security.googleblog.com/2019/06/improving-security-a...
They lead with the point that most-seemed to get lost in translation, even on technical sites like HN.
>No, Chrome isn’t killing ad blockers
>>No, Chrome isn’t killing ad blockers
They’re just neutering them with a ridiculously low limit of rules.
One of the benefits of doing this declaratively is that they can optimize the shit out of these blocking declarations. There is no need for such a low limit. Or even a limit at all.
However I do agree that there shouldn't be a limit, in accordance with https://en.wikipedia.org/wiki/Zero_one_infinity_rule. I think they were just trying to get people to actually clean up EasyList to prune dead rules.
I just checked and my uBlock Origin -- which doesn't have any social filter lists applied -- has 148,339 rules applied in total.
I read 42K from a source earlier today. That's where I got my figure.
https://www.ghacks.net/2019/01/22/chrome-extension-manifest-...
Might it be different for different extensions?
150,000 is definitely better than 30,000, but it's probably not a future-proof number.
EDIT: Actually, given Apple's 50,000 filter limit, I wonder if your source was looking at a version of Easylist that was trimmed down specifically for Safari.
whose ads are getting blocked as a priority? doubleclick is going to be in there so is there really a difference to them either way.
Declarative rules engines are better and preserve privacy better. This is Google following Apple's lead and doing the right thing.
> Yes, but network requests won't be observable without permission on the relevant hosts. And host permissions are changing significantly in Mv3 (e.g. they can't be granted as blanket install time permissions).
https://twitter.com/justinschuh/status/1138889508512866304
there's also some stuff in the design doc: https://docs.google.com/document/d/1nPu6Wy4LWR66EFLeYInl3Nzz...
Which explains the specific hosts part, but only makes vague mention of how extensions that users do want to run on every site will get permission to do so.
If this is part of a longer-term plan to deprecate webRequest and completely remove that ability, then they should say so now.
uBlock Origin has over 10M installs and the author has stated publicly that they cannot effectively do what they do if this change goes ahead. I can guarantee that uBlock Origin does more to protect my privacy, security and performance than this change. How many people have been affected by malicious extensions? I seriously doubt it's greater than 10M. So to improve the privacy, security and performance for a few, they are going to make it worse for the many.
Surely there are better solutions to tackle this problem? Maintain the API but add in additional checks/vetting for any extensions that use it.
> Last year, hackers phished Chris Pederick, who runs Chrome Web Developer, an extension with over one million users. Once they had grabbed Pederick’s developer login details, they swapped his extension on the Chrome Web Store with their own malicious version, designed to inject adverts into users’ browsers. Then in September, someone loaded the popular Chrome extension for file sharing service MEGA with code that could steal login details for Amazon, GitHub, Microsoft, and Google accounts, as well as pinch cryptocurrency keys. And a malicious browser extension appears to be behind a recent dump of private Facebook messages. Once hackers gain control over the extension, through hacking, coercion, or otherwise, they can replace code as they see fit.
There's a lot more where that came from.
Aside from hacking the extensions, a couple of years ago there were severe issues with people buying control of popular extensions and then converting them into malware.
> with their own malicious version, designed to inject adverts into users’ browsers
Your very first example wouldn't be prevented by these changes.
> 42% of malicious extensions use the Web Request API
So 58% don't event need the Web Request API to do something malicious. So these changes don't really improve safety at all.
> hacking the extensions, buying control of popular extensions
Both these scenarios need to be addressed at a different level. Things like enforcing 2-factor for high value extension authors (> 100K installs or something). Also remember Google operate a walled garden here. No extension can be published on the store without being vetted. They should be identifying high profile/value extensions and subjecting them to additional checks.
For me, and I'm sure many people, my biggest threat vector is ads. They threaten my security (malicious/malware laden ads), my privacy (tracking), and performance (slow, bloated ads). My ad blocker protects me every day from this threat vector. Google are now definitively weakening mine and a lot of other peoples protection in order to improve the security of the few. To me this does not seem like a reasonable compromise, especially when there are alternative ways to address their concerns.
And I'm sorry, but I really don't trust the motivations of a company that makes billions of dollars a year off online advertising, that has publicly stated that Ad Blocking is a threat to their business model, to then be making a change that just so happens to cripple Ad Blockers.
With the new changes to the spec outlined in this post, this may not be true anymore. Rules can now be updated with an extension update, network headers can be blocked, and the number of rules has been increased.
I don't know exactly what other features may be needed, but I imagine this helps close the gap. We'd have to hear from gorhill to know more.
https://github.com/uBlockOrigin/uBlock-issues/issues/338#iss...
> I don't know exactly what other features may be needed
I know this sounds like a dismissive, catch-22 answer, but the big thing missing is being able to run arbitrary code to decide whether or not a request should be blocked. And that is something the Chrome team has been pretty clear that they are not going to compromise on.
Right now, people are compiling lists of different filter capabilities they want, but (at best) that is only going to close the gap some of the way to what ad blockers are currently doing. Ultimately this puts Chrome in the position where every time a developer comes up with an interesting strategy for blocking ads or filtering/redirecting web traffic, they'll need to go to Google and ask them for permission to do it.
The fundamental functionality block is the thing Google wants to block -- arbitrary, developer-defined logic.
This is not nearly as complicated of a situation as Google is making it out to be. Google is arguing that developers should not have the ability to arbitrarily block requests, and developers (particularly ad block developers) are arguing that they should. Gorhill's position is that matching algorithms are fundamentally not powerful enough to support a healthy ad blocking ecosystem.
“API Changes WebRequest: Restrict the blocking capabilities of the webRequest API.”
Cannot they just instrument the code that intercept the requests and tell the user when performances are affected? Something like "It seems that extension xyz is slowing down Chrome. Do you want to disable it? Yes No [] Do not ask anymore"
In addition, because of the Chrome extension team's obsession with using js workers instead of persistent processes we're going to have to deal with finding work-arounds for holding important and unencrypted things in memory. Someone brought up the issue that some extensions will have to create a newly pinned tab to the top left of the browser because of these inane bullshit rules removing the background page. Imagine a stack of pinned tabs just so your extensions can do their jobs. Does that sound dumb enough yet? Sometimes it's necessary to hold a lot of information in memory because the user actually wants the extension to do a certain task in a reasonable amount of time. They are making our jobs harder because they're too stubborn to understand that extension developers are the ones who create the magic. This is central governance at its finest.
These rules are stupid as hell.
Declarative rules engines are better and preserve privacy better. This is Google following Apple's lead and doing the right thing.
And Firefox and most Chromium forks have no limit. Does that mean they have infinite generosity?
Seriously, how is Apple being bad at something an argument for lowering other options to the same level? Right now I have ~200K filters enabled on this machine. EasyList alone is sitting at 90K according to uBlock Origin.
No its not. Plus, if you combine Easylist with a bunch of additional filter lists, then it very easily crosses 50k.