Google Took Down My Chrome Extension for Using Lodash
roadtoramen.com
roadtoramen.com
I used to work on Mozilla's Add-ons store, and god I hate the reviews. Mozilla used to do them manually. I WORKED there, and wasn't able to get unblocked (for an extension used by almost everyone at Mozilla) because I was using jQuery... which was started by a guy who ALSO worked at Mozilla. So I feel ya completely.
The only thing I'll say is... Extensions have a scary level of access (they're basically almost as easy to use as a website, but with unlimited powers such as recording every site you visit and stealing passwords), and the average user will never understand that. The source code is hidden and updates are pushed to users silently, so a compromised GitHub account could result in a huge hack of everyone's everything. They're far more dangerous than a malicious iPhone app in Apple's app store.
Luckily, this seems automated, and seems like they'll fix it. Especially if it gets traction here.
The malicious code can add third party script references which then behave differently depending on target user or time of access, and these can be credential thefts/keylogging, (very inefficient) cryptocurrency miners, ad injection or participation in click farms, or just general connivery.
I'm very glad Google is starting to do source code review on extensions, it's long overdue given the access they have.
In this case, they flagged a pattern of unicode that Google felt nobody with good intentions would be using. Turns out, Lodash used that string for something, hence the blocking. It had nothing to do with obfuscation, but rather a good-faith attempt to block bad things.
Similar to how you might trigger heavy ratelimits on a request like `wp-admin/index.php`, since it's clearly a bot trying to find vulnerabilities.
It looks like it triggered on text having too much entropy, which is really reckless on such short sequences. If there's no specific thing it's trying to block, there's less good faith to give credit for.
In this case, the review was pretty much correct. The code absolutely looked obfuscated, forcing the author to dig through a bunch of dependencies just to figure out where the offending strings (in HIS OWN EXTENSION!) were coming from.
And the answer turned out to be benign. But it sure didn't look benign until a relatively deep analysis was done. Extension reviews aren't supposed to be for deep analysis.
However. Banning an old app suddenly and not really providing any response to deal with it... that's not a good solution.
It has billion dollar startup written all over it.
This is something I chew over all the time. How can we have a large corporation dealing with the number of users that Amazon/Google/etc are dealing with, as well as the number of malicious parties trying to break them without resorting to either systems that create excessive false negatives and provide little to no appeals process of any use, and without having a massive overhead cost for handling the appeals in a judicially reasonable way.
Everytime I work on it, it turns into that it requires more people who are properly trained in almost legal theory, and then you end up quickly with the same problem that many policing and court systems have which is very quickly individuals with an axe to grind work to achieve regulatory capture.
It sucks, but remember that Chrome Extensions have basically unlimited power, and Google is doing their best to protect users.
[EDIT: And it's resolved! The extension is back]
Only if gets traction here, history proves.
I’ll second that. I worked for a group which was a startup acquired by AVG. They were a data science acquihire so that avg could better monetize the data they’d been collecting from their extensions over the previous decade.
Once I found decade old content from my dead father’s browser, I decided to never use an extension again.
Seems to me that the solution to that is to make permissions extremely granular and grade the level of review scrutiny an extension gets by its permission granularity. Then, only extensions that want to access or modify something truly sensitive would run the risk of getting smacked down randomly like this (as is always liable to happen with an automated process).
Not just "read all websites" but "read the URL only", "read the visuals but not the HTML/CSS", "inject predefined code onto only website X".
It also incentivises extensions to ask for less permissions so they can pass review more quickly.
Even on Android (and I presume iPhone) where apps ask for/document their permissions requirements, most people just say "Yes" and move along. Most people have neither the time nor the expertise to critically evaluate whether or not an extension needs any of a dozen permissions.
I sympathize, but I have no idea how you can realistically solve this.
That seems ripe for more of the same articles of this flavor. The specifics would probably be just something akin to "turns out that using this permission in a way that Google didn't anticipate immediately gets your app shelved and leads to loss of users."
That's probably better than the scenario we have right now. Such a system of smaller permissions would be an improvement over every extension asking for all permissions and being able to do what they want with your browser.
When an extension update changes the required permissions it gets disabled and you get a prompt to re-enable with the changed permissions.
Probably not going to be all that popular on HN, but this is why I'm very hesitant about using extensions on Firefox (and I suppose Chrome too, but it's been a while since I've used that browser). I'm not going to audit the source code of these extensions, so why should I trust them with privileges to essentially record every website I visit?
I wish Chrome and Firefox would adopt a content blocking system similar to Safari, where the author of the ad blocker can essentially tell the browser what content to block, but the ad blocker itself doesn't get any information about user activity. This could exist alongside the existing extension system, in case additional capabilities are needed.
Yes this is less powerful than giving the ad blockers content to all the user's browsing history, but I'd happily exchange less effective ad blocking in exchange for enhanced privacy
"Hesitant" doesn't convey the level of fear everyone should feel after reading "Has acess to: All data on all websites".
Same applies to apps, of course.
By the way, I think a pi-hole is safer than an ad-blocker extension, but I'm not sure. What would I be opening myself to if it gets hacked and my DNS server is now the enemy?
They could track where you go online via the DNS requests ( the sites at least ), and they could mess with anything unencrypted.
Used to be if an HTTPS site loaded an HTTP script resource it provided a dangerous place middlemen could invade your secure session.
chrome, IE ( presumably now Edge as well ) and firefox rightfully block mixed content by default, and have for some time
https://developer.mozilla.org/en-US/docs/Web/Security/Mixed_...
( the mentioned Firefox 23 came out in 2013, so like I said, a while )
Please, no. This is exactly what makes Safari all-around worthless, despite all the hard work they put into it.
Chrome will get forked the minute they do this, and the forked version will restore that functionality, and I will immediately install that fork and uninstall Chrome.
Keep in mind that I said that the legacy extension system should remain in place, for people such as yourself that wish to utilize it.
That said, I wouldn't characterize Safari's content blockers as worthless. It's not as effective as Firefox's ad blockers, but I still rarely see ads. If I had to guess, I'd say under five ads per month manage to sneak through.
This is exactly what Chrome's Manifest v3 offers. As they put it: "There's a new declarativeNetRequest API which lets extensions modify and block network requests in a privacy-preserving and performant way."[0] MV3 went live earlier this year so you can build on it now, but no word on when extensions will be required to migrate.
Manifest v3 has been widely criticized [1] for weakening ad blocking which of course aligns with Google's self-interest. It's a tough balance to strike.
[0] https://developer.chrome.com/docs/extensions/mv3/intro/mv3-o...
So yes, it's technically possible to observe network requests but in practice few should be doing it.
At least with the mobile app stores, a good review process with some human recourse is baked into the incentives of the platform, since lots of money is involved.
With the Chrome web store, you get the feeling it is just there because Google capitulated in the past, acknowledging that browser extensions are necessary to be a mainstream browser. Beyond that, it seems to be moderated with two goals: reduce overhead cost and mitigate any security threats.
The result is the complaints you hear from every extension developer:
* Google arbitrarily blocks a decent % of updates, because of automated flagging
* Google removes extensions with basically no recourse, often for unclear reasons
* Google itself (if I remember correctly) even recommends using a Google account separate from your personal/work email, since often these automated suspensions take down the account connected to it too
Don't get me wrong, browser extensions have a lot of power and that's often an issue (malware, browser history collection, phishing, etc). Still, building an extension for Chrome involves the constant reminder that you are under the thumb of a big monolith organization that is indifferent to you.
The removal process ought not be automated, unless it's a serious violation that can be detected algorithmically with high confidence.
The algorithms detecting violations are clearly buggy as hell. Our take down also made little sense, and we had to resort to support, which is itself a nightmare. There are numerous similar stories a Google search away.
Extensions are growing to be a fundamental part of the browsing experience- Google ought to invest more into improving the ecosystem.
Scaling the human review process should protect both users and developers.
It appears the algorithms are crude by Google's standards.
If I were a determined malicious actor, bypassing a crude algorithm seems like a low bar to clear.
In contrast, if I knew a human were reviewing things, and if perhaps submitting extensions required the same business verification process iOS development does... the bar for a malicious actor is a lot higher.
I’m not Google but I’m not sure if I agree here (though I agree with you and the parent comment generally). It’s extremely uncommon, lodash notwithstanding, for JS code to have a dense set of regex partials full of Unicode escapes, even with modern build tools. It’s a lot more common for malicious scripts.
Determining which is which would require a level of static analysis of dynamic code that, if Google had it, they’d be showing it off in hot new build tools. They’re just making the somewhat reasonable assumption that it smells bad, and overreacting.
And if they were to whitelist known popular libraries there’d be an uproar about playing favorites and excluding some extension which uses, idk, xregexp, to improve accessibility.
A very good point, I hadn't looked at it that way. It makes me wonder why lodash would need this kind of construct.
We use one of those static analysis tools to find 'threats' as a sort of audit gatekeeper before release. Any issues found have to be addressed either by code changes or by comments. I'd estimate around 95% of the issues found are false positives.
Examples include flagging the 'Random' class (C#) for not being cryptographically secure. This is true of course but that does not mean that there aren't a plethora of valid use-cases for using such a class.
Then there's obviously all the cases where the tool is just not clever enough to follow the program flow.
> having countless hours of work be obliterated by an automated system without manual review (presumably) is terrifying
The code in question was "obfuscated" to an automated system, yeah they're unicode escape string for a utility library but a lot of usages of binary/unicode escape strings in code is considered to be some malicious obfuscation.
What they did is smart. They have their analyzer set to fire on obfusacted code. They direct it to you. If you're a perpetrautor you'll say oh noes they got me, if you're innocent - you'll request manual review, contact support or escalate in some other way and they get their manual review from you actually.
I would guess that their false-positive rate for this static analysis has to be low. There aren't many legitamate usecases for these escape strings.
The solution they could consider is to after this alert is fired to check whether the file in question matches a checksum of some recent lodash releases but in the end "using lodash" is not an excuse, as a developer you're also responsible for the dependencies you bundle with your extension so you should be mindful of what code malicious or not is in there.
Ah, yeah... Because that works so well. You have a good argument, pointing out that extensions require huge scrutiny, but it's tainted by a dismissal of the actual issue:
Google's escalating process is utterly broken.
They've effectively managed to recruit HN as front-line support, for free. Why pay people when they can just fix the issues that bubble out to the frontpage of whatever social media.
You're right. I didn't look at it this way.
They recommend adding the main `lodash` package and then importing only the functions you need.
import throttle from 'lodash/throttle`;
https://til.hashrocket.com/posts/dnzttruljf-prefer-lodash-es...
https://itnext.io/lodash-es-vs-individual-lodash-utilities-s...
https://reclaimthenet.org/firefox-rejects-free-speech-bans-f...
https://reclaimthenet.org/google-chrome-web-store-bans-disse...
"Developers must not obfuscate code or conceal functionality of their extension. [...] Minification is allowed, including the following forms:
- Removal of whitespace, newlines, code comments, and block delimiters
- Shortening of variable and function names
- Collapsing files together"
If "script execution is much more expensive than image rendering" it makes compression even less relevant for scripts. If it takes 2s to execute a script the 0.1s gain on the wire is negligible.
Because it’s not the same benefit. Executing the JS is more expensive than rendering the image. That’s just starting at square one. To the extent executable code can be delivered faster, it also opens possibilities of using those gains to support newer codecs with much smaller image sizes and reduce time on the wire even further; this generally leads to WASM, but the idea is the same.
> Maybe what you were trying to say was that scripts are more important than images for a web app (which I would agree).
Nope. I’m saying that JS execution has a bigger impact on websites feeling slow than large images rendering. It’s probably more noticeable on content driven sites because they’re expected to load faster (and more likely to paint but have a long janky interactivity gap).
> If "script execution is much more expensive than image rendering" it makes compression even less relevant for scripts. If it takes 2s to execute a script the 0.1s gain on the wire is negligible.
I honestly can’t even follow your logic. It’s not just “expensive” in the abstract, it’s the largest contributor to pages feeling slow for users. Shaving 5% (which is an arbitrary measure, and far less likely to be true on slower connections) off that might mean the difference between whether they even see your page, or its gargantuan images loading in, at all.
In particular: Why do we assume that all apps and extensions should have unrestricted access to the internet? Why is that not a privelege like "read and write" contacts in mobile apps or "read all tabs" in extensions?
My only qualm would be if instead of removing your extension entirely, maybe it would be better to say "App is currently being reviewed and will be made available soon."
It's getting old.
(To be clear not hating on this guy, his extension, post, etc. Just the status-quo of how these problems are "solved")
Wish it didn't require getting on the front page to find a timely resolution on these sorts of issues, but it's heartening to see that the community is so supportive whenever something like this pops up.
Or just not use lodash. most of their helpers are easily done these days.
Random binary strings looks a lot more like obfuscated code when it's minified. The easier to make it for the reviewer to understand what the code does, the less likely it is to get taken down.
At the end of the day, the goal is to put a stop to extensions being bought off and tracking code being added to them in an obfuscated way.