Hacker News Reader

Comment by open-paren | Hacker News Reader

I manage a medium-sized browser extension at work. We also offer(ed) it on Firefox. But I have spent the past year struggling to get back into Mozilla store after a manual review. As far as I can tell, there are maybe two reviewers that are based in Europe (Romania?). The turn around time is long when I am in the US, and it has been rife with this same kind of "simple mistake" that takes 2 weeks to resolve. "You need a privacy policy"–we already have one. "You are using machine generated and minified code"–no you are looking at the built code, not the included source. "We cannot reproduce your source"-that's because you didn't follow instructions and are in the wrong directory. Very frustrating.

View on HN

Similar boat. I release an extension with about 1 million installs across Chrome/Firefox/Edge for work.

Firefox (despite being the smallest usage) is utterly insane with regards to process. They demand a reproducible build, but then can't do things like install the right version of yarn (no - npm install -g yarn is not correct, our readme says it in bold like 5 times and provides the exact correct command to install the right version), or follow basic setup steps like "Use this version of node (complete with exact steps to install it and a script to automate that for them)".

God fucking help you if you try to do something completely crazy as a private company like - checks notes - use a private NPM module. Despite providing them with access on a pre-configured account, or offering to give a review account access according to Mozilla "It's too hard to use external accounts during review".

Honestly - having to interact with the browser review team is a BIG reason I no longer recommend Firefox. They're incompetent at best, and I'm fairly convinced they're just milking the google search deal income for as a much as it's worth - I don't think they really want to provide an alternative and secure browser anymore.

On the flip side, having to interact with addon review has raised by confidence in the browser. The steps they take to review, while not perfect, seem like they could weed out a lot of potential garbage and malware. I was expecting a much more minimal review process, which would have raised my fear about the extensions I use and set to auto-update.

Reproducible builds and open source sounds like a good thing.

I wouldn’t expect the reviewers to deal with every add-ons bespoke snowflake build. Even less so if it requires access to a private module. Mozilla should provide a baseline of how a build is intended to be done, then extensions just have to follow this template. Though yes, you would expect them to have some familiarity with basic stuff like yarn and that the baseline supports a few of the most popular builders.

We use a relatively simple build. at the base of it, if you have node and npm, a complete build is as easy as

yarn npm login

yarn --immutable

yarn build

Personally - I don't really find it reasonable to place demands on build tooling for an external company.

I'm assuming you would also find it reasonable for Google to suddenly ship chromium with a requirement that you use "google-pack" for all js builds or they don't run it?

To be entirely blunt, what exactly do you think is going to change when we're already giving them bare JS? It's not like we're shipping a binary blob here, we're literally handing them a zip file with perfectly fine & inspectable javascript inside it.

Further, do you realistically believe that a single low grade QA/Support engineer who can't even install the correct tooling is going to catch malware?

Because I read their matrix chats and I can fucking promise they aren't catching the malware all that fast....

> I don't really find it reasonable to place demands on build tooling for an external company.

I'm not sure I agree, plenty of OS distributions do this. If you want to distribute on Arch in the official AUR you're going to need a PKGBUILD file. The difference though is they make it very easy to integrate custom distribution channels where you can build the package however you want, and I would really love to see browsers move more in that direction. Requiring centrally managed signatures from a corporation to install extensions in a purportedly open and community-driven product is just absurd to me.

> I'm not sure I agree, plenty of OS distributions do this. If you want to distribute on Arch in the official AUR you're going to need a PKGBUILD file.

This is fine. This is actually also roughly in line with what you need for an extension (a manifest.json file).

What the poster here is proposing is rather this: You cannot build that PKGBUILD file using any tooling other than the standard. Ex - you want to script how that PKGBUILD file get made? Fuck off, not allowed.

That's a COMPLETELY different take. It's not dictating limitations on the output (which I find reasonable as a required integration between products) it's dictating limitations on how a company produces that output (I find this monopoly behavior, why should they get to tell me what tools or processes to use? My output is the SAME.).

Docker

It seems reasonable that they'd have a requirement there's a single file they'll run, maybe even with a predetermined name like ./build, and that's it.

The developer can then juggle all their dependencies and run make/yarn/npm/etc within that. It's really not different from having a CI build script.

And surely they should accommodate controlling the ENV of that build process as well?

Perhaps by doing things like setting NPM_TOKEN for the correct account?

Just like you would with CI?

... except they claim that's too hard.

> I'm fairly convinced they're just milking the google search deal income for as a much as it's worth

That's exactly what the ex-McKinsey C-suite are doing. Regular employee talent suffers because of it, as you've found.

This is exactly what the review process for the Play Store is like, even worse for Google TV apps. Often times just re-submitting multiple times without changing anything at all will get it pushed through.

Yeah but despite how much HN hates Google everyone here will do whatever it takes to get on their app store. Google has the power to make the entire industry their bitch.

Mozilla not so much.

So much this. Mozilla barely breaks 10k installs out of our total 1million installed base.

We had a really frank internal discussion about just dropping support for Mozilla because their review process is also the most expensive out of every client we currently ship (And not "reasonable expensive"... Useless time sink expensive, back and forths with folks who I would frankly not hire as a junior because they can't read a readme file and follow basic and clear instructions.)

They are acting like they have the position to demand these reviews... and they just don't.

Good devs just leave because they're a waste of time and money, and they're STILL rampant with malware on their store (Mozilla is literally the only one of the major vendors that will make a listing live with no vetting, and then 4 months later yank it because of "problems"...).

It doesn't make me feel secure, it makes me feel like they're trying to market security. It makes me really dislike mozilla, and firefox was a formative part of the my tech career early.

I totally understand if it isn't your hobby supporting Firefox doesn't make sense.

This sounds super frustrating, as someone who has an idea or two for browser extensions I'm not looking forward to all the bureaucracy. I actually love the idea of requiring and validating reproducible builds but they really should invest in reviewers competent enough to manage that.

I do have half an idea to deal with it that I plan to try, thought it might be helpful to suggest: implement a Fisher-Price build system that checks and automates every single step and cannot go wrong. Ideally if the reviewers can run Docker, do it all in a container. Wrap package.json scripts with functions to validate the build environment before proceeding and either fix it automatically or fail and print clear instructions to the console. A preinstall hook could verify they have proper NPM auth and prompt for it if needed.

Annoying to have to do that at all though. I'm starting to come to similar conclusions on Firefox, using it currently but I've been thinking about jumping ship for a while. What browser would you recommend now? I wanted to get away from Google but I'm considering just Chromium since any remotely comparable options I've found are poorly thought out wrappers of it.

> ... it might be helpful to suggest: implement a Fisher-Price build system that checks and automates every single step and cannot go wrong...

Programming is a race between the programmers, trying to build better, idiot-proof software, and the Universe, trying to build better idiots. Do not underestimate the Universe.

You really feel this in UI design and it’s hard to get the balance right.

They should switch to an fdroid like model that does public builds on cloud infra.

It sounds like they are doing their job attempting to review random code from strangers to be honest.

Honestly I have to side with Mozilla team here. Kudos to them for trying to actually care about security and privacy. I can imagine the nightmare that people are submitting and trying to recheck everything and build those random extensions with private npm repos and whatnot.

It’s funny to think of Mozilla like landed gentry where they have captured serfs (their users) and get a payout from the king (Google) for their loyalty and support.

> We cannot reproduce your source

This is the biggest issue we had, and we had to add a decent bit of complexity to our builds to support reproducible builds in the exact way they want. But the silly part is that our extension involves building a wasm file from Rust, and after some back and forth it turned out that they don't require it to be reproducible (despite being core of our extension and containing 99% of our logic), which honestly feels like it defeats the point - who cares if JS reproduces if you can hide any arbitrary possibly-malicious code in wasm.

For a while we were seriously considering putting our prebuilt wasm in the source package or on npm, just to make the "reproducible build" on AMO side simpler, despite this making it even further from how it's actually built.

What kind of harmful code could you put in WASM? You could return a string that you eval on the javascript side, so the reviewers could possibly ask for the WASM source if they saw the eval, but other than that the purpose of WASM is to be a safe sandbox after all, right?

I'm not familiar with the security guarantees of WASM in the browser but I imagine they're more along the lines of preventing data exfiltration from the browser/OS, it would be difficult to prevent something like abusing your CPU resources to mine Bitcoin in the background for example.

One way to protect yourself from bitcoin mining is to not give a WASM program both an access to get incoming data and send data both into a 3rd party server. Another possibility is to threshold computation power on the WASM interpreter so that there's a limit of opcodes processed.

Every time I hear about the review processes for browser extensions I'm shocked that the it involves humans having to read your README and manually plumb together the build process. Sometimes I hear that reviewers are even reusing VMs when doing reviews, or even not using VMs at all. I'd have expected the review form to have a textbox where you paste your git link and a well-documented automated pipeline that stands up a specified VM with a specified amount of RAM and disk, clones the git, descends into it, and executes `docker build -t ./docker/review/Dockerfile`. I'm surprised that the reviewers themselves haven't outright demanded such tooling from their larger organization, just as a matter of job satisfaction - I can't imagine all the abuse they get from angry app owners.

Browser extensions really seem like they're slowly failing and just not supported. Kinda like PWAs.

I want to write a chat program, but it has to work on phones, and the DevEx for native phone frameworks compared to desktop apps looks like hell, and PWAs seem to be barely supported.

It's easier than ever to make a CLI or desktop app, but phones seem like the worst of all Microsoft dev history - Learn these arcane lifecycle vocab words that make no sense, like using Win32 directly, but also it changes every year or two like when MS invents a new GUI framework, but also if you can't get into The Store, nobody but your power user friends will be able to run your app anyway. What is this shit?

Someone will come up with a solution that is utterly ingenious. Like the ability to install a plugin without third party intervention with a single click.

>Browser extensions really seem like they're slowly failing and just not supported. Kinda like PWAs.

Ya, totally!

lmao

ublock origin has 8mil users on Firefox alone.

I don't think OP means for the lack of need or popularity, more so because vendors and platforms do not want them to be.

Yeah especially with Mozilla's new focus on promoting less-tracked advertising wih their anonym acquisition. Ublock origin of course hampers those efforts. I wouldn't be surprised if they want it gone just like Google does.

The problem is - I can switch the browser and not even notice. But give me one without uBO and I will switch immediately.

Also had these issues when working on my previous job's extension. The Firefox review process was a real nightmare to work with. Same heavy delays and misunderstandings your mentioned. Eventually the company just stopped updating the Firefox extension as often since usage was low and the review process was such a pain. Unfortunate for me, as the only engineer (maybe employee) at that company that used Firefox.

Same here. We even had a special "mini" Firefox version that didn't require any additional Javascript build step, to make the review easy. But there were so many issues with the review and so few users that we just decided to give up.

The whole extension change Mozilla forced on Firefox seems like some sort of sabotage.

Mozilla sneaking in more and more spyware and ad friendly functionality seems in line with the same conspirators.

And given how high profile all these changes are, it runs to the top of the company.

So, which browser are you using?

Firefox... I wont give up on them just yet :)

The problem with these types of things is that the people who are qualified to do good reviews are also the sort of people who can typically get a far more interesting job building stuff, rather than just reviewing code. It's work that does require a certain level of skill, but at the same time is also quite boring.

And that more interesting job will probably pay better as well.

Id rather hire a senior dev as a reviewer and a mid dev as the coder at a company. pay the reviewer more since they will be dealing with shit practices and having to train the dev.

Not only that, but properly reviewing code would take forever. Heck I don't know how many senior engineers at my fancy tech company could do it and reliably spot problems.

I think that is one way that “tragedy of commons”.

> that's because you didn't follow instructions and are in the wrong directory.

You just need to have a shell script in the root directory that assumes the person running it has 0 clue about your extension.

Also some of this reminds me of Apple. They clear something up, then bring it up again the next time review is needed.

Even this we had issues with - we wrapped the entire build environment and script in a dockerfile, but depending on system configuration you may or may not have to run docker with sudo - it just so happened that reviewer's environment required it, while ours didn't, and the reviewer needed specific instructions on what to do in this case.

Another time, they failed the review because the reviewer's VM _ran out of disk space_ (which we only learned after digging into the issue, as the first report just mentioned "build errors"; according to later inquiries the VM had ~9GB available) and we had to add some extra build logic to delete intermediate files, just for them. The build is quite large because it involves rust->wasm compilation, but I'd still expect the reviewer's machine to have a bit more space...

Simeon from the Firefox Add-ons team here. Sorry about the rocky experience. I realize this is a bit late for your situation, but earlier this year the source code submission docs were updated with information about the default reviewer build environment[1].

It's not a huge improvement, but it sounds like one thing we could do to improve the communications process around build errors is to include a link to this documentation in the notification email sent to developers. I'll create a ticket for this now.

[1]: https://extensionworkshop.com/documentation/publish/source-c...

Everything described here sounds like your team, your extension, and your software development process are the problem. Demanding >9GB of disk space to build a browser extension is capital F, capital I Fucking Insane. Go yell at the Rust folks about their shitty toolchain and your engineering lead for buying into it instead of blaming people who have enough problems as it is just coming into contact with the quagmire you described.

The 9GB limit was not just the Rust stuff, that was for the entire docker environment with compiler, JRE, node, wasm toolkit, typescript, webpack etc. Yes, we need all of these to make a "true" reproducible build from scratch.

> to build a browser extension

It shares 99% of code with a desktop application; you can compile it to wasm while preserving most features. The extension wraps the wasm.

For reference, when making a single clean build, the `target/` dir reaches 700MB.

> The 9GB limit was not just the Rust stuff, that was for the entire docker environment with compiler, JRE, node, wasm toolkit, typescript, webpack etc.

None of this is surprising or exculpatory. Demanding >9GB of disk space to build a browser extension is insane.

> we need all of these to make a "true" reproducible build from scratch

You need and them to reproduce your build. You definitely don't need all of them to build what you're building.

You certainly are confident that you know more about GP's situation than they do.

When you took your desktop app and built a browser extension version, did you really rewrite the entire app in vanilla JavaScript just got the Mozilla review team as you seem to be expecting GP to have done? How long did it take you? What sort of opportunity cost was there from investing your time on that instead of adding value to your product?

For someone who opened their post with a first sentence like that, you're making a lot of (bad) assumptions on your end; most of your questions are unanswerable or have answers that you are clearly expecting to go the other way.

Demanding >9GB of disk space to build a browser extension is insane.

Thank you for setting such a good example. If I were you, I don't know that I could have given such a good and dispassionate reply to such an arrogant, overconfident, and rude comment as you did. Your comments are not only technically interesting, but also epitomize. What a healthy online community should be. Thank you for doing what you do!

> that assumes the person running it has 0 clue about your extension.

I would tend to assume that a person given responsibility for reviewing this software, supposedly to protect end users, would not be this clueless.

What value is the "Firefox Store" actually offering then?

> What value is the "Firefox Store" actually offering then?

That anyone dumber than such a reviewer cannot sneak malicious extensions in.

Which, sadly, is probably a non-trivial number of submissions.

> That anyone dumber than such a reviewer cannot sneak malicious extensions in.

Although people smarter than such a reviewer are free to? What kind of standard is that?

> Which, sadly, is probably a non-trivial number of submissions.

Then they're not, as an organization, actually capable of doing what they're promising here. There are more ways to get this wrong than to get it right, and borrowing the Google strategy of just not caring about your end users seems completely inappropriate for a non-profit like Mozilla.

> What kind of standard is that?

That's the standard of all currated stores.

We can argue about whether Mozilla's reviewer skillset is too low, but there's always going to be someone smarter than a reviewer, when reviewing is a cost center that companies want to spend the minimum amount of money on.

> That's the standard of all currated stores.

This seems to ignore how boutique stores and high end retail operates. This is the standard of rent seeking middlemen stores. You still haven't answered why this model is appropriate for Firefox.

> We can argue about whether Mozilla's reviewer skillset is too low

We're not. I'm pointing out how simply taking the opposing view reveals that your reasoning could not possibly be correct.

> reviewing is a cost center that companies want to spend the minimum amount of money on.

Which is weird because I assumed the cost of re-creating the plugin yourself would be much higher than that. It's almost like continual failure of these simplistic analyses reveal that a broader examination is required.

You think the best analogy for the Firefox extension store is boutique brick and mortar retail?

A minimal cost reviewer model isn't appropriate to Firefox.

But, example counterargument as to why it might be: Firefox needs to ensure they don't open themselves up liability but doesn't want to fully fund/staff a review team.

It could be $0, volunteer labor. I doubt it’s a paid position.

> I would tend to assume that a person given responsibility for reviewing this software, supposedly to protect end users, would not be this clueless.

would you do that job 8+ hours a day for little pay?

Would you run a foundation that forces it's users to be dependent on such a job?

Ya'll are putting the cart before the horse. I'm not being critical of the reviewer but of the large non profit organization that is responsible for creating this failure. Which apparently only exists to pantomime what the for profit players have built and is unsurprisingly equally wasteful of open source developers time and skill set.

Why does Firefox even need a curated "store?" They could have built anything better. I'm sure they were paid, er given "donations," that ensured they would never try. And from what everyone has been saying here those donations got exactly what they were intended to get.

Even Hacker News seems to unquestioningly assume this is a rational way to manage an open source plugin ecosystem. That this is the fault of the plugin author somehow or the store reviewer somehow. It's really disappointing to see.

That's not just mozilla. Google's review team all are in India and they cannot write clear English. It's a mess.

>Google's review team all are in India and they cannot write clear English.

Which is ironic considering the reason they went to India and not other countries with cheap labor is that English is an official language there.

The problem is that the set of "Indians who can speak fluent English" and the set of "Indians who will work for the absolute lowest bid" are exclusive. And I don't blame them, really.

These execs mistake "English is an official language" for "English is a widespread first language". Only 0.02% of Indians speak English as their first language, while total speakers (of first, second, or third language) are 10.6% of the population.[0]

[0] https://en.wikipedia.org/wiki/Languages_of_India#Multilingua...

Also don't forget it's an Indian dialect of English, with words and usages of English words that don't exist elsewhere in the world.

Indian dialect is derived from the colonial English. So, lot of words and usage can be found in British English.

I don't think that most of Brits are "doing the needful". Indian English has plenty of expressions that are exclusive to India.

Pretty sure "why did you redeem it?!" is a British English slang from the victorian era :)

While English is not a first language for the vast majority, it is used a lot in daily life because the native languages vary wildly by area and nobody understands them all. English is the common denominator, not just for communicating with foreigners but also to other Indians from other areas.

The focus on primary language makes it seem less used than it actually is.

The last time I had realistic numbers, an outsourced engineer in India cost a bit more than a comparable one in the Midwestern US.

I’d guess they’re more expensive now, despite the obvious timezone problems.

omg I work with some Indian people since 2000, and I canonly understand about 80% of what ONE OF THEM says, the others less than 60%. :(

Also Apple and Meta. It's awful dealing with infallible gatekeepers.

Why make such a generalized statement?

I had these issues too a few years ago. Now the review time is shorter than Chrome’s and hasn’t been flagged in a few years. However my extension has about 10k users, if that makes any difference.

this seems like the kind of place where user-based reviews would be more efficient, better, and more open

having the makers of a browser do this is bound to create both efficiency and political problems for extensions. im remembering dissenter now

That's interesting to hear. Do you also offer your extension on the chrome store? How did the review process differ? I ask because I've only published on the chrome store in the past.

The reproducible build requirement seems to be a major blocker for many addons, including one I use for Twitch: https://github.com/FrankerFaceZ/FrankerFaceZ/issues/1495#iss...

Reply on news.ycombinator.com