A brief history of code signing at Mozilla
hearsum.ca
hearsum.ca
For a long time a had problems with some websites not loading videos when opened in background because I had an addon which disabled the page visibility api. So I download the source of the addon ( straight from the addons store right click the install button and save as..) changed the manifest to exclude the said sites, zip it back and install it again?
Well your addon needs to be signed to be installed permanent even when you manualy install it. I understand that this is to keep tech-illiterate users safe from malicious addons. So is there a flag/about:config setting for advanced users to enable which skips the check? Nope, there was one but not anymore. So what options do I have? Either use beta version of Firefox as my main browser (no thanks). Or create an account as a developer, upload my addon and wait for Mozzila to check and sign my addon (that I created for myself).
I can murder some trees and poison the environment for all of us, to do pointless mutli-hour re-builds of Firefox for each release and point release to have it accept my add-ons, though.
I've also never seen a reason, why I can't at least place my CA into Firefox /usr/lib/firefox folder or /etc/firefox and have it be respected. Or just place local extensions there and have firefox not require signatures for them, because there's no way these can be installed accidentally from web by clicking some link.
And if someone can trick me into modifying /usr/lib, they can just trick me into replacing Firefox completly with their malwared build, so signing will not save me anyway.
As you said yourself, that's a much bigger hassle and cost. In other words, it's an effective deterrent. Writing to a user owned file is a very low bar for allowing privileged code execution in the browser.
A long time ago browsers used to be infested with all kinds of toolbars and extensions automatically installed by third party software, I for one am glad to not have to worry about that in my computer and on networks I manage or frequent.
Only preview versions and developer versions can run unsigned addons. Both coming with their own set of reasons why you shouldn't use them as your daily browser.
There's no hassle free solution. Only way to run your own code on normal branded Firefox release is to rely on third party signed extensions (eg. Violentmonkey), but that's not really hassle free either if you have 10s of userscripts and multiple browser profiles, and you have to trust some third-party to not go rogue. I got pretty terrible malware from mozilla add-on store in the past.
One ESR build, you can disable signature checks though in about:config. Not sure how this fits into the standard Mozilla orthodoxy. Remember that core tenet of the orthodoxy is that users can't be trusted to protect themselves...
So maybe Mozilla cares less about safety of users that want to use their ESR (extended support) build. There are way fewer of these users than that of main Firefox build, so their safety is maybe not that important on the grand scale of 2.5% marketshare that Firefox still holds.
Well, great. This is at least something :)
With apt, there is a bit of method to the madness: if one imagines that apt’s job is to do the thing the user requested and then deliver a correct system, then clearly essential packages can’t be removed.
But there is not a reason to prevent the user of the software from using the software in the way he wishes on his own machine.
Chromebooks at least get this one thing right. You can disable firmware signing, but only if you are comfortable taking the laptop apart to remove a magic write protection screw. That is something a scammer cannot easily convince a grandma to do.
Modifying a morass of C++ source code is not nearly the same thing as writing an extension in Javascript.
Mozilla is just fundamentally in the wrong here.
Is that what is ailing Firefox? I mean is Firefox losing because it tries to be safest and the best for every user, while Chrome just carries on with basic safety and nothing more?
[EDIT: Clarified the question.]
Besides, this isn't "safest and best for every user", just like the quality vs speed tradeoff in your friend's story is not something he can decide entirely on his own for his customers (though there are other complications in real life)
In a trusted society full of honest people, no.
In an untrusted society with dishonest people, the amount of resources (money, time, cpu, memory) required to protect yourself from abusers will only ever keep growing.
But requiring it for running the code, that's just anti-software-freedom behavior.
(Yes, I know Firefox is open-source and you can theoretically compile it with the check removed, but that is going to take far more time and effort than finding and NOP'ing a jump.)
> In addition to the data collection described in this Privacy Notice, these versions by default may send certain types of web activity and crash data to Mozilla and in some cases to our partners.
Is there any way to switch off that additional snooping around in stuff they shouldn't do because it's not their business what I do on the web with the only version allowing me to run unsigned addons?
[0] https://www.mozilla.org/en-US/privacy/firefox/#pre-release
Even Edge allows me to run unsigned addons (even if it's with a warning, but for my use case I am fine with that one).
Anyway if you want to be able to run normal Firefox version (no beta,developer or esr) and want to install custom extensions there is an easier way but not well know.
Firefox Unbranded
https://wiki.mozilla.org/Add-ons/Extension_Signing#Unbranded...
It works fine expect for the "ugly" icon, but it is possible to modify it.
Mind, it would probably be easier to write your own autoupdater than to runtime patch the signature check in official Firefox...
https://bugzilla.mozilla.org/show_bug.cgi?id=505521
Surely there is some rich lore to be mined.
I also had a shrinkwrapped netscape enterprise server with its server side javascript, and bought a shrinkwrapped sybase -- both at the same time I bought a Sun desktop computer. I bought all 3 at the campus bookstore and paid with a G/L code. I learned solaris, web stuff, javascript, and databases on that job. The novell netware was a directory server based on LDAP or x.509.
This makes me doubt how often these signatures are actually verified on client machines. For example, I've never done it manually because the process is so tedious. Some tools automatically check signatures, but even that is fragmented.
I’m distributing unsigned binaries not because I care about the $200 for a cert, but I don’t want to spend hours figuring out a signing flow.
...and you need to have a corporation?
I thought corporations were pseudo-humans. Turns out it may be the other way around.
Import HSM-protected keys to Key Vault (BYOK) https://learn.microsoft.com/en-us/azure/key-vault/keys/hsm-p...
I guess that allows you to get your own cert (example: from DigiKey) for your own HSM (example: YubiKey) and then upload it.
That's what we researched before we abandonded the approach and kept code-signing by manually downloading the build, signing on a specific machine where only one architect had access. What could ever go wrong?
How would that ever get worked into GitHub Actions?
Although I think even OV require HSM now for local, with some options for hosted services like Azure Trusted Signing.
But I really don't know. I read details only to end up not certain.
Smartscreen, AV warnings, etc use a reputation-based system. No amount of money makes you truly immune from warnings, but paying for EV does give you a higher default reputation. Since there is no longer any file-vs-HSM distinction between EV and OV, it is simply a cost for more default reputation. Any business will pay it easily.
(My prediction is that OV/EV will consolidate to a single offering, in the same way that EV for web SSL has been phased-out.)
Azure Trusted Signing is an all-in-one service that creates keys, purchases the certificate, and hosts the HSM for you. It's still OK to do all these things separately e.g. purchase the cert from Globalsign and use Azure Key Vault as the HSM (you have to pay for the 5 USD/mo service for the larger key size, instead of the cheap one).
ATS is still somewhat new and has not fed into toolchains like https://github.com/sassoftware/relic yet. Although https://github.com/ebourg/jsign does support it.
1. https://knowledge.digicert.com/alerts/code-signing-changes-i...
Seems not to make the process easier, because apparently your organization has to be more than 3 years old[2] to be eligible.
For the traditional way with a third-party cert provider someone on Reddit made a comparison table[3]. The whole thread is interesting.
[1] https://learn.microsoft.com/en-us/azure/trusted-signing/over...
[2] https://learn.microsoft.com/en-us/answers/questions/2082252/...
[3] https://www.reddit.com/r/electronjs/comments/17sizjf/a_guide...
> We are working on making the functionality available for organizations that were incorporated less than 3 years ago. We don't have an ETA yet to share.
From your link at [2]
Here is the GitHub action for file signing using Trusted Signing: https://github.com/Azure/trusted-signing-action.
[1] https://techcommunity.microsoft.com/blog/microsoft-security-...
1. Get an EV certificate from a trusted CA [1]. That will run $400-800.
2. Use managed Trusted Signing from Azure. [2] Identity validation takes ~a week. Signing can be done via web or CLI. This does require a tax history of 3+ years. [3]
[1] https://learn.microsoft.com/en-us/windows-hardware/drivers/d...
[2] https://learn.microsoft.com/en-us/azure/trusted-signing/quic...
[3] https://learn.microsoft.com/en-us/azure/trusted-signing/quic...
https://support.sectigo.com/IS_KnowledgeDetailPage?Id=kA03l0... is an example explaining how to get both a CSR and an attestation certificate from a YubiKey 5 FIPS, on Windows.
https://support.yubico.com/hc/en-us/articles/360016614840-Co... explains how to use Windows' signtool with a Yubikey.
Using a YubiKey does require that you provide your PIN every time you want to do a signature, which limits how much you can automate things. A YubiHSM would remove that requirement, and might be able to work with a self-hosted GitHub Actions Runner, but it's more expensive, and you'd want confirmation from your CA (Sectigo, for example) that a YubiHSM is OK.
AutoHotkey is your friend!
When I set up our code signing machine at AltspaceVR ten years ago, I wrote a simple little AutoHotkey script that watched for the signtool PIN popup and typed in the PIN. It was maybe 15-20 lines of code.
Problem solved.
https://www.mgtek.com/smartcard
For automating pin entry
Some other signing that works in the cloud and has support for GitHub Actions would be DigiCert's KeyLocker (note: every signed binary is counted and by default the subscription only contains 1000 signings): https://www.digicert.com/signing/code-signing-certificates#c...
My formula was: Github actions, Sectigo usb key, physical windows machine in the office that runs a self-hosted action runner that does the signing step.
You can execute it on any platform that supports Java, so I have Linux builds cross compiling to windows with clang and then sign with jsign: https://github.com/BrowserWorks/Waterfox/blob/7eda3b998a56ad...
[1] https://trustzone.com/knowledge-base/purchasing-an-ev-code-s... [2] https://trustzone.com/knowledge-base/purchasing-an-ev-code-s...
That part is trivial, the real pain comes from dealing with the certificate providers.
you can see a usage example here: https://github.com/mscrivo/OotD/blob/main/.github/workflows/...
There's a lot to it, and expect a ton of a back and forth emails with a CA to get an EV Cert.
Here's some example code for generating the CSR (Certificate signing request) to submit to a CA.
This blog post below is great. It works through the process, including the truly weird/awful UX choices Azure makes. I archived it on wayback in case it ever disappears.
https://www.hendrik-erz.de/post/code-signing-with-azure-trus...
(I made and used to maintain Autograph at Mozilla)
First challenge: the tools provided by the OS developers only run on their OS and are of questionable quality, with bad error messages, poor documentation and unintuitive defaults. So I used a mix of open source and custom code to make signing work for every OS on any OS with good errors, docs and sample apps. The open source libs out there are slowly catching up with this, but they don't solve every part of the task and aren't integrated with everything else you need to ship apps (like laying out a working app tree, adding a software update engine etc). It's nice because it lets you ship from cheap Linux CI workers.
Second challenge: you get a proliferation of private keys. Keys to sign your Mac and Windows binaries, keys to sign your Mac update feeds, keys to sign apt repositories. So, Conveyor can derive them all from a single root key that you can back up on paper with a pen. Except for Windows where this worked up until Microsoft started requiring all keys to be held in an HSM.
Third challenge: said HSMs. They're a pain. As the Mozilla blog alludes, some tools like to require things to be done with a GUI. So, Conveyor can detect PKCS#11 drivers and drive the signing process itself using passphrases supplied via CI secrets or similar. It can also store the root key in the Mac keychain if you want to use that, which is a pretty nice way to keep it secure as macOS will ensure that only Conveyor can read it.
Fourth challenge: Windows CAs decided to squeeze extra revenue from devs by exploiting the new HSM requirement's incompatibility with cloud CI. They started offering "cloud HSMs" that can be used without any hardware, just a textual credential again (so removing much of the security an HSM is supposed to create!) and these are popular because everyone wants to ship from CI. Unfortunately these services all charge extremely high prices per signature once you go over a fairly small monthly allowance. If you want to ship builds on a daily basis, for instance, you can easily blow hundreds or even thousands of dollars on signatures alone. So Conveyor has a disk cache that stores signatures for binaries, and separates them from the underlying DLLs/EXEs so the cache doesn't fill up too fast even if you're shipping very large binaries, and a GitHub Action to integrate that cache with GitHub's own. All this saves a lot of signatures and therefore money.
Fifth challenge: Windows chooses to identify apps by the hash of their X.500 name as decided by their CA, but CAs constantly change their mind on how to format this name and change it for pointless reasons. Because the Windows ecosystem hates developers this can easily break app reputations/updates that are checking code signatures, and nobody at the CAs or Microsoft cares. So Conveyor has a whole system that recognizes when this has occurred and can drive an automatic reinstallation to repair things. The user doesn't notice except for a short delay on app startup (and seeing a progress bar).
Sixth challenge: platforms like the JVM that like to hide native code inside ZIP files, which may also contain native libs for the wrong CPU architecture or platform. Conveyor finds them, deletes the irrelevant binaries, signs the ones that remain and puts them back (or extracts them into the app directory and ensures they get found).
Seventh challenge: enterprises that have custom internal signing servers. So you can write a little script that submits binaries to the server and gets them back again. This works with all the above stuff too, when relevant.
Eighth challenge: MSIX. Microsoft's only non-deprecated packaging tech has a lot of benefits, and allows non-admin users to install packages, but the signature format is pure madness. By far the craziest scheme I ever saw. Reverse engineered and implemented.
Next challenge on the list: Azure Trusted Signing, Microsoft's new CA. It's marginally cheaper than the others, so some users want to use it, but it also doesn't provide a standard driver or use standard protocols. It'll require custom support.
Looking to the future, the path forward is to use kernel sandboxing tech to eliminate signing requirements for our users when possible. For apps that can fit inside the sandbox it should be possible for us to sign and distribute their apps for them, with full code signing only required if the app has special needs. That would give a much more web-like experience for non-web apps, where a developer can just create a scaffolded app with one command, ship it with a second and not think about any of the platform specific details or code signing at all. Unfortunately this is a way off yet.
Some of the HN threads:
https://news.ycombinator.com/item?id=20421948
https://news.ycombinator.com/item?id=19871989
Edit: removed speculation if consequences of privacy loss, but come on … that’s still pretty serious.
We didn't think that the intermediate CA expiring would break the signatures, because code signing generally doesn't care about expiration, but we never tested the code path until the intermediate expired and the signatures broke. That was a hard lesson to learn...
Got any more info on this? Seems like an absurd exaggeration.
Toss in another 150+ human rights activist murders per year, dozens of whistleblowers, 60+ transgender people, and any number of other at-risk groups who tend to rely on technical measures to preserve their privacy and safety abroad.
Did that info being leaked definitely get them killed? No clue. Did it "probably [get] people killed"? I don't know how those people are typically uncovered, so it's hard to say if a few privacy extensions actually mattered. It doesn't seem like an outlandish claim though (and with a prior that those extensions do matter, it's a likely claim).