You can have a signing committee vet the asset files and publish the signatures, which the browser extension looks for. It allows an upgrade path without having to go through the browser app store. For example, browser extensions installed using the Chrome app store updates automatically, which implies trust in Google.

The browser extension would only load vetted assets onto the DOM. In other words, you visit the site by opening a new tab and clicking on the extension. It then loads assets and checks that all the assets are signed by a trusted list of code-vetting signers. If all the signatures look good, then it loads the assets onto the DOM. This is no less secure than having all the client code in the extension.