Agreed that it feels like there should be another mechanism to publish subresource integrity hashes for your own resources somewhere so that people can spot if your server or TLS gets compromised though (similar to CT). Sounds like someone else had the same idea at https://lists.w3.org/Archives/Public/public-webappsec/2014Ju... - I wonder why it didn't go anywhere?
Meanwhile, if you don't want to trust TLS or the host which originates the code, then for now you're probably better off using a desktop app (e.g. electron, although that comes with a whole different attack surface) or a browser extension which can be distributed as signed code.