> Let's hope the HSM supports secret backup well enough
Secrets backup for HSMs from a certain vendor—that you may or may not have named in your comment—I've worked with is actually the easy part. You just make copies of it and all the key data and check it into a git repo, because all of that data is protected by an HSM secret. Distributing that HSM secret among several HSMs for redundancy is also pretty easy.
The hard part is all the administration around it, specifically around custody of the smart cards that contain chunks of the HSM secret: where are they protected, where are the backups of the cards, who has access to them, coordinating sufficient card custodians to meet quorum, etc. You need to meet quorum to provision HSMs with the same secret.
The real "trust me" part of this is arguably less that the vendor backdoored the HSMs, and more that Apple pays the vendor support contracts (that hardware eventually fails) and maintains the knowledge continuity for the teams responsible for administering these HSMs as people join or leave those teams over time.
For what it's worth, this is pretty much why you don't see HSMs used often at less mature companies.