Or that perhaps you have a perfect alignment algorithm which you are unwilling to share with the broader research community (evil)?
If you're doing the training yourself, you at least have a verifiable supply chain and an audit trail. Today, we have no idea if a black-box model handed to us is coded to recognize specific domains or patterns and back door an application in a sneakily targeted way.
Black box is a black box. Many open-weight models clearly haven't been trained or created in the manner their creators claim---which raises the obvious question: if they lied about the recipe, what else did they lie about? Putting those models in a production capacity scares the living crap out of me.
That said, building from scratch isn't about achieving mathematical perfection or manually auditing 15 trillion tokens---that's impossible. It's about eliminating third-party supply chain risk and having actual governance over the pipeline.
Of course, that doesn't mean we can magically guarantee gradient descent won't produce weird emergent behaviors, or that we can blindly trust OpenAI not to backdoor things. But at least with the latter, you're making a calculated operational decision rather than blindly trusting an opaque black box of entirely unknown provenance.