The docker case as an example:
- your prod docker images are already very likely to be built by your CI
- a Dockerfile typically does: apt update + install, git clone a private repository with your scripts, then build whatever you have to build
- so you already have secrets to manage, given that you need to access your private git repository
- in an environment such as Jenkins, Travis, Github Action, GitLab equivalent, secrets for your CI pipeline are equivalent to env vars that you add to your pipeline settings. It's quite unlikely that you have setup your own.
- let's say you want to avoid to have the private repository as a point of failure, just store the downloaded archives to your own registry. Not too different from storing build artifacts.
I'm not saying there isn't some friction, but it doesn't seem to be a deal breaker to me.