> How do you provide your secrets to your apps? Using an external service? That would still require another set of credentials. Using environment variables? A file only the user running the app has access too? Another way?

A credential/key storage service, either on device/server or as a separate device, with IAM to control whether the user executing that process can use that secret or not. The user in this case for prod services should be a prod (non-human) user.

When all your services are like this, no person should have direct access. You generate key/secrets depending which services you want to allow to communicate with each other and the keys live in the key store. For secrets for external services, e.g. API keys, someone would have to enter them once, yes.

You also should try not to rely on secrets, rather invest in proper authentication/authorization logic.