You can even change some webpack configs without ejecting using react-app-rewired. Its pretty nice if you only care to change one or 2 simple settings. (like name of build files, babel plugins, etc...)
How difficult would it be to use this to add a staging environment? It seems like that's the only build config change that I ever desire.
I'm not sure with what you mean by staging environment in this context. You can run your app in a webpack-devServer or build it then use your own server.