Have a look at multi stage container builds. Your images should not need a build step at start, the result should be in the baked image. Else you become reliant on fetching packages during build etc.
"Builds" are the same as deploys, so when working with server(s) instead of larger scale deployments, I'm not seeing the benefit of the whole "build image, pull on server" pipeline when I can just ditch the registry and added layers by doing those steps on the server as I would normally in other kinds of scenarios.
But I have seen this in action, which is why I'm wondering if I'm missing something.
The clearer benefit to me seems to be in this scenario to use it as a fast environment provisioning tool.