Until there is a critical vulnerability in one of the components present on that image (system packages or application packages).
Instead, build your artifact and publish it to an artifact repository, just like we used to.
_then_ wrap that artifact in a Docker image.
Vulnerability found in the docker image? No problem. Build a new image with the same artifact.
What you said about "wrapping" I interpret as: based on an image for instance with OS level dependencies you create another image with application level artifacts, e.g. a python application. When your app changes, you don't build the base image again, you only build the app image. This makes sense to me.
You could break the build simply by cleaning some things from the cache, without changing the Dockerfile, even thought that Dockerfile "succeeded to build"