The answer to that would be to make a builder image and copy out the installed files. That makes a smaller container anyhow so has several advantages.