You start from a base image of your choice. You only build the user replacement part.

You run docker-compose build ONCE and you're set. On my machine, it takes five seconds.

Heck, you can even run docker-compose build everytime you start the application, it will use the cached build and take less than one second.

---

Correction: the docker-compose up -d takes care of the build process the first time it runs.

Literally, it takes more to complain about the issue than build the image ONCE.