Hm, i found this to be pretty straight forward through Docker:
e.g:
FROM caddy:2-builder AS builder
RUN xcaddy build \
--with github.com/greenpau/caddy-security
FROM caddy:2
COPY --from=builder /usr/bin/caddy /usr/bin/caddy
You can keep your container image, and just do the "COPY --from=builder" step in there. We do this in CI/CD on every build.