At least with systemd caddy starts as a non-root user, if you use the provided unit-file:
https://github.com/mholt/caddy/tree/master/dist/init/linux-s...
Give the caddy binary the ability to bind to privileged ports (e.g. 80, 443) as a non-root user:
sudo setcap 'cap_net_bind_service=+ep' /usr/local/bin/caddy