I'd make systemd unit files around the docker-compose deployments. Upload everything with something like Ansible. If you want to go perfect go for NixOS instead, but that might be controversial.
I’d also suggest Podman as a Docker alternative, especially for these scenarios. Images and Dockerfiles (and many commands) are 1-to-1 compatible but there are some nice things especially with this scenario.
I wouldn’t say NixOS is controversial, at all, but it’s a very steep learning curve.
https://docs.docker.com/config/containers/start-containers-a...
So if you have proper Docker installation, containers will restart on reboot.