The technology exists to do all of these things, but no one has taken the time to glue it all together in a truly good UX (I'm working on it). Pretty much every solution in this space is targeted at the developer market, not self-hosters.
So for now I'd recommend using a VPS. Your main challenge is going to be learning a lot about security. There's currently no way around that. A VPS limits the scope of damage that can be done if you get hacked. Once you've learned enough you can move to your own hardware. At that point I'd recommend setting up tunneling[0] and using either Docker or QEMU/KVM.
EDIT: I see you're already using Tailscale. That can operate as a tunnel. Basically you'd want to run a reverse proxy like Caddy (recommended) or nginx on the VPS, and point it at services running on your other devices using the IP addresses from your Tailscale network.