Do you have any more information about how you're doing this? Whenever I've tried to use Docker as a remote development environment the process felt very foreign and convoluted.
If you have a secure network then it’s perfectly fine to expose the docker port on the network in plaintext without authentication.
Otherwise you can use Port forwarding over SSH.
To set up networked docker you can follow this: https://docs.docker.com/engine/security/protect-access/
I’m on the phone so can’t give a detailed guide.