Very easy, assuming the machine has external access which a VPS does
Install postgres
Set the postgres configuration - passwords, user accounts, port for external access
Set the firewall configuration - such as iptables or whatever is used on your distribution assuming Linux.
That's it, then you should be able to access it from external mgmt UIs and external applications.
It's also possible to setup whitelisted IP access which is a good idea for a database, so that only the IPs of the client applications servers are allowed to connect.
If your data is particularly high value/likely to be targeted, then you should consider further security steps. That can get more complex, but the above basics are fine for simple standard/low security applications. This is pretty much equivalent to what you would get from a cloud database service in standard configuration.