On Linux, I use a shell script like this:

   servers=( ip1 ip2 ip3 ... )
   
   # Can fwd over internal network
   iptables -A OUTPUT -s 192.168.0.0/16 -d 192.168.0.0/16 -j ACCEPT

   # Can fwd over loopback
   iptables -A OUTPUT -o lo -j ACCEPT

   # Can fwd over the tunnel
   iptables -A OUTPUT -o tun0 -j ACCEPT

   # Can send packets to VPN
   for server in "${servers[@]}"; do
      echo "Installing rules for $server"
      iptables -A OUTPUT -d $server -j ACCEPT
   done

   # Otherwise drop
   iptables -A OUTPUT -j DROP

Use it with a package like iptables-persistent so you don't have to run this every time at boot.