I am having the same issue now with the Cloud compute servers at the Tokyo region.
In terms of a workaround, it seems like a server created without an IPv6 address works fine and assigning an IPv6 network to the server causes the issue again for me.
So I guess that something is going wrong with Vultr's network configuration at this point.