By running everything through the VPN, you'd be able to have TCP connections that didn't break when the network switched, since your device's public IP address would be in a datacenter somewhere.
Also with a VPN you'd be able to send voice traffic over both a carrier connection and the wifi connection at the same time to avoid dropouts.
There is something similar called Multi-path TCP (MPTCP) which uses latency to decide which TCP path to send traffic over.