I guess you'd have to worry about malware breaking out of the VM as well, but are there many exploits that do that these days?
That's hard to say, but both VirtualBox and VMware were hacked this year[1]. Definitely most secure would be both dedicated PC and running everything facing outside from VM. (and using SELinux or AppArmor as well).