I also had severe performance problems with VirtualBox on Windows but they were magically fixed by increasing the number of virtual CPUs from 1 to 2. I am talking 10-20 times performance boost just by doing that and nothing else. Maybe the same will help you. I blogged about it here: http://thoughtflow.dk/2012/12/07/prevent-slow-linux-performa...

VirtualBox, with 2 CPUs and 4GB of RAM for each OS. There are some CPU settings you need to change in VirtualBox, I don't remember them now but experiment, it made things 10x faster.