I can't see in the raw HTML any indication that you're using caching; when you have a spare minute, I'd advise installing the W3 Total Cache plugin. Going forwards, if you're not using nginx already then I'd highly recommend making the switch - in combination with caching it'll make things like this a thing of the past on a 512MB box :)