All that work you did to make your IO sequential goes to waste as soon as you put it on a VPS. It will be interesting to see if SSDs overcome the typical VPS limitations.
Basically, CPU time is an abundant resource that's easily partitioned, but can be shared. So, you're guaranteed your share, but you often actually get more than your share.
Memory is strictly partitioned. You always get your share, no more, no less.
Disk is almost impossible to partition fairly (performance, not space). Furthermore, the more active disk users there are the worse the total performance is, so if you're unlucky enough to have a neighbor that uses a lot of disk IO than you effectively get double screwed.
What would be interesting is if someone came up with a VPS with dedicated disk resources, but as far as I know no one has done that.
I strongly suspect this is a platform-specific complaint. As in, if you are running your Linux VMs on a z-Series from IBM this is manageable. It's just a question of what your customers are willing to pay for...
One of the nice things about Xen is that disk I/O can be controlled somewhat because each domU has a specific process in dom0 for disk access. So you can ionice things & provide a somewhat more controlled access to the disk.
My experience? it's easy to make one customers's I/O suck with such tools, but they work for shit when it comes to fairly balancing everyone's I/O.
As far as I can tell, the best practice is to monitor everyone's usage, and if someone goes above some pre-set threshold, to make their I/O suck using ionice or what have you.
But that's a long way from solving the problem.
Many websites have production rigs involving DB machines with a lot of memory, web workers that rarely do IO and files stored in CDNs or S3 so the disk problems are less of an issue.
As you point out though when your dataset fiscally or practically won't fit in RAM then disk IO catches up to you and I'd certainly appreciate an SSD's random seek time.
It's also important to remember development becomes an entirely different beast with cloud infrastructure though. Even the best tested and reviewed code can't approximate all the variables in production so being able to launch an exact mirror of your production rig and throw recorded traffic at it becomes an amazing test. This is something not feasible in the past.
Why? You can run only one VM inside each hyprervisor. Or you can run only one IO intensive guest.
I hardly see any reasons to run my servers on hardvare instead of VMs in 2011.
Also, the article discusses VPS systems which are generally much different than installing a VM on hardware you own.
I'll admit that the last time I tried to run an IO intensive task on a VM it was about a year and a half ago. (VMWare ESXi)
I think the processors installed in that box didn't support IO virtualization.
My be I'm wrong but I don't understand where I could loose performance.