In my experience, scalability issues are often rooted in either not understanding your infrastructure, or not testing it. My WordPress blog survived a pretty heavy Reddit effect with no caching enabled and "only" 6 PHP processes. Similarly, my other applications seem to scale fairly well on VPSs with under 1GB of RAM. Knowing the infrastructure causes me to profile and optimize frequently.

My other issue with the cloud is the vendor lock-in. Google App Engine is pretty cool, but who else provides Big Table support? Amazon's datastore is similarly unique although AWS lets you run MySQL at least. Lastly, I found that it is cheaper to run your own VPSs. The AWS is at least twice as expensive for the same amount of RAM.