Ask HN: When is $20 VPS not enough?

I know this question is vague, so if someone can share their stories like when they started out, what was their setup and what was the point when they had to think about scaling up.

I have an RoR app which I am about to launch in beta. The response time/load time is somewhere between 50ms to 300ms(fluctuating). I would like to know how soon I might have to think about scaling up(hardware).

Is going for dedicated hardware initially a good idea? My guess is that will reduce the load time further.