> when you are measuring performance metrics, they can be hell lot misleading
This is true regardless. For example, if your I/O benchmark is basically sequential, and your application's access pattern is basically random, then your benchmark is useless.
Ultimately, the only test that matters is whether a particular piece of hardware runs your particular application. So as a startup, you write your website, put it on a VPS, if it's slow then you get a bigger VPS or switch providers. The common case is that a VPS is good enough, and once you grow beyond it, that means you're getting enough revenue to afford an upgrade to a bigger VPS or dedicated server out of money coming in.
If you can't afford the upgrade when you need it, then it means either you're inadequately capitalized (you don't have enough investment to survive the bootstrapping regime where necessary server spending outstrips revenue), or your code is too inefficient for your business model to be viable (making $X in revenue costs more than $X worth of server resources, the "bootstrapping" regime has no end).
> 1-2 GB Ram
Upgrade this. RAM is cheap, and you get a lot of performance out of it.
> I haven't done the cost calculation
The available information at my fingertips says buying a full rack (40U), power included, costs $650 per month [1]. So that's $16.25 per 1U. That leaves $3.75 per month, or $180 total over a 4-year lifetime. Let's say you use a Raspberry Pi as the server, it costs $35. That leaves $145, or $3.02 per server per month, to pay for bandwidth, labor, billing/accounting, storage media, power cables, networking gear, and profit for the owner.
$20 is really aggressive pricing. $30 might be more reasonable.
If you have sunk costs, then you might be able to make it work. E.g. if you just signed a multiyear deal for a ton of rack space, but the project you were planning on using it for isn't going to fly due to your cofounder getting hit by a bus, you can regard the rent on server space as a sunk cost since you'll be paying for it anyway. Filling it with $20/month servers would almost certainly be better than letting it sit idle (though maybe not better than filling it with more expensive servers, subletting it to someone else, or having your company declare bankruptcy).