There's a lot of revisionist history around what AWS allowed.
RackShack and other companies offered pretty cheap (started at $99/month IIRC) dedicated Linux servers which loads of startups used before AWS. There was also various virtual machine providers in the market.
These articles make it sound like every company before AWS had to build their own datacenter or at least buy a million dollars worth of Sun hardware to put a website online.