The thing about AWS is that it's basically a one stop shop for everything you need infrastructure wise. Many startups can avoid having specialists for infrastructure in the early stages if the default choice is "use AWS" for all infrastructure decisions.
Choosing not to go the AWS route might mean having someone on staff in the early stages who is dedicated to systems management and engineering. Not every early stage team has access to someone like that so that is the main draw with AWS.
Other cloud providers still haven't caught up to the richness of AWS's offerings, although Google and Microsoft are starting to come close.