The downside is of course that Stripe takes over more and more of your important infrastructure. A question should always be how to include several suppliers and/or how to change supplier in order not to put all in your eggs in a single basket.
Nobody ever wanted to switch; the expertise in using one provider created immense inertia. And even when a few folks (fans of $other working side-of-desk) tried to make a proof-of-concept swap, it turned out that the code was anything but agnostic and dependent on the first provider it was built for in pervasive and fundamental ways.
So, too, with service providers and partner businesses. Spending time/money on being able to switch suppliers and keeping all your eggs from being in one basket is a waste of time: you won't switch voluntarily, and if you ever have to switch involuntarily it'll still be about as difficult as it would have been if you hadn't prioritized redundancy/diversity.
For some services it's also not just being able to switch, it's having more than one used in production. If you manage that then switching is no longer a problem since it's already implemented! That's important for key services.
Unless your provider really sucks, it's always important to evaluate carefully and ideally stick with them unless it is really that bad.
Yes, that's exactly my point: At some point you are completely locked in with a single supplier that holds your entire income stream and perhaps even more than that.
Ideally (easier said that done, I know) you want to have at least 2 suppliers for any key piece of infrastructure as early as possible and to avoid letting a supplier 'expand' the number of tasks they do for you too much.
The latter seems to be Stripe's strategy: They start with payments then expand step by step in everything related et even in things like company incorporation.