Payment networks are where the network effects and scale are. Visa and Mastercard have been known to ban people for political reasons.

Payment providers are just a front end for their networks, so it's easy to switch providers like Stripe, but you can still be banned by underlying network like Mastercard and your business will suffer.