Why can't you do business with cash/money orders and run a website on your own hardware exactly?
They could also accept checks via the mail. Or cash via the mail. Or cryptocurrencies. Or gift cards. Or barter. There are multiple options available to them.
The only thing you mention there that is a serious alternative are cryptocurrencies. And those are constanly fluctuating, needing very complex hedging against sudden value changes. And still something that most consumers will really struggle with. Most will not have a clue how to obtain crypto or how to deal with it (safely).
It might work for a really highly educated niche, but not for 99% of consumers. They just want to put in their paypal or credit card details and click buy now.