You can, and many people do, live a no-credit-card lifestyle. Need something from an online store? Amazon and other retailers offer store accounts with direct billing (search for "Amazon.com Store Card") and more favorable terms.
If you find the practice shady, you as a customer can choose not to use a credit card. Use a debit card, or just pay cash. The problem is that customers like credit cards: it lets you buy stuff now!
Merchants would lose business if they didn't accept cards, and card companies were basically able to squeeze them until such loss of business was barely more painful than just going through with it.
A slight market correction came in the form of competition between card companies to give cash-back deals. This essentially rebated some of the fee back to the consumer (if an Ayn Rand nutjob saw a government bureaucratic scheme half as complex they would explode with anger).
People with cash couldn't opt out; they still had to pay the same as card users, but now those merchant agreements have been restricted by law.
Typically only in states that had their own law enforcing that restriction on merchant agreements.