Brick and Mortar retailers are concerned about being able to conveniently accept payment and minimizing service charges for the convenience of accepting non-cash payment.
Ideally retailers wouldn't have to accept cards, they would only accept cash, cash = no service charge by a third party. However, that is the whole point of the financial industry to create an economy that permits middle men to take a percent of every transaction (as you say "grow thanks to your hard work"). Why have a store and sell anything when you can have someone else do the work and get a % of every transaction? But Amazon is not the first to do it.