The costs are distributed evenly, in the form of the markup on all products that covers the interchange fee and losses to charge backs.
You do pay them, but in small undetectable increments all the time instead of random surprise large losses.
You do pay them, but in small undetectable increments all the time instead of random surprise large losses.