If a restaurant runs on a 9% net margin and pays around 3% in card fees, then roughly one-third of its net profit is going toward payment processing.
If anything, they are benefitted by accepting cards, since they get customers who purchase on credit. Or just in general because many people have less resistance towards making a card purchase compared to a cash purchase.