A majority of credit card fees goes to the issuing bank. Like a huge percentage of it. Visa and credit card processors take a much smaller percentage relative to the banks.
We have a bizarre system where retailers mark everything up an extra 2%, to give their more well-off customers (that pay with decent credit cards) an effective 2% discount and the ability to dispute charges.
I think retailers would love a new system with less transaction fees, but banks definitely would not, and some customers may not either if it means sacrificing their credit card rewards and perks.