This is main objection to e.g. negative returns on deposits. "it just isn't done". You do that, people will have zero faith in banking system. Tell you, living in society where nobody has faith in institutions is very unfun.