So it's a private agreement? I don't understand your point. You agree to pay a certain amount and if you don't a bad thing happens. When it's a bank and a mortgage, you lose your house and your family is harmed. When it's the government and taxes, your wages are garnished and your family is harmed.

Having a banking system helps people buy houses and having a government helps literally everything a society does.