> And unfortunately I can't leave them because it is a loan and not just a checking account.

You may be able to refinance the loan through another bank, which would pay off this one and let you go with someone more reasonable.