The reason it's associated with morality is because the idea of wronging someone implies that the one who is wronged is owed something. By offering forgiveness, what was owed is remitted. This also applies in the idea of sin, an offense against that which is divine.

And it applies in the context of a loan, where one party is owed something by another.

So, in the concept of forgiveness, it is the owing that is important. Any implication of morality is incidental.

But, also, there are indeed moral systems that consider borrowing money to be wrong. Although that's uncommon, and perhaps you were arguing that those moral systems are incorrect.