Well, obviously I can afford it because it's a small marginal cost; waiting for him to cough up the required sum is quite bearable.

And you lose because contracts under duress are unenforceable.

It's not so simple. Duress is where someone forces another into a bad position, eg by setting the house on fire first. A contract made under necessity (where the house is already on fire) will probably stand, unless it's for some wildly disproportional amount.

But in any case, a contract is not always required. In cases where one party knowingly accepts the beneficial services of another, they can be billed for it as if a contract existed.