It's important to distinguish between things designed to take advantage of poor people, and completely reasonable penalties/structures that happen to affect poor people more because, not surprisingly, it's hard not to have money.
20% interest is not inherent a bad thing and shouldn't be outlawed out of hand. If you lend me $20 today and in exchange I pay you back plus buy you a beer on Saturday, that's an astronomical interest rate.
I co-own a brick and mortar business and we charge the maximum late fees allowed by law because people paying lay absolutely destroys our cash flow. It's not because we actually want the late fees - we'd much rather have people pay on time and be able to accurately plan for less money than have an influx of onerous fees. It's not uncommon for a profitable business to fail because cash flow is a problem.