It's like arguing that punitive fines for minor infractions, like jaywalking or speeding, would be justified because "you broke the law".
The outrageous maximum penalties ($100k/infringement), the nuance of fair use and copyright law, and the resulting cost of defending yourself means that the only solution if you don't have excellent in-house counsel is you have to settle for what they ask. It's legal extortion.