The problem is simply that people don't read this stuff in the first place. They simply sign without reading, assuming/hoping/praying that the terms are reasonable.
And really, can you blame them? Contracts are so vastly overused and they're made with the assumption that the consumer isn't going to read them. Most companies won't give you a two-page contract if a twenty-page contract will do. Many contracts are excessively long and are part of a sales process built on speed that assumes nobody will take the time to read before signing.
I would like to see a change in how contracts are handled, such that if one side knows the other side didn't read before signing, the contract is void. That wouldn't take care of everything (American Express's contract is probably handled by mail, so they can reasonably expect you to read the contract at leisure in your home before you sign) but it would put a stop to abusive situations in retail, where they had you a bunch of paperwork to sign. If they had to actually watch you and make sure you read it all before signing, the contracts would probably become a lot shorter. If people got used to reading the things, then it might even change their habits for other scenarios like doing stuff through the mail.
However, I also think it is completely insane that contracts are allowed to put any restrictions on either party's access to courts. The whole point of courts is to be the arbiter when something goes wrong. A clause saying that you must use some third-party arbitration service instead of the courts, or a clause saying that you agree not to participate in class-action lawsuits, should be completely unenforceable, just like a clause that says you agree to become the other party's slave.