You give them too much credit; check out twitter.com/GSElevator for typical banker humor. Bankers pride themselves on borderline-sociopathic cynicism mixed with arrogance and workaholism. It's not pretty.
Not that I'm saying they're all like that. But they're not all arrogant or cynical. Maybe workaholics.