I don't at all disagree with you, but at the kind of money you'd be making at an org like OAI, it's easy to envision there being a ceiling, past which the additional financial compensation doesn't necessarily matter that much.
The problem with the argument is that most places saying this are paying more like a sub-basement, not that there can't genuinely be more important things.
That said, Sam Altman is also a guy who stuck nondisparagement terms into their equity agreement... and in that same vein, framing poaching as "someone has broken into our home" reads like cult language.