SWE bench scores, like a lot of other metrics for LLMs, are pretty divorced from reality IMO. It's a lot like only learning to pass tests vs actual understanding.

Once GenAI companies stop hiring SWEs, I'll believe the doomers.