Last year’s model were at 50-60% on SWE bench-verified actually
Important point because people have a bias to underestimate the speed of ai progress.
Here’s the launch card of the sonnet 3.5 from a year and a month ago. Guess the number. Ok, Ill tell you: 49.0%. So yeah, the comment you replied to was not really off.