Hacker News Reader

Really long-term task benchmark showing significant improvements in very recent models, while also showing really bad regression rates across the board.

View on HN

Uh, Opus 4.6 avoids introducing regressions 75% of the time?

So 1/4 times it does not introduce a regression. That's still pretty bad imo. If 1/4 commits introduced regressions, what would your team do?

We are talking about regressions, what once worked no longer does, and should be measured in 9s

You overestimate many teams I think.

Reply on news.ycombinator.com