HN
Hacker News
Top
New
Best
Ask
Show
Jobs
Comment by saberience | Hacker News Reader
Parent
Full thread
saberience
·
SWE Bench Pro is also gamed and shouldn't be trusted.
View on HN
mnicky
·
Then we are left with what? FrontierCode maybe? IIRC that one evaluates not only if tests pass but also code quality - e.g. whether the maintainer would accept the pull request as is.
Sabinus
·
Why that benchmark in particular?
Reply on news.ycombinator.com