Hacker News Reader

If you were writing a promotional post for your new model, would you include benchmarks of a competitor that's spanking you across the board? This is marketing.

> Who are they trying to kid?

People who do not know how reproducible research works.

Any benchmark that is presented by AI labs must be reproduced reliably by someone else independent of that AI lab presenting these results.

Otherwise, not only it is biased, these numbers can be just made up for marketing purposes.

There aren't really any of the typical benchmark suites targeting Codex 5.3 because it's still not in the API.

SWE bench for example creates a predictions file and evaluates the results in the harness. Without Codex 5.3 being in the API, it can't.

gpt-5.3-codex isn't available via the API yet. Pretty sure they were only testing via API access.