Hacker News Reader

Then explain to us how they game it? The test is blind. You don't know which models give the answer. If you trick them into disclosing their name, it is discarded right away and doesn't count towards the score.

Genuine question, because as far as I know there is no feasible way to game this score. But if there is, I want to know.

View on HN

> If you trick them into disclosing their name, it is discarded right away and doesn't count towards the score.

However that process works would be the thing to circumvent, like by getting it to disclose some letters of its name, or any value that is different between models but the same or similar for each one. I assume it doesn't provide the numerical token values in the output or it would be trivial.

Reply on news.ycombinator.com