HNHacker News
TopNewBestAskShowJobs

PollardsRho

60 karma · joined June 25, 2024

submissionscomments
PollardsRho··on Claude 3.5 Sonnet Reproduces BIG-Bench Canary String
The point isn't that it's a secret, it's that it's a string that shouldn't appear in any context besides the benchmark. It's an easy way to identify contamination in datasets. People often don't explicitly repeat them because that leads to false positives.
← PreviousPage 2 of 2