Although I believe it, I wish there was some observability into what data is included here.
Both Sonnet and Opus 4 say Joe Biden is president and claim their knowledge cutoff is "April 2024".
Both Sonnet and Opus 4 say Joe Biden is president and claim their knowledge cutoff is "April 2024".
The web interface has a prompt that defines a cutoff date and who's president[1].
[0] https://console.anthropic.com/workbench
[1] https://docs.anthropic.com/en/release-notes/system-prompts#c...
People use "who's the president?" as a cutoff check (sort of like paramedics do when triaging a potential head injury patient!), so they put it into the prompt. If people switched to asking who the CEO of Costco is, maybe they'd put that in the prompt too.