We need results from these harder/different benchmarks which give pretty bad scores to current top LLMs.

https://www.wolfram.com/llm-benchmarking-project/

https://help.kagi.com/kagi/ai/llm-benchmark.html

Edit : There are few other benchmarks that give pretty low scores (<20%) to top LLMs. Can't find them atm. There was a benchmark with common sense easy looking questions.

Edit: found two more papers

https://arxiv.org/html/2405.19616

https://arxiv.org/html/2406.02061v1

Edit: How about Wordle?

https://www.strangeloopcanon.com/p/what-can-llms-never-do

https://news.ycombinator.com/item?id=40179232