pampas

What's interesting is how well the models know the exact wiki text. If the wiki excerpt is rewritten the pattern matching doesn't work quite as well. Gemini is only able to one-shot about 1/3rd of the time which is still very impressive considering not long ago LLMs couldn't count the letter 'r' in strawberry.

In my niche Redactle puzzle solving benchmark [1] I noticed Gemini 3.8 flash is slightly faster than 3.7 flash. They both smoke every model I've tested. I have not yet run 3.5 flash. Gemini models are great at this task because they seem to have exact Wikipedia text baked into the weights. When I rewrite the wiki text a bit it's not able to one-shot the game so much.

[1]: https://redactle.net/llm-leaderboard

Gemini 3.7 Flash was already smashing more expensive models on my Redactle benchmark https://redactle.net/llm-leaderboard which mostly tests omniscience.

It's the modern "Never argue with a man who buys ink by the barrel". I think it's why outdoor advertising is so rife. It's hard to fight something when you rely on it to keep your position.

Gemini 3.8 Flash is top of the Redactle LLM benchmark but so was Gemini 3.7 Flash. Both one shot all puzzles in the evals though 3.8 is just a bit faster. It also does the evals cheaper and faster than almost all the other models I've tried.

https://redactle.net/llm-leaderboard

I made a 3D browser game a few months ago where you play as a pelican riding a bike via a bluetooth trainer.