This is not really true. The Chinchilla paper showed that a 4% difference in loss between Chinchilla and Gopher led Chinchilla to blow Gopher out of the water at most tasks, including 30x performance in physics.

Empirically, LLMs have shown to have emergent abilities appear at different loss levels. So, a 10% difference could really matter.

It's about causing your competition to waste millions of dollars in compute time and power doing something unproductive.

There is not a huge pile of excess TPUs laying around for people to use. Any strategic advantage can quickly compound and put you well ahead of others.