DeepMind to OpenAI and everyone else[1]:
> your hyperparameters are bad and you should feel bad
It's amazing to me that such a big goof was missed by so many for so long. All these multimillion dollar language models and people just took the scaling laws at face value.