The bitter lesson is about the balance of human ingenuity and compute being thrown at the problem. We've seen a few years of LLM compute being scaled up 10x every year, but this is hitting limits (fabs), and we will see more human effort, as it becomes compartively cheaper.

Also the current crop of models are inherently limited. Even for something as simple as following a JSON schema, models alone are not good enough [0]

Of course as the Moore law refuses to die, we'll continue seeing 1.5-2x or so every year, but that's far from 10x.

[0] https://openai.com/index/introducing-structured-outputs-in-t... - see plot