The current generation of LLMs will never generalize on problems whose expected complexity is significantly harder than the training set. One way to think of why this is the case is because transformers are learning shortcuts to multistep reasoning tasks [1, 2], but do not learn to reason systematically and unsurprisingly fail to generalize when presented with more challenging tasks.
The proper way to evaluate systematic reasoning is by training on a curriculum of problems of increasing average-case complexity. Think random k-SAT in the critical region, bisimulation games, or interpolation in certain logical theories. Although there is a growing evidence that Transformers and GNNs have foundational limits to expressivity [3, 4, 5], if the scaling crowd could show strong generalization results on these sorts of problems they would have the last laugh...
[1]: https://arxiv.org/pdf/2210.10749.pdf
[2]: https://arxiv.org/pdf/2305.18654.pdf
[3]: https://arxiv.org/pdf/2106.16213.pdf