>> With our advances in long-term reasoning and planning, Devin can plan and execute complex engineering tasks requiring thousands of decisions.

They'd better have really advanced reasoning and planning capabilities way beyond everything that anyone else knows how to do with LLMs. There's a growing body of literature that leaves no doubt that LLMs can't reason and can't plan.

For a quick summary of some such results see:

https://arxiv.org/pdf/2403.04121.pdf