Because this is just simple substitution of common math/CS functions.

One concern is that coding/logical puzzles are verticals where LLMs have lots of training data, they require small context window, and that's why they are doing well, but they don't necessary scale/generalize on other topics. For example I yet to see agents which would grab say Postgres codebase from github, and add untrivial features, send patch which is accepted.

I think the usual bearish sentiment comes from:

1. The marketing (even "research" papers) vastly overstates these systems' capabilities

2. We've observed unacceptable failure rates for the majority of useful tasks

3. Despite a bunch of money being poured into it, failure rates are decreasing slowly even with exponentially more training and inference resources, and there are precious few tasks where we've arrived at an alternative reframing making those errors tolerable

4. The capabilities are increasing slowly, and it's not obvious that pouring resources into the problem will lead to the phase shift necessary to either overcome (3) and (4) or to generate some novel application that makes it all worth it

Almost every attempted application I've seen has "worked" by using the AI to push externalities to other parties. It hasn't actually made the composite system meaningfully more efficient, with notable exceptions (e.g., the chat interface, once you learn the tool, if you use it correctly, can meaningfully improve certain kinds of productivity).

Plus, for people who actually work in AI, the current approach is obviously wrong. There are roughly two sorts of architectures in use for variable-width data: recurrent networks which store everything and process everything (e.g., transformers), and recurrent networks which compress everything and process the entire compressed space (e.g., mamba), with a special mention for compression-based architectures which have a large enough state space that compression is unnecessary, and another special mention for slight variations yielding constant speedups like MoE. The problem with all of those ideas is that you're necessarily trading off accuracy for compute. To have sufficient accuracy on arbitrary problems, you _must_ store everything, and you _cannot_ meaningfully compress it. The thing that isn't necessarily required though is processing everything at every iteration (a direction MoE works toward), and past a certain scale, unless we get vastly more efficient chips in a hurry (even if we do, depending on the target application), it's obvious that you need a new architecture to build around.

And so on. I work in AI (old-school ML for $WORK, new-school at home). I'm more bullish than a lot of people, especially since I see a direct cause and effect in things that make the business and my life better, but I'm still skeptical of the current craze. At a minimum, it's one of the most wasteful ways we could possibly do this research, and society isn't yet in a good place for the meager results we're getting. New-school AI is, currently, a net-negative to society, and I don't see that changing in the next several years.

Because it is reassuring.