This perfectly echoes my own thoughts. The advances being trumpeted in AI are functions of hardware advances that allow us to have massively overparameterised models, models which essentially 'make the map the size of the territory'[0], which is why they only succeed at a narrow class of interpolation problems. And even then nothing useful. That's why we're still being sold the "computer wins at board game" trope of the 90s, and yet somehow also being told that we're right on the verge of AGI.
(OK, it's not only that. There's also a healthy amount of p-hacking, and a 'clever Hans effect' where the developer likely-unconsciously intervenes to assure the right answer via all the shadowy 'configuration' knobs ('oversampling', 'regularisation', 'feedforward', etc). I always say: if you develop a real AI, come show me a demo where it answers a hard question whose answer you - all of us - don't already know.)
[0] Or far larger, actually. Google the 'lottery ticket hypothesis'.