The fact is that no neural network can solve sudoku puzzles. I think it's hilarious that AI proponents/detractors keep worrying about existential risk when not a single one of these systems can solve logic puzzles.
To me the existential risks are pretty boring and current LLMs are already capable of them: train on some biased data, people embed LLMs in a ton of places, the result is spreading bias in a black box where introspection is significantly harder.
In some ways it mirrors the original stochastic parrot warning, except "parrot" is a much significantly less loaded term in this context.