Some skepticism is healthy, but I think you have a misconception about what an LLM is. Neural networks use machine learning to automagically find connections between data. Handwriting recognition is an example of this, as it's trivial to build a model that can recognize the differences between characters/digits, whereas writing an algorithm to do that effectively would be very difficult. You can train a model on sample data, and it will "learn" the patterns that make up a '7', 'G', 'O', etc with the same or better accuracy than a person.

Language models are more or less the same thing, except they're trained on human language. Since human language is an encoding of human thought processes, then the model learns the patterns of human thought which are embedded in text, and can both generate and recognize them.

This is why LLMs can "understand" and answer complicated questions that don't exist in their training set, and do things like debug code. Even though it never saw that exact question, it recognizes the high level/abstract concepts that make up the question (since those abstract concepts exist in various different forms in the training set).

So giving an LLM a textual description of an algorithm, and having it detect logical errors in it, is not that unbelievable. Whether current models are good enough to do it well is the question to ask.