If you can crank out some moderately complicated algorithms, even having rehearsed them, and get all the little details right, like all the cases that occur, that does show you kind of know what you're doing.
Someone without the proper CS/programming background who is just trying to rote memorize will screw up the details, and not be able to explain anything if asked about it: like what is the purpose of that line of code?
Only problem is, even if this interview question is good at testing that which it is testing, it might not be selecting people along the right dimensions in terms of being suited for the job. It's not probing all the attributes of a software engineer.
Five candidates, A, B, C, D and E have done equally well on the exercise, excellent in fact, and are hired.
Yet, in actual work, things go funny.
A doesn't seem to believe in testing. Deliveries are sloppy and break things all the time. A is proud of her skills and treats criticism of their code as an insult.
B has never heard the words "version" and "control" in one sentence. B works on an out-of-date version of the code to make a change, and then follows the Wiki instructions for getting a fresh checkout (which pulls the latest baseline) and then just copies their previously changed files into the new checkout. Their code submission therefore reverts numerous changes from other people (incompletely: just in those files), while asserting their own changes. On top of that, B makes gratuitous whitespace changes, doesn't follow any consistent coding standard (never mind the prescribed one), and randomly fixes things he sees wrong that do not relate to the topic of the change.
C cannot handle changes in requirements. Even after code base has undergone many changes, C keeps thinking in the same concepts from when the code was two months old and C had been mostly working alone on that code, knowing everything about it. C's ideas about making changes no longer fit the new code. E.g. C cannot remember that some logic is now distributed, running in several places at once, because that's not how it once was, and consistently forgets that aspect.
D cannot say no to any new task. Whenever someone talks to D, D gets distracted and excited and starts looking into what that person was talking about, and doing work on it. Whenever D gets stuck on something, D easily finds something else, and forgets about being stuck, or to report having been stuck. D leaves numerous tickets in an in-progress state, and that list just grows over time.
E is simply lost in any large project. E has done nothing but programming challenges under 1000 lines of code. E has no skills with regard to navigating a large code tree and understanding the structure of something made of tends of thousands of lines (or orders of magnitudes more). E gets paralyzed when faced with making a change to a large system that will involve making some small changes to eight or nine files. But have you seen E's Tetris in five lines of code?