Hacker News Reader

Chain of thought is essentially a recursive architecture. In its current form it a way to "debug" the reasoning process. This moves the cot process back into the transformer itself, thus never being exposed. Like trying to find a bug in a recursive function that has no logs or breakpoints.

My (layman) understanding is, that currently there is a way to monitor the "thoughts" of the LLMs and that by looping more you lose that ability. The danger is presumably an AI that escapes human oversight.

Why not click on the link above?

Beyond safety concerns, it’d be sad for working users to lose some ability to understand and steer thinking, too. These are tools for us, after all.