Asking the LLM if it "understands" and only proceeding if it says yes feels very weird to me. Do we really expect the LLM to be able to introspect in that way and give a meaningful answer?
What I am doing in that prompt is ensuring the LLM can follow the instructions. I specifically ask it to write "yes" if it does. If it can't do that part, then I don't want it to even generate code or try to analyze my data.
Hence why I treat it as an assertion failure if it can't follow that instruction, and thus exit the app.
Thanks for reading the code + for the very thoughtful question.
Sounds a lot like humans