Hacker News Reader

Comment by Esras | Hacker News Reader

But agents don't _know_ why they gave an answer. They can only give "reasoning" that links to something in their context, and even then, you would have to parse out their response with some heuristics to try to match against something upstream of that turn.

I could see it being done, and if you're fond of the "models all the way down" mode of thinking, you could use a smaller model to identify it, but it could just as well be a "load-bearing seam" (ha) for something else in the conversation.

View on HN

I’ve struggled with this line of thought. In a strict sense, it’s true; an AI model doesn’t actually know why it fired the way it did. But I don’t know how practically useful that observation is because, using that strict definition of “know”, humans also do not know why they do what they do.

If I asked you why you posted this comment, I suspect you’d be able to articulate a good reason why from your memories. But the firing of your subconscious and the entirety of your lived experience that molded the state in your brain that lead to you pressing that reply button and writing those exact words are inaccessible to you.

It reminds me of the timeline exercise in SRE work. I always start with a timeline and plunk pagers, metrics, slack messages, etc onto that timeline first. Then I go to individual people to get their understanding of the incident and get that into the timeline. People’s understanding of the incident and why they did what they did rarely matches the timeline.

When I ask an LLM to review its reasoning over prior conversation turns, it has access to its tool call results, web searches, and reasoning traces that aren't visible to me in the UI.

There's definitely value in the "meta-cognition" that an LLM can do even if it is blind to the underlying weights and layers of the model itself.

I think we need to distinguish between "what the model received" and "why the model generated this answer." ThoughtDAG currently focuses on the former: accurately displaying the context of the incoming request and allowing users to modify it.

Sometimes it’s just an awkward turn of phrase on my part that creates a wrinkle in the conversation. Sometimes agents identify that, and we can work together and direct that, but correction itself eventually loses competition to the original error.

I agree.

Most LLM tools (Claude Web, OpenAI, and their harness) offer re-editable questions. That is how I avoid such problems by myself.

In ThoughtDAG, you can re-edit questions by double-clicking the question. Or edit the answer by clicking the edit icon at the end of each answer text. Or.. you can just remove the connection or delete the node.

That would give you manageable context

Reply on news.ycombinator.com