Hacker News Reader

Comment by manmal | Hacker News Reader

One weird trick is to tell the LLM to ask you questions about anything that’s unclear at this point. I tell it eg to ask up to 10 questions. Often I do multiple rounds of these Q&A and I‘m always surprised at the quality of the questions (w/ Opus). Getting better results that way, just because it reduces the degrees of freedom in which the agent can go off in a totally wrong direction.

View on HN

This is more or less what the "architect" mode is in KiloCode. It does all the planning and documentation, and then has to be switched to "Code" in order to author any of it. It allows me to ensure we're on the same page, more or less, with intentions and scope before giving it access to writing anything.

It consumes ~30-40% of the tokens associated with a project, in my experience, but they seem to be used in a more productive way long-term, as it doesn't need to rehash anything later on if it got covered in planning. That said, I don't pay too close attention to my consumption, as I found that QwenCoder 30B will run on my home desktop PC (48GB RAM/12GB vRAM) in a way that's plenty functional and accomplishes my goals (albeit a little slower than Copilot on most tasks).

Workflow improvement: Use a repo bundler to make a single file and drop your entire codebase in gemini or chatgpt. Their whole codebase comprehension is great and you can chat for a long time without the api cost. You can even get them to comment on each other's feedback, it's great.

Unfortunately that only works with very small projects.

I can get useful info from gemini on 120k loc projects with repomix ignoring a few select files. If you're in the enterprise world obviously it's a different thing.

I often like to just talk out out. Stream of thought. Gives it full context of your mental model. Talk through an excalidraw diagram

I do that as well, with Wispr Flow. But I still forget details that the questions make obvious.

This is a little anthropomorphic. The faster option is to tell it to give you the full content of an ideal context for what you’re doing and adjust or expand as necessary. Less back and forth.

Can you give me the full content of the ideal context of what you mean here?

Certainly!

It’s not though, one of the key gaps right now is that people do not provide enough direction on the tradeoffs they want to make. Generally LLMs will not ask you about them, they will just go off and build. But if you have them ask, they will often come back with important questions about things you did not specify.

They don’t know what to ask. They only assemble questions according to training data.

While true, the questions are all points where the LLM would have "assumed" an answer and by asking you get to point in the right direction instead.

It seems like you are trying to steer toward a different point or topic.

In the course of my work, I have found they ask valuable clarifying questions. I don’t care how they do it.

This is the correct answer. I like to go one step further than the root comment:

Nearly all of my "agents" are required to ask at least three clarifying questions before they're allowed to do anything (code, write a PRD, write an email newsletter, etc)

Force it to ask one at a time and it's event better, though not as step-function VS if it went off your initial ask.

I think the reason is exactly what you state @7thpower: it takes a lot of thinking to really provide enough context and direction to an LLM, especially (in my opinion) because they're so cheap and require no social capital cost (vs asking a colleague / employee—where if you have them work for a week just to throw away all their work it's a very non-zero cost).

My routine is:

Prompt 1: <define task> Do not write any code yet. Ask any questions you need for clarification now.

Prompt 2: <answer questions> Do not write any code yet. What additional questions do you have?

Reiterate until questions become unimportant.

Reply on news.ycombinator.com