Hacker News Reader

Comment by jerpint | Hacker News Reader

I think there will be lessons learned here as well for better agentic systems writing code more generally; instead of “committing” to code as of the first token generated, first generate overall structure of code base, with abstractions, and only then start writing code.

I usually instruct Claude/chatGPT/etc not to generate any code until I tell it to, as they are eager to do so and often box themselves in a corner early on

View on HN

aider.chat has an /architect mode where you can discuss the architecture first and later ask it to execute the architectural decisions

works pretty well, especially because you can use a more capable model for architecting and a cheaper one to code

I didn't know about this, thanks for sharing

This is literally chain-of-thought! Even better than generic chain-of-thought prompting ("Think step by step and write down your thought process."), you're doing a domain-specific CoT, where you use some of your human intuition on how to approach a problem and imparting the LLM with it.

We already have languages for expressing abstractions, they're called programming languages. Working software is always built interactively, with a combination of top-down and bottom-up reasoning and experimentation. The problem is not in starting with real code, the problem is in being unable to keep editing the draft.

Not a problem with the correct tooling.

I agree.

On the other hand, I expect that programming languages will keep evolving, and the next generation or so might be designed with LLMs in mind.

For instance, there's a conversation in the Rust's lang forum on how to best extract API documentation for processing by an LLM. Will this help? No idea. But it's an interesting experiment nevertheless.

Yes I frequently do this too.

In fact I often ask whatever model I’m interacting with to not do anything until we’ve devised a plan. This goes for search, code, commands, analysis, etc.

It often leads to better results for me across the board. But often I need to repeat those instructions as the chat gets longer. These models are so hyped to generate something even if it’s not requested.

That's exactly what I've understood, and this becomes even more important as the size of codebase scales.

Ultimately, LLMs (like humans) can keep a limited context in their "brains". To use them effectively, we have to provide the right context.

Reply on news.ycombinator.com