You need to let the AI cook | Hacker News Reader

You need to let the AI cook

ivan.codes

25 pointsivandotcodes5 comments

> Letting the model run feels like giving up control, while inventing a workflow feels like the responsible move, even when it produces worse work in more time.

It feels to me like the discretion of the humans is the main problem in your solution space example.

Some tasks require more structure than others. Sometimes, the constraint a human faces is “you must do the work in this module or on this function”.

If the prompt says “only change this function” then the human has a good reason for that. If the LLM can’t figure it out then letting them cook on it feels like a waste of my time and tokens.

Too often I’ve sent Opus or Fable now on what I thought was a simple task to find it chasing its tail.

Fortunately I find more success than tail-chasing with the LLMs but I’m dubious of any suggestion to just let them cook.

However, a solution to the tail-chasing for certain could be to intentionally expand the solution space and to consider how my prompts might be restricting the LLM’s options.

Enjoyed the blog thanks for sharing!

I don't agree. The time loss spent on optimizing prompts is well documented to be a major efficacy loss. This is why studies confirm use of AI slows down development and increases expense.

I agree, don't pre-emptively put too much control over your agents. But they do need guidelines and guardrails. All of the constraints I've added on my agents, I've added because when I let them cook, they went down the wrong path. Examples:

* Don't do ad-hoc fixes. Don't mutate the production data just to fix an issue. Fix the bug, ingest the bad data in code if necessary, or write a migration/repair script that an ops agent can run. * Write infrastructure as code, don't rely on remembering complex manual workflows. * Make sure there's a single model in the app with a single source of truth. Don't let truth get spread out across representations, it will get out of sync and bugs will result. Derived truths can be projected from the single truth, but they must update when the truth updates. * Write all logic in the application server, and send only a minimal visible payload to the web client. No calculations, except those needed for rendering the view, should happen in the client.

It goes on. These are all driven from the model making mistakes and having to have it go back and refactor to correct them.

I suspect that a lot of these “you need to guide the AI” rules worked better in the past when the models were dumber.

Now less is more.

>Both work because the priming step alone ticks a bunch of boxes the over-prompter is trying to hit one at a time. Once the agent has read the reference it already matches the conventions, the naming patterns, the way functions are structured, and how errors get handled, all the stuff people otherwise try to spell out constraint by constraint.

This only works either if you are living in the future where AGI already happened, or if you're trying to work with extremely mundane things. My experience is the complete opposite, current models produce hot garbage when trying to familiarize themselves with the codebase. That's precisely the reason I'm trying to correct them by adding manual constraints. Not vice versa. Predicting what's going to be important and what's not seems to be a major bottleneck for them. They're easily overwhelmed.

Reply on news.ycombinator.com