Hacker News Reader

Comment by julesrms | Hacker News Reader

It's all very glitzy, but I'm failing to understand how "One prompt in. One result out." is of any importance.

This and other recent AI hype-fests all seem to be obsessed with making agents do more work unattended.

But surely in the real world, anyone who's got a real product to make is going to want to steer what's happening. It's ridiculous to think that anyone with a deadline would write a prompt so perfect that they walk away for 4 days and come back to find the finished product ready to ship.

If you really can write a prompt so complete and perfect that it needs nothing further, then any regular harness could probably also do the job. But if like normal people you need to try something, think about it, iterate, and repeat.. then you also just need a regular harness.

View on HN

The Venn Diagram of people building the AI hype-fests and people building a real product is just two circles

> But surely in the real world, anyone who's got a real product to make is going to want to steer what's happening.

The promise of AI is that you won't need to pay people in order to think. A bet that there will be a long term need to steer is also a bet that AI will fail.

The people footing the bills for it are paying because they believe AI will succeed, and there will be no more need for a human in the loop. The reason people are pouring trillions into these AI companies is that they expect we'll make the breakthroughs that make it possible for AI to succeed at automating everything humans are able to do today.

So, skate to where the puck is going, not where it is, and all that.

Until the AIs are running entire companies unattended, then at some point, some human has to transfer some instructions into some actionable representation than the agents can build. That human might one day be a manager rather than a developer, but that transfer of intent is always going to be the problem to solve.

And IMHO while humans are still in charge of these processes, the way that most people can best design and explain what they want is via conversation, exploration, iteration, etc. Not by a single fire-and-forget prompt!

Entire companies don't have to be fully unattended; there just has to be enough automation that they're just a small handful of humans only partly in the loop.

Basically, why doesn't the prompt "make me wealthy" work? Find and iterate on the bottlenecks.

> But surely in the real world, anyone who's got a real product to make is going to want to steer what's happening.

In my own harness I log each and every user message by hook and use the model to extract user intent by re-reading the chat log from time to time. The raw messages are very important, they contain information that can be used to refine the harness on the one hand, and to validate if the agent still follows user intent on the other. Models tend to get lost in the details and forget the big picture.

Not saying you're wrong. But right now I'm trying to craft some skill prose to instruct an agent how to optimize a certain process based on my own heuristics, and it's failing. Maybe I can let a super agent divine the right skill prose, iterating to see what works. It's worth a try.

OK, but by the time you've worked out the correct prompt to tell your meta-agent how to iterate on optimising the right prompt for your actual agent, perhaps you could have just tried a few things and got it going!

Nice try, but there's a meta-meta-agent that just tries a few things and gets it going while the other agents optimize the prompt.

Or put another way: "You can't fool me, it's turtles all the way down!"

You can't fool me - this prompt refinement turtle stack only has 3 turtles. Not much way down.

Reply on news.ycombinator.com