Hacker News Reader

Comment by tankaiji | Hacker News Reader

And then more hours cleaning it up and re-prompting.

I program as a hobby, personal projects because I can.

I recently set up a local llm to see what the fuss is about and other than the few ringer solutions my experience is as you described. 2min promping, 5min waiting, 3hrs debugging or just doing it myself.

I am very likely doing it wrong, and it does speed up some aspects, but I wouldn't say I trust llm code any more than my own. Until it runs and throws an error, the llm is 100% confident that it has written perfect code.

Local llms aren’t super exciting unfortunately.

Neither are proprietary blackbox systems unfortunately.

I don't think setting up a local LLM is a reasonable way to get a good idea of how enterprises are using this stuff.

Try using Fable and report back. Local LLM is to Fable as Little Tike car is to a Porsche.

A normal agentic loop will have the agent using a type system and basic tests to do some basic validation of changes. A good agentic loop would give the agent a very easy way to verify if it’s on the right track. I think agents are better than many humans at writing error free code (runtime errors, not bugs. The code could still be buggy or incorrect.)

Fundamentally though, an agent cannot produce great code, because great code requires intent, which is the opposite of the statistical mean.

You will get a solution that works with a proper workflow, but you won't get one that scales or would be truly maintainable. Which is also what you get with random midwit drive-by contributors, but faster. I'll give it that.

If you built a task management system, you'd have very different code bases depending on whether it's for internal use at a mid-size development org or as a SaaS.

So I wonder whether, in your experience, the results you've seen, could have improved by providing sufficient context? - or what context was given.

I.e. if you have the agent that same context, as one of your colleagues would have/require to solve a problem.

"You're holding it wrong"

You've missed my point. I didn't dismiss agents. I did dismiss the industry.

I don't need to add more context to a statement that operates on a layer above where context injection would influence it. It is a conceptual impossibility. Not a technical roadblock.

> because great code requires intent

If can put properly engineered intent in the prompt that is verifiable, it works wonders. Anything that can defaults to the llm doing its way, you're right, it just can't converge to good, not with proper constraints.

I'm not sure I agree that LLMs solve for the statistical mean

But more importantly: isn't what you call "intent" just a series of optimization goals? You want your code to satisfy the constraint of being correct™, while also maximizing various other goals like being maintainable, easy to understand, having few lines of code, as little tight coupling as possible, etc. Goals that often conflict, but when given two implementations you could likely tell which hits the better tradeoff (in your engineering experience)

Those are all things that theoretically - with a tight enough specification and enough compute - a constraint solver could solve. No human intent necessary.

The issue is more that we can't fully specify all those side goals, and even if we could the LLM would struggle following them. A classic paperclip maximizer problem (where nobody told the paperclip maximizer to keep the planet inhabitable and all the other side conditions we implicitly assume)

> I'm not sure I agree that LLMs solve for the statistical mean

The good thing is that you don't have to agree with that, as the fundamental technical reality does it for you. LLMs do work like that. They are just statistics and probabilities.

Doesn't the universal approximation theorem apply to typical LLM architectures? LLMs solve for the shape of the input data, as well as they can approximate it in the constraints of their architecture, the optimizer and the amount of compute spent in the training process

And where those constraints prevent us from matching the shape there is no guarantee that we match anything like the mean. Though maybe we can agree that that's where the optimizer would tend to steer towards when it can't do any better

This is a weird framing, but the user is the "intent". If you can produce "great code" by yourself (whatever that is), then you can get your agent to produce it for you. If you are talking about vibe-coding then yeah, it is going to be trash

Once agents run autonomously for longer periods preventing vision drift becomes as important as correctness.

Epiq solves this with an architecture that supports workflow auditing, allowing you to time-travel state in a filtered view to reconstruct what happened, when, by who, and where intent started drifting, while also allowing you to correlate the evolution of the board with the corresponding commits.

https://ljtn.github.io/epiq

That has been my experience as well. Having the agent have full access to what it is building feels like a cheat code. A recent example: Godot (the game engine) has a feature where GPUParticles can spawn another particle system when they collide with something. Today (as in, download the engine right now and see it for yourself) this is broken, particles spawn beneath the collision box and the faster the particle is the worse it is.

So I asked Claude to first figure out a way to instrument GPU particle code so it could itself check his solutions. After that I left the agent running for 3 hours and it came back with a solution (continuous collision detection) and a bug fix (particles get their velocity applied an extra frame after colliding). I'm sure someone would find something to complain about the code (which is why I haven't upstreamed it) but it looks visually perfect and I had plan to fork the engine for my project anyway

Models that run on (average) consumer hardware are not even close to comparable to models like Fable or Sol. Its like comparing an ant to the largest dinosaur.

You don’t even have to go that far, recent Opus models are quite impressive.

Felt that. I already gave DeepSeek the HTML and it still said the UI was good to go, then told me it has no vision.

Reply on news.ycombinator.com