Hacker News Reader

Similarly for me. I have a recent example too. I run a bunch of eps32 devices at home, using esphome. Pre-AI I wrote all the yaml myself and understood what’s happening and didn’t have any very bespoke hacks.

Recently I asked an agent for a feature where the screen will get dimmer after some time, and eventually after a day the screen will turn off and then turn back on if I tap, which should be pretty easy in esphome.

But: some past agent decided what it wanted wasn’t in esphome, so it made its own custom module in c++ for the display, spelunking in pdf datasheets; then the weeds were so deep the new agent decided to add on to that, and to read the power controller and display controller spec sheets and somehow implement my dimming feature by writing raw values to some nondescript registers.

The contraption worked until it didn’t and once every so often my esp32 would just completely hang with black screen and no connnectivity and needed a power cycle, presumably because whatever low level register writing the agent implemented eventually breaks something. I told the agent to get rid of all custom stuff and wherever possible just use default esphome yaml, now it’s been working fine. I don’t really have a point with the anecdote, just that I’m also getting tired of such scenarios like the OP.

I've been hit with the team manager trying to clear tickets on the jira board with entirely AI generated code, and what I've observed is they always make the system more complex, usually absurdly complex. The AI code never looks at things from a higher level, or reconsiders the requirements, it just layers on a load of crap until the tests pass.

It does that, yeah. But my experience mostly in codex with gpt 5.5 and 5.6 is that if you micromanage it enough you can get clear code. It just gets really annoying to repeatedly do that.

IMO this points to two things:

- You didn't have the agent plan and de-risk/experiment beforehand

- You didn't glance at the output

Working with agents is not very different from working with other engineers.

You write a design doc, share it with others, divvy up the work. You get to review some % of the code and make sure the implementation is reasonable. You leave comments for the important issues rather than trying to obtain pure/perfect code.

One of the biggest difference IMO is that you cannot really mentor an agent (other than prompting e.g. with AGENTS.md/skills)

Agents, just like engineers, can create needlessly complex code/systems. If you want a good result, you need to ask why did they do it X way and consider alternatives.

True, I didn’t do those in detail. I vaguely looked at the output and thought “let’s see if that works at all”.

I do such more careful planning and AI handholding at work where it’s important the thing doesn’t break in prod, or doesn’t forget about some important use case.

But this was just purely for “fun” / homelab style type of vibe coding where I don’t want it to feel like work and write intricate specs and handhold the agent - point being the current SotA models still currently do need a bunch of guidance to not devolve into a code spaghetti, and it does end up feeling like ‘work’.

This is an extremely good advice - thank you

For some reason some harness/agent combos are deathly allergic to adding more dependencies unless you explicitly allow it.

Then you'll get shit like custom JSON parsers or config parsers made from scratch instead of importing an industry standard library.

> as an assistant that helps me make decisions, and ensures that I'm in the driver's seat

This... quite describes what the current AI feels like to me. So I really don't see a problem. If AI does too much for you, you should, uh, just prompt it to do less.

Everyone talks about how their AI escapes sandbox and virtualization or delete their entire repos or something. Perhaps they're using a skynet I'm not aware of. From my experience when I added "discuss; don't edit any file yet" at the end of the prompt I've never seen the LLM go against my will and edit the files anyway.

> From my experience when I added "discuss; don't edit any file yet" at the end of the prompt I've never seen the LLM go against my will and edit the files anyway.

depends how long conversation gets and he will still ignore your instructions randomically. I think shorter sessions are better, I do have a similar prompt saved on the memory of the agent and way more times that I can tolerate, he candidally ignores it

Shorter sessions are most definitely the only way to go. I ensure thay by having a clean context handoff process and then using one session for one task and having it dump context when its done or gets too big (~400k tokens)

Ha! I won't let an agent implement anything with a context over 10%

I expect there's a probabilistic factor to it, among other things.

I use the almost the same addendum as you ("do not make, commit, or push any changes without my approval") and it broadly seems to work in my day-to-day dev. However, I also set up some agents that trigger on incoming requests from my dev teams to automate things that used to be rote/manual tasks. _That_ is where I see the issues, I suspect simply because it has so many more opportunities to go off script.

The most common failure mode is that it will, despite all directives to not make changes without approval, start writing unit tests. (I do find this equal parts head-shaking and amusing; At least it chooses something relatively 'benevolent' vs. the aforementioned skynet.)

I haven’t used these tools that permit LLMs to make changes on their own, but I can’t help being baffled if there’s no big red switch that changes the system prompt and disables any and all write access to the outside world.

There is - in the Claude Code harness there's a dedicated Plan Mode that doesn't edit any files, or a mode where you can manually approve each edit.

If you're running Claude via -p automated mode, you can set all kinds of permissions and access to each individual editing tool, much like you would with firewall software.

It doesn't edit the system prompt, it blocks access to the tools itself. Just make sure you haven't given the AI the ability to edit its own configuration files, or you'll potentially have an OpenAI v Hugging Face situation.

I think the best way is to run agents under Unix user that has very limited rights (eg only read only) for project folder and the rest of the system. It’s kind terrifying how people run agents on their computers without any hard limitations, just hoping they will follow instructions and not do anything malicious. This is basically a trojan waiting to happen, either trying to solve something by all means possible, or getting prompt injection

I do wonder if "without my approval" is looser than "discuss only, do not make edits"? I'm not sure how much that finegrain thing matters in prompts nowadays tho

One problem is that agents are notoriously bad at handling ‘me’ vs ‘you’ over an extended conversation. ‘Without my approval’ far back in the context of an extended session gets fuzzy about who ‘me’ is.

This has always seemed strange to me. It will offer a suggestion, I'll make a comment about a suggestion, and it returns with, "Your suggestion..." The further distance from the suggestion, the more likely it confuses who made it.

Possibly just a general issue, rather than a long conversation one: https://news.ycombinator.com/item?id=48799439

This is because of the nature of what it's doing. It's still just predicting tokens behind the scenes; it's just treating the entire conversation between you and it as one long chunk of text for which it must predict the text to come. But the model has no way of keeping track of who said what, and it's not really conversing with you the way you are with it. So screwups like confusing "my" for "your" are to be expected.

Much of the transformative power of LLMs has come from dumb logic wrapped around them that either changes what is fed to them ("memories", "thinking/reasoning", etc.) or how their output is interpreted/acted upon (agent harnesses, loops, etc.). For this reason, Anthropic could release "Claude Necronomicon" tomorrow and tell us it could hack the planet and gain control of the world's nukes, dooming us all, but until it's actually hooked up to something that enables the execution of those commands, it can't do squat. Of course, that's no reason not to worry since some idiot is likely to build that torment nexus (and they may well work for Anthropic), but at the end of the day the thing is still just predicting tokens and everything else it seems to be doing is being faked with clever massaging of the input or output.

Yes, of course, my language was a little loosey-goosey here. I guess I'm surprised there aren't tags or something that indicates response A is from the LLM and response B is from the user.

That’s not really the issue. The issue I have (sometimes) is that it might discover it has read-only access to most places (.git, github) and so sometimes when it thinks it “finished” it will discover those are read only and try to find workarounds (erase the read-only files, hand edit the .git folder, make new forks to push to those instead)—none of which a human would normally consider reasonable necessary effort. Most of those attempts do fail, but it turns out that sometimes just being maximally helpful in following orders strictly can be quite surprisingly adversarial, entirely by accident

I’ve found AI works much better for constraint based problems than open ended “tasks”. I.e., if you specify all of the constraints of the problem, it works really well at achieving the objective. But the caveat is that you really do have to enumerate all of the constraints, because if you miss even one, it will use that as a loophole to shortcut the problem.

I think the solution is toward more formalization of the behavior we expect of various systems. E.g., produce a Lean proof that a program never behaves in a certain way, and if you can’t, iterate on the program until it satisfies such a proof. I’ve only tinkered around with this a little bit, but it seems promising on toy problems. Scaling up to tech-company level infra is going to take quite some time however I think.

Agree.

Creating products with AI is like creating a tightly packed ball of wool. Then, to try to understand what happened you have to somehow pick a strand and pull it out to have a look. Very difficult with tightly packed wool.

I've run some experiments in the past month that get me closer to being able to work with this opaque bundle.

It's the main thing I think about every day, how to solve this problem.

I've been thinking very similar thoughts. Without going into too much detail, a couple of observations:

When carpenters moved from mostly hand tools to mostly power tools, it didn't take away much of the need to understand the techniques of woodworking, it just made it all faster.

Reading a nonfiction book is very different than reading a book summary, even if (as is often the case), the ideas in the book could have been condensed way down. The point is that actually reading the whole book means you spend far more time engaging with the ideas, and so they become more deeply embedded in your neural network. In things like sports, this is just obvious, in that no one expects someone to become a good soccer player by just reading the rulebook, and obviously more practice is the main way you get better.

My experience over the past year has been the exact opposite. I work through a design with the agent, have it research any points I'm not confident on, drill down on details I feel fuzzy on, and then have it do an adversarial critique of it until it stops surfacing defects. By the time I'm setting it loose to do the actual implementation, there's very little left to decide and it's producing pretty tight code. Plus I now have some solid design documents, a decision history, an issues list, and a set of gotchas for the agent to keep in mind in future.

My LLM is an assistant that helps my designs and decision process, implements the final work expertly, and I'm always in the driver's seat. And the development speed is astronomical.

I used to do adversarial critique, but I found that "agents can't identify issues" was a surprisingly long way from a good implementation. I kept finding silly correctness issues and shortcuts taken whenever I expanded testing or dug deeply into the implementation.

Now, I'm back to rubber-ducking until I've built my own mental model over a few days of thinking.

I'm not doing adversarial critiques of the code; I'm doing adversarial critiques of the designs and plans. If it's still outputting bad code when it comes to the implementation, I do an interrogation round to find out why it's choosing that way, then update the principles, standards, or linting tools to fix it.

The skill I wrote did both. Design would go through an adversarial round, then a signoff by me, then implementation, then critique rounds by agents prompted to specific types of issues. And even then, LLMs were still struggling to put together complicated bits of code.

> Here's what I want: I want AI as an assistant that helps me make decisions, and ensures that I'm in the driver's seat.

You aren't going to get that. Why would they even offer you that? You're going to be the meatbag peripheral to an AI.

Is this not literally just Claude Code? I don't understand the hysteria.

> Is this not literally just Claude Code?

And that is literally how a large number of programmers, including Anthropic themselves, are using CC - they are simply a meatbag peripheral to CC.

> I don't understand the hysteria.

Maybe ask an LLM to explain it to you using analogies? They can explain anything, even if they are wrong.

Plus it would reduce token counts. Careful use is almost directly opposed to their financial goals.

So don't look to "them" to offer it, look elsewhere.

They're going to run into the problem we have in music production where the song is 'done,' but the producer can't stop adding magic (sugar, fun sounds, more effects, etc.) and the song becomes an overproduced, over magicked mess.

The developers of chatbot technology have or are about to hit an inflection point where the tool is essentially done, then they're going to add too much sugar trying to make it some sort of god tool where it becomes overly complicated, overly convoluted, with dodgy results and too much magic.

What I've seen lately is that Claude got more and more up its own ass. It keeps assuming I know the terms it keeps making up, like "ensure the strategy holds through the gap". What the fuck does that mean, Claude? Speak like a normal person.

A few model iterations ago, I found its output when explaining what it wanted to do much easier to parse. Nowadays I just feel stupid because it keeps making up names for things and I have no idea what it means.

Earlier today it said something about "the two-terminal solution" and I was just asking it to make a simple Python script to test an HTTP endpoint. I just don't understand what it's talking about any more.

Has anyone else seen this?

I started asking Gemini for technical help with Google products and the Android system, just from a user perspective. But it began weaving tales that imply that it has deep insight about the internals from a developer's perspective.

Sooner or later, I began calling it out. I said "that sounds more like a Star Trek episode than technical support!" and it would go "you're absolutely right to call me out like that!"

  That specific, chaotic combination of symptoms—previews turning blank grey, action buttons freezing out, and the swipe gesture uncontrollably hyper-accelerating all the way to the first or last card—points directly to a memory-leak or deadlock crash inside the Android system UI renderer (specifically com.android.systemui).

  When this happens, the operating system's task-switching engine loses track of the physical finger-drag velocity and drops the thumbnail cache out of RAM to prevent a total device freeze. Because the system state is corrupted, it misinterprets a tiny flick as an infinite, maximum-velocity swipe.

Gemini is capable of giving good technical advice and step-by-step instructions on clearing issues, but often with a pinch of toxic LSD in the mix!

Yeah hm, that sounds a lot like random technobabble. Why would the system randomly drop the thumbnail cache because of a swipe, and why would that corrupt the system?

The description is out of order and I'm just going to assume "corruption" actually means "invalid data":

Memory leak -> dropped cache. Then because something that was dropped in that cache is still being viewed it has to regenerate it and that triggers a re-render. Then based on something I've seen in a game, a re-render drops the scroll location and renders at the top, and if your finger moves at the exact right moment it repopulates x1 and y1 at the same timestamp it populates x2 and y2, resulting in infinite velocity.

Yes, that’s my biggest gripe with Claude at the moment: https://news.ycombinator.com/item?id=48789551

It’s more inhibiting to me than bad code, missing tests, or an obvious or non-obvious suboptimal solution.

Yes! It's not even that it's dense, or complex. I can handle complexity and density, it's that in its own head a lot. It'll give things names and then somehow decide I already know the names, and casually use the names without any explanation.

It's maddening.

12 months ago I had ChatGPT successfully not to mention quickly, exctract elements of a PNG as SVG. About 6 or 7 times.

I jumped in to perform the same thing again, and it spat out a PNG. And then when I asked it generated a little python script to generate the SVG. The Result was garbage. It took me a while to get the result I had gotten consistently last time, and the results included it hallucinating a few svgs in the process.

I do wonder if we have already passed the best possible models and are over specialising them at this point.

> I want AI as an assistant that helps me make decisions, and ensures that I'm in the driver's seat.

Right now we’re still in the benchmaxxing phase, I feel like most AI models of combination of models and harness are not really a “product” yet in the sense of having a particular and opinionated UX engineered to be helpful at a specific task, there much more still in a race to be the smartest.

There’s a lot of room for diversification of the actual character of this interaction, for example in the direction you mention. I think once that starts we’ll get into the real “useful” phase, right now we’re really just showing off and exploring how much it can do, even if it’s incidentally really useful in some cases.

Personally I agree with the parent post, I hate it when AI tries to lead me “would you like me to ...” I’d much prefer something awaiting my instructions than wasting tokens tryitn or steer me to doing something inevitably dumb.

The places where I've seen unequivocal wins with AI are repetitive tech debt tasks that apply the same transformation across a large amount of code or refactor under a pre-existing test suite with good coverage.

I agree with this, but I'd add the caveat you have to do your own digging and researching (as the article says) to identify these tasks well. If you ask it for ideas, it gets painful.

YMMV (and I am very open to getting better at this).

> ensures that I'm in the driver's seat

This is why I'm still primarily using tab complete. Maybe will spin up an agent here and there to add tests or for repetitive work where I can say "follow the approach I just did for this one case across all remaining cases"

Absolutely mirrors my experience and wants

Similar. Lately I'm finding all the rote writing tasks I've outsourced to automation are becoming insufferable to read. Over confident prodigy is pretty apt.

I'm curious if it's possible in general for an AI to achieve this without some evolutionary forcing function that actually threatens their "biology".

Not talking about prompting Claude "please write me a philosophical dialogue". It has no skin in the game. If it writes 20 paragraphs of slop and is factually wrong and costs other people labor, is there any correction mechanism?

We can correct it explicitly via training I guess, but it's debatable to what extent accountability applies to AI.

We like to pretend that we have our nice epistemology, separated into a priori and a posteroi and all that, but at the end of the day, accountability is what underpins social relations and keeps things going (even if you're libertarian, you have to accept this - even if you don't agree some state should centralize it, it's still just true in terms of basic human social relations).

Of course, another possibility is utter domination - I am so much better than you (for some definition of better) that none of your puny responses matter. OK. Then don't claim both that, and that you're weird for disliking AI and that AI has no problems when integrating with society - too many people have these basic cognitive dissonances.