Hacker News Reader

Comment by noir_lord | Hacker News Reader

Do we add a third one to check the second one which is checking the first?

Asking slightly tongue in cheek but at what point does this stop making sense if we can't trust the output, the people creating the models are already getting surprised in bad ways (if we take their words at face value) with how the models are behaving already etc.

We have the folks over here saying "AI is amazing" and the other other folks over there saying "AI is terrible".

I've largely sat it out so far and I listen to both camps (and people in the middle as well) and I keep half an eye on what they are up to (including periodically evaluating them) but my overarching impression is still "Why would we trust this when it hasn't shown it's trustworthy?"

View on HN

YMMV, I’m pretty AI-pilled in the sense that I think AI is one of the most pivotal things humanity will ever invent, and it’s going to radically change our civilization over the next few decades (not necessarily for the better!)

But I wouldn’t say I “trust” these agents. The degree to which I double check their work depends heavily on the consequences if it gets something wrong. Not too dissimilar from another human dev in that sense.

So for the SaaS that supports my family, there are some things I have it build where I glance at the PR for a minute or two, but if it broke something on this admin page that only I see, there’s no real downside and I’ll find out pretty quickly next time I use it. And it’s fine 95% of the time, so it doesn’t feel like the best use of my time to double-check it carefully.

But for some of the complex internal flows where a bug could be both catastrophic and difficult to even discover for awhile, I still check it very carefully.

For a little one-off vibe coded demo thing like OP shared, I wouldn’t look at the code at all, I’d just have another agent check it and fix anything it finds. Very low stakes.

Humans are also not trustworthy. So we build systems to capture their nondeterministic output and solve the issues that emerge.

yeah, isn’t that the main focus of discipline of software engineering?

We also have sanctions and incentives to induce specific behaviors, but they don't apply to agents. We can put a muzzle on Guile 3.8 but we can't turn it into Genuine 3..8.

Adding another agent to check the first one feels like putting a band-aid on a band-aid. If there is an issue with the third one, we adding a fourth one as well

Hey you just described my dev team!

I mean if each agent reduces probability of error by 90% then after 9 agents you would have “nine nines” of reliability.

Obviously maybe it’s not composable like that exactly in real world but that’s the intent of agents checking agents

It's turtles all the way down my friend.

I mean sure, you can add a third, and a fourth and a fifth one if ur ok with the added cost, latency and it actually helps. Redundancy is a core concept in software and CS and at the heart of making many systems, complex or otherwise, reliable.

It's agents all the way down

Reply on news.ycombinator.com