Hacker News Reader

Comment by t-writescode | Hacker News Reader

> Ditto for stuff that's simple. A JSON endpoint that runs a SQL query and returns some JSON? If it works and a glance at the tests looks OK then I trust my agents wrote it properly.

That is *exactly* the sort of area I *wouldn’t* blindly trust AI, there’s a huge security boundary there. What if the AI is doing string concatenation with user-provided data???

View on HN

Once you've seen the AI not make mistakes like that a few dozen times you start to trust it not to mess that up in the future.

Is this really a rational strategy for something whose nature is to be right most of the time and then spectacularly wrong a much lesser amount of the time?

Yes, because the mistakes in code are easy to spot. I wouldn't use this to write me an unreviewed legal brief.

But you said that in this case you don't review the code because you trust the AI having seen it write that kind of code before.

you literally just said you don't review this code!

I agree with a commenter above/below (depending where this comment lands). For some time models won't do this. And any review from review agents would caught this. For most of the AI programming there needs to be a more stricter (automated) review process now. Most SAST tools would caught this type of security issue.

We're pretty far past this if you're using anything close to the sota models.

But you could be defensive with a security checklist in agents.md and have adversarial review, if you wanted.

Reply on news.ycombinator.com