Hacker News Reader

Comment by LegionMammal978 | Hacker News Reader

> Does the model capable of tool use feed the uninspected user input to a sandboxed model, then treat its output as an opaque string?

That was one of my early thoughts for "How could LLM tools ever be made trustworthy for arbitrary data?" The LLM would just come up with a chain of tools to use (so you can inspect what it's doing), and another mechanism would be responsible for actually applying them to the input to yield the output.

Of course, most people really want the LLM to inspect the input data to figure out what to do with it, which opens up the possibility for malicious inputs. Having a second LLM instance solely coming up with the strategy could help, but only as far as the human user bothers to check for malicious programs.

View on HN

In your chain of tools are any of the tools themselves LLMs? Because that's the same problem except now you need to hijack the "parent" LLM to forward some malicious instructions down.

And even if not, as long as there's any _execution_ or _write_ happening, the input could still modify the chain of tools being used. So you'd need _heavy_ restrictions on what the chains can actually do. How that intersects with operations LLMs are supposed to streamline, I don't know, my gut feeling is not very deeply.

Well, in the one-LLM case, the input would have no effect on the chain: you'd presumably describe the input format to the LLM, maybe with a few hand-picked example lines, and it would come up with a chain that should be untainted. In the two-LLM case, the chain generated by the ephemeral LLM would have to be considered tainted until proven otherwise. Your "LLM-in-the-loop" case would just be invariably asking for trouble.

Of course, the generated chain being buggy and vulnerable would also be an issue, since it would be less likely to be built with a posture of heavy validation. And in any case, the average user would rather just run on vibes rather than taking all these paranoid precautions. Then again, what do I know, maybe free-wheeling agents really will be everything they're hyped up to be in spite of the problems.

Maybe I don't understand your idea.

I thought it was the LLM deciding what chain of tools to apply for each input. I don't see great accuracy/usefulness for a one time chain of tool generation via LLM that would somehow generalize to multiple inputs without the LLM part of that loop in the future.

not sure how much we can apply this here, but how about specific LLM judges that look for manipulation of I/O?

Same problem with humans and homoiconic code such as human language

Reply on news.ycombinator.com