Hacker News Reader

I agree. It's not the _method_ of the output that matters as much as what kind of operations the LLM has write/execute permissions over. Fundamentally the main issue in the exploit above is the LLM trying to inline MD images. If it didn't have the capability to do anything other than produce text in the client window for the user to do with as they please, it would be fine. Of course that isn't a very useful application of AI as an "Agent".

View on HN

> If it didn't have the capability to do anything other than produce text in the client window for the user to do with as they please, it would be fine. Of course that isn't a very useful application of AI as an "Agent".

That's a good attitude to have when implementing an "agent:" give your LLM the capabilities you would give the person or thing prompting it. If it's a toy you're using on your local system, go nuts -- you probably won't get it to "rm -rf /" by accident. If it's exposed to the internet, assume that a sociopathic teenager with too much free time can do everything you let your agent do.

(Also, "produce text in the client window" could be a denial of service attack.)

Reply on news.ycombinator.com