Hacker News Reader

The vuln is that a user can be tricked into exfiltrating data without it being obvious.

But what "data"? LLMs don't know anything except whatever they were trained on, right?

The article describes how the content of a document (which in theory should only be sent to OpenAI) can be exfiltrated to an attacker via URL parameters.

The human can "audit" the entire text he's copy-pasting and still miss it.

And it provides a way to exfil without it being visible.

Lots of LLM applications involve using an LLM to process external data, which makes it part of the prompt. Intuition driven by systems where instruction/code are strictly distinct from data input for processing may be failing you here.