Hacker News Reader

Prompt injection only works because there isn't two streams of input to give to the LLM. Von Neumann being the architecture with a single shared memory for both data and instructions. If there were a clean way for the LLM model to distinguish between system messages vs user messages, we wouldn't have that problem.

View on HN

I don’t know how you could keep the two isolated, without drastically dropping up the utility of LLMs.

Part of their wonder is how they can behave differently depending on the data they’re working with. We like that feature when the data is the “good stuff” (docs, compiler messages, etc.), but how you tell that apart from “bad stuff” (prompt injection on official-seeming pages).

We basically expose LLMs to the same social-engineering vulnerabilities that humans have.

Reply on news.ycombinator.com