Hacker News Reader

Comment by williamcotton | Hacker News Reader

Whenever I’m including context from a remote query I’ve done so in the context of another completion request that is executed and parsed outside of the scope of the primary prompt completion. All that this attack vector would accomplish with such an approach is either mangled or incorrect data. I’m also not feeding back the history of prompts and completions, aka, it’s not a chatbot.

View on HN

Pretty sure we address this issue in the paper/repository? Some of our demos rely on letting the LLM copy the injection into the final response, getting around the issue of things in subprompts not being visible later on, depending on the chain-of-thought method used. I'm not sure if that is what you mean. There are ways of utilizing these models in a safe way; we're just saying connecting them to anything at all can be easily unsafe. If you are not affected, almost all proposed use-cases for LLMs are, as they rely on integration and context to provide the utility they promise.

It’s more like this: subprompts don’t ever inject the full context from a remote query back into the primary prompt. The completions of subprompts are (via few-shot or a fine-tuned model) structured, eg, JSON, which is then parsed. The main prompt is orchestrating the subprompts and never needs to even process the results if there’s a Python or JS interpreter involved.

Here’s the kind of approach I’ve been using:

https://github.com/williamcotton/empirical-philosophy/blob/m...

The initial call to the LLM will return a completion that includes JavaScript. There is no third-party data at this point. The JavaScript includes further calls to the LLM that returns JSON, but at this point no further calls are made to the LLM. This means that responses from remote queries are never sent to an LLM. The text presented to the user could be some instructions to talk like a pirate but all the user suffers from is a surprisingly incorrect result.

Even with LangChain the issue is the chatbot UX. LangChain can also be used in ways that make it not vulnerable to this problem.

Orthogonally, I don’t think that chatbots are a very good UX in general and that there are much better ways to interact with an LLM. If anything your work should accelerate this process!

Isn't this similar to your idea? https://github.com/openai/openai-python/commit/75c90a71e88e4...

Sounds interesting, I'll be sure to have a look!

Reply on news.ycombinator.com