Hacker News Reader

Comment by c7DJTLrn | Hacker News Reader

Getting bored of seeing posts every single day about prompt injection. I got it the first in the first thousand tweets and blog posts, it's insecure. It doesn't take a genius to understand how it works. Nobody ever claimed it was safe to pass arbitrary data in prompts. If you're hooking up a browser plugin to ChatGPT, you are frankly living under a rock or a moron.

Can we actually discuss solutions now? And good solutions, I don't mean using an LLM to check prompts before passing it into the same LLM.

View on HN

As the person responsible for the majority of the posts about prompt injection that have showed up on HN in the past few weeks, I could not be more desperate to discuss good solutions.

So far I haven't seen any.

I don't think there's anything that we can do. In my view, we need OpenAI to build models capable of working with separated instructions and data. I would be interested in seeing some papers that show how this could be done.

The whole benefit of ChatGPT is that instructions and data are mingled in a natural way. Users generally do not want to prompt ChatGPT with:

  "Please summarize this for me. Then create an email from the template below with the summary included.
  
  <start_article_to_summarize>
  
  ...
  
  </start_article_to_summarize>
  
  <start_email_template>
  
  Hi Bob,
  
  I read the article you pointed me to. It is an interesting view, but it falls short at ... . Here is what the authors propose.
  
  <insert_summary_here />
  
  Thanks,
  
  Alice
  
  </end_email_template>"

Plus ChatGPT cannot really trust that the separators are added appropriately, so injection of some sort is still possible.

Does it need to be either/or? We can have models for chat and models for APIs.

I would say: run with it. Assume the model's output is always untrusted. Assume that no rules can be applied to the model itself, only to the container it runs in. What kind of sandbox does the model need to be placed in given those assumptions?

Simple rules can be followed like: never take a potentially harmful action based on the model's output unless the user has had the ability to review the action, clear the context and apply whitelist validation to the action's parameters before using the model itself to ask the user for confirmation, never put data in the context that you don't want the user to find out, etc...

I think it's mostly a matter of UI design, not of prompt engineering.

> It doesn't take a genius to understand how it works.

Unfortunately, the bar is substantially lower than you think: https://www.rollingstone.com/culture/culture-features/texas-...

There are no solutions

>I don't mean using an LLM to check prompts before passing it into the same LLM.

I mean this works for GPT-4 lol. It's just twice as expensive

Reply on news.ycombinator.com