Ask HN: How to distribute proprietary LLM prompt? | Hacker News Reader

Ask HN: How to distribute proprietary LLM prompt?

An interesting use-case for LLMs is to distribute an OSS LLM (like ollama) but specialize it with a custom, large prompt. This prompt may be proprietary. Is there any way to achieve this goal without putting the LLM behind a service API?

View on HN

You could try generating many Q&A examples from a LLM with the desired prompt, then finetuning that model using those examples, but leaving out the prompt. If the prompt is very long and contains lots of information, you may even try to use a LLM to generate the Qs.

You can confuse the LLM to reveal its system prompt even if you put it behind a service API IMHO, as long as a user has access to the chat and can freely interact with the LLM.

I've heard this is the general case, but is it proven that a prompt can always be recovered? I'm thinking particularly of cases where the LLM supports very large prompts, on the order of millions of tokens.

It would be interesting to see if there is a way to bake in the prompt with the weights which may also be a valuable performance improvement, in addition to obscuring the prompt.

Will it be a problem for you when--even with a hosted API--your proprietary prompt is leaked and other people or companies start using variations of it for their own purposes?

In theory, yes. Consider the case where the prompt includes considerable amoounts of (proprietary) source code.

Disclosure: I'm generally pessimistic about LLMs, so the subtext was: "It's best to assume it'll leak eventually, so have a plan for that."

Anywho, delaying that moment means a remote service under your own control, and some code which checks all output for prompt exfiltration attacks (which may not be plaintext, but base64 and weirder things) while also minimizing false-positives or legitimate use cases, like a code-method they can use with an implementation you want to keep secret.

Reply on news.ycombinator.com