I’ve tried to identify parts of the prompt in the output, hopefully recognizing someone trying to dump the prompt, but that’s pretty tough to do. I’m sure even fine tuned models are susceptible to spilling proprietary training data
- deliver/inject adversarial prompts
- remotely control LLMs
- deliver hidden multi-stage payloads
- spreading payloads/injections to other application-integrated LLMs
- manipulating data
- exfiltrating arbitrary user data with only search capabilities
- target code completion engines
- target automated systems
So, recognizing parts of the prompt or fine tuning may not be sufficient mitigations.