Ideally, something that’s lightweight (like cheaper than fine-tuning) and also harder to manipulate using regular text prompts?
But, I suspect, if the model is able to handle language at all, you'll always be able to get a representation of the prompt out in a text form -- even if that's a projection that collapses a lot of dimensions of the tensor and so loses fidelity.
If this answer doesn't make sense, lmk.
> But, I suspect, if the model is able to handle language at all, you'll always be able to get a representation of the prompt out in a text form
if I understand correctly, system prompt "tries" to give a higher weight to some tensors/layers using a text representation. (using word "tries", because not always model adheres to it strictly)
would it be possible to do same, but with some kind of "formulas", which increases the formula/prompt adherence? (if you can share keywords for me to search and read relevant papers, that would be also a great help)
Seems like it could be possible to lobotomize the ability to express certain things without destroying other value (like a human split brain). Of course possible doesn’t mean tractable.