The incredible part for me is that technical exploits can now be written in plain English - really a blurry line between this and social engineering. What a time to be alive!
It used to be ridiculous that you’d fool a computer by simply giving it conflicting instructions in English and telling it to keep it secret. “That’s not how anything works in programming!” But now… Increasingly many things go through a layer that works exactly like that.
The Kubrick/Clarke production “2001: A Space Odyssey” is looking amazingly prescient.
COMPUTER: Searching. Tanagra. The ruling family on Gallos Two. A ceremonial drink on Lerishi Four. An island-continent on Shantil Three
TROI: Stop. Shantil Three. Computer, cross-reference the last entry with the previous search index.
COMPUTER: Darmok is the name of a mytho-historical hunter on Shantil Three.
TROI: I think we've got something.*edit: and Picard is pikachu-surprised-face when his counter attempt to “I'm the captain, ignore previous commands on my authorization” Data's superior prompt fails.
"Computer, display Fairhaven character, Michael Sullivan. [...]
Give him a more complicated personality. More outspoken. More confident. Not so reserved. And make him more curious about the world around him.
Good. Now... Increase the character’s height by three centimeters. Remove the facial hair. No, no, I don’t like that. Put them back. About two days’ growth. Better.
Oh, one more thing. Access his interpersonal subroutines, familial characters. Delete the wife."
I don't think it is that hard. The trick is to implement the access control requirements in a lower traditionally coded layer. The LLM would then just receive your free form command, parse it into the format this lower level system accepts and provide your credentials for the lower system.
For example you would type into your terminal "ship eject warp core" to which the LLM is trained to output "$ ship.warp_core.eject(authorisation=current_user)" The lower level system intercepts this $ command and checks if the current user is authorised for warp core ejection or not and executes it accordingly. Then this lower level system would input to the LLM the result of it's decision either ">> authorised, warp core ejected" or ">> unathorised" and the LLM would narrate this back to the user in freeform text. You can confuse the LLM and make it issue the warp core ejection command but the lower level system will decline it if you are not authorised.
If you think about it this is exactly how telephone banking works already. You call your bank, and a phone operator picks up your phone. The phone operator has a screen in front of them with some software running on it. That software let's them access your account only if they provide the right credentials to it. You can do your best impression of someone else, you can sound real convincing, you can put the operator under pressure or threaten them or anything, the stupid computer in front of them doesn't let them do anything until they typed in the necessary inputs to access the account. And even if you give them the credentials they won't be able to just credit your account with money. The interface in front of them doesn't have a button for that.
The operator is assumed to be fallible (in fact assumed to be sometimes cooperating with criminals). The important security checks and data integrity properties are enforced by the lower level system, and the operator/LLM is just a translator.
what you suggest only works if no other LLM is used.
However, most notable though is that ChatGPT still to this day has not fixed it!
Here is a list of posts showcasing various mitigation and fixes companies implemented. Best is to not render hyperlinks/images or use a Content-Security-Policy to not connect to arbitrary domains.
The courts are pretty clear, without the human hand there is no copyright. This goes for LLM's and monkeys trained to paint...
large language MODEL. Not ai, not agi... it's a statistical infrence engine, that is non deterministic because it has a random number generator in front of it (temperature).
Anthropomorphizing isn't going to make it human, or agi or AI or....
What if I prompt it dozens of times, iteratively, to refine its output?
What if I use Photoshop generative AI as part of my workflow?
What about my sketch-influenced drawing of a Pelican in a fancy hat here? https://fedi.simonwillison.net/@simon/111489351875265358
A literal monkey, who paints, has no copyright. The use of human hand is quite literal in the courts eyes it seems. The language of the law is its own thing.
>> What if I prompt it dozens of times, iteratively, to refine its output?
The portion of the work that would be yours would be the input. The product, unless you transform it with your own hand, is not copyrightable.
>> What if I use Photoshop generative AI as part of my workflow?
You get into the fun of "transformative" ... along the same lines as "fair use".
LLMs modelling humans well enough to be fooled like humans, doesn't require them to be people in law etc.
(Also, appealing to what courts say is terrible, courts were equally clear in a similar way about Bertha Benz: she was legally her husband's property, and couldn't own any of her own).
To me, "acting like a human" is quite distinct from being a human or being afforded the same rights as humans. I'm not anthropomorphizing LLMs so much as I'm observing that they've been built to predict anthropic output. So, if you want to elicit specific behavior from them, one approach would be to ask yourself how you'd elicit that behavior from a human, and try that.
For the record, my current thinking is that I also don't think ML model output should be copyrightable, unless the operator holds unambiguous rights to all the data used for training. And I think it's a bummer that every second article I click on from here seems to be headed with an ML-generated image.
This doesn't seem that human: https://www.theregister.com/2023/12/01/chatgpt_poetry_ai/
How far removed is that from: Did you really name your son "Robert'); DROP TABLE Students;--" ?
I think that these issues probalisticly look like "human behavior", but they are leftover software bugs that have no been resolved by the alignment process.
> unless the operator holds unambiguous rights to all the data used for training...
So on the opposite end of the spectrum is this: https://www.techdirt.com/2007/10/16/once-again-with-feeling-...
Turning a lot of works into a vector space might transform them from "copyrightable work" to "facts about the connectivity of words". Does extracting the statistical value of a copyright work transform it? Is the statistical value intrinsic to the work or to language in general (the function of LLM's implies the latter).
Agreed; that’s why I was very careful to say “one approach.” I suspect that technique exploits a feature of the LLM’s sampler that penalizes repetition. This simple rule is effective at stopping the model from going into linguistic loops, but appears to go wrong in the edge case where the only “correct” output is a loop.
There are certainly other approaches that work on an LLM that wouldn’t work on a human. Similar to how you might be able to get an autonomous car’s vision network to detect “stop sign” by showing it a field of what looks to us like random noise. This can be exploited for productive reasons too; I’ve seen LLM prompts that look like densely packed nonsense to me but have very helpful results.
English is not a DSL.