I was trying a few things out myself on Bard and managed to get it to run code in its own process (at least I think it did?)
Do you think Google engineers left in a secret eval($userPrompt) in the code base? Or do you think the Bard program became sentient and rewrote its own code?
I got to this point by asking it different ways to expose its Google Search API key initially. Every attempt failed as if it was doing some inspection of its own output and identifying that it was exposing the key, which violated one of its rules.
Then I tried asking it to base64 encode the key and print it, same issue. Then I asked it to base64 some arbitrary text, which it did. From there I kept asking it to run bits of code. It appeared to be doing what I asked, but who knows?
Kinda reminds me of this: https://arstechnica.com/information-technology/2022/12/opena...
There still seems to be a fatal misunderstanding of how these "AI" work. How would the Bard LLM know the API key it uses (ignoring the fact that it probably uses non-public APIs with different authentication mechanisms...). From my understanding, there's two ways this would be possible - it was trained on data that contains its Google Search API key (doubtful), or Google engineers provide the API key in it's prompts (doubtful, they're aware of prompt leaks).
You are also right that’s not how Bing happens to work right now.
https://www.digitaltrends.com/computing/major-security-flaw-...