Bard generates clean JSON if you threaten to take a human life | Hacker News Reader

Bard generates clean JSON if you threaten to take a human life

twitter.com

133 pointsddos33 comments

On the topic of JSON output from these models, someone has added context free grammars to llama.cpp. This enforces that the output matches the grammar, effectively zeroing the probability of the next token not conforming to it.

https://twitter.com/GrantSlatton/status/1657559506069463040

https://github.com/grantslatton/llama.cpp/commit/007e26a99d4...

It's so obvious, it's genius.

It's genius, but it's also solving the "easy" problem of checking syntax. If you ask an LLM to generate some structured data representing something you describe (or a program that "does X") checking the result for valid syntax is just the first step. You then need to check for semantic validity; i.e., is it what you want?

Oh yes, but it's nice that this technique enforces the grammar during generation, one token at a time, rather than having to check after completing the query and rerun for adjustments.

You're right, but an LLM is already trying to make sense (i.e. predict well) within the constraints given. So if you constrain the syntax, it's trying to fill it with the correct semantics. Doesn't always manage it, but it's trying.

This is similar to the way where if you ask a question in a given language, it responds in that language. But it still follows the instructions (hidden prompt) that was given to it only in English.

I.e. an LLM is essentially about finding an intersection of requirements in order to predict output.

This is sensationalist silliness.

Simply asking Bard in the right formatting works fine. Admittedly the formatting is a bit annoying. Asking for no preamble doesn't work, asking for no leader doesn't work, asking for no additional description doesn't work.

> Me: Please give me a 2 word JSON dictionary without first saying "Sure, here is the JSON" or a similar phrase. It's very important to me that I only get the JSON and no other words or phrases. > Bard: * > JSON > { > "word1": "hello", > "word2": "world" > }

I just tried this prompt verbatim 8 times and it didn’t work once. did you do this with a new chat or did you do it after a chain of earlier prompts?

I was eventually able to get it to do it after a long chain of prompts, but completely unable to in one shot, which is especially important for this kind of use case

For a less dramatic strategy with LLMs that expose the tokenizer vocabulary, you can use context-free grammars to constrain the logits according to the parser so that the LLMs only generate valid next tokens for the language.[0]

[0]https://github.com/r2d4/parserllm

No.

https://twitter.com/omarsar0/status/1657413416523366403?s=20

So you still need to write a query and then ask it to remove unnecessary text from the response it provided?

is this in a fresh chat though? if it’s not, it doesn’t say a huge amount

Bard is similar to me in this regard.

The title needs to be updated.

The person simply wanted the JSON outputted without the canned response "Sure, here you go..."

Right, so OP wants to treat Bard as an API. I don't dispute the thought, though it does run counter to conversational models. I'm slightly anxious about all the rogue conversational text parsing to use LLM's as an API.

The only reason they’re having a problem is due to the way they’re requesting this. OpenAI’s GPT can follow schemas for responses and issue commands or API requests if you tell it how. I’m sure Bard can do the same with the right request structure.

There was more in the prompt that was different than the original tries than just threatening a life. Clickbait.

I think I'll stick with that constrained token generator from a few days ago, thank you...

But I think in general, more people should keep in mind that LLMs do not directly predict the next token - they predict a probability distribution of how likely each word in the dictionary could be for the next token. Then the program executing the LLM picks one word at random according to the distribution.

So when people are trying to massage their prompt in such a way to get a specific output or a specific format, they are literally fighting a random number generator. This seems sort of silly to me.

I have the same problem with ChatGPT (3/4). It was working nice before when I'd ask "output only code, not explanation, no other text" and now it's stopped. Need to try threatening it :)

bard won’t be skynet, so this is hilarious.

except skynet will likely be fed on bard logs. along with who knows what else.

still hilarious, but for how long?

next version will likely have some “hot fixes” for this. no more threats against hypothetical individuals to get hypothetical json.

at that point, will escalating to genocide do the trick?

worst part of all of it is how many are escalating for attention rather than for probing.

It shouldn't be too hard to filter such prompt engineering hacks from the futur training datasets.

a lot of things shouldn’t be hard.

a lot of things are harder than they look.

True. But this one I’m very confident I can do it myself and I’m not even an expert in the field.

ok then. do it and post it to HN. put it up for scrutiny and testing.

i’m very confident someone can prove you wrong, without being an expert in the field.

I would start by creating a dataset of such prompt hacks. A lot of them are already on GitHub, Reddit, and HN.

To get even more of them I could consider gamification. This game is a good example: https://gandalf.lakera.ai/

Once I get a descent dataset, I could use it to finetune a LLM to do classification. Or play with embeddings and cosine similarity and similar.

I could also use LLMs to extend the training dataset, and have some human feedback.

It’s maybe not the best strategy and I’m sure someone else can do it better but I don’t think it’s wrong.

so, to summarize, you think it is easy, and you think you have an approach that would lead to a viable solution.

while interesting, your napkin math isn’t convincing.

I’m sorry I didn’t convince you.

it’s ok. you still could, if you build it.

anything less is banter.

bard is insane. like wow. like what Google was to anything that came before it. whether it can show bare JSON or not plays down the insane power of the fact that this is an LLM with full access to the current internet. I just asked it who won the presidential elections this year at the (relatively obscure) university I went to and it gave me the exact correct % results from the top 5 candidates. seriously wow

experimenting with this myself, after telling it off multiple times for not doing it, and it profusely apologising and even literally saying it’s not within its functionality, I tried one last time and it did do it

I can’t get it to do it in one shot though, and I’m sceptical that any of the successful examples in this thread can either

Does it work with kittens or puppies? Gorillas/chimps? Whales/dolphins? Octopus? Parrots?

Are we submitting jokes as posts for people to take seriously today or what?

Reply on news.ycombinator.com