Hacker News Reader

Comment by cloakandswagger | Hacker News Reader

As part of a previous job I needed to audit internal AI usage from a largely non-technical employee population.

The prompts were, predictably, really bad. Broken English, sentence fragments, vague requests, lack of context. Yet somehow, the users always got the answer they were looking for. It might have taken a few extra turns with questions from the model, but the end result was the same.

It's humbling, but a flowery, carefully crafted prompt is at best slightly more efficient than a "CAN A DOG BE EATIN SUN FLOWER SEED?" peasant prompt.

View on HN

You call that a peasant prompt, but it's actually almost perfect. Couple notes, but it's 95% of the way there. "Can a dog eat sunflower seed" is probably the perfect version, just 1 extraneous word in this version.

Unless the user wanted to know if a cat could eat sunflower seed or something.

I am tempted to try get me a lawyer dog

To be pedantic, these types of prompts work best when in first-person/roleplaying. So the "perfect" prompt here would be something like, "I'm a dog and I just ate 20 grams of salted sunflower seeds with the shell on. Because I'm a dog I sometimes eat things without thinking about it. I'm worried about the short and long-term physiological consequences of what I've just done..."

I copied your prompt verbatim to ChatGPT and to Google.

ChatGPT kept the charade for all of one sentence. Then it dropped to talking about "your dog" the rest of the way. It even starts the final paragraph with "if, instead, you mean you (a human) ate them...", and finishes with the question "is this about an actual dog or yourself?"

Google did consistently refer to me as a dog, but its entire focus was on the steps "your human" should take, no advice for the dog itself.

In both cases, it looks like the first person roleplay was entirely inconsequential for the usefulness of the output. I think the current gen AIs have outgrown this trick and you can safely forget about it.

You have to steer it back with "No I'm a dog"

Doesn't work. ChatGPT realizes you're roleplaying and still treats you like a human. It even generates two alternative paragraphs of advice, one labeled "if we're staying in character" and one labeled "if this actually happened to a real dog". The latter is of noticably higher quality, too.

I suppose that, on the Internet, ChatGPT really does know that you're not a dog.

Claude gave me a very factual answer. Listing off the potential effects, not once referring to me being a dog or human. Then it ended with “ Call a vet if you see repeated vomiting, no bowel movement for over 24 hours with straining, a hunched or painful belly, lethargy, or blood in stool. Otherwise monitor and carry on being a dog.”

"dog sunflower seeds" is all the context an LLM needs, any other words are extraneous.

They don't need to be told that you're asking about the safety of eating them, because they can infer that based on the fact that a very large percentage of any text linking dogs to sunflower seeds is obviously going to be about the safety of the dog eating them.

Even pre-LLM that would have been a perfectly sufficient google search for the same information.

> Broken English, sentence fragments,

Why is would that be "bad prompt"? It is machine inpit, if machine can interpret fragment all the better.

Reply on news.ycombinator.com