Hacker News Reader

Comment by Xenoamorphous | Hacker News Reader

> We always extract json. We don’t need JSON mode

I wonder why? It seems to work pretty well for me.

> Lesson 4: GPT is really bad at producing the null hypothesis

Tell me about it! Just yesterday I was testing a prompt around text modification rules that ended with “If none of the rules apply to the text, return the original text without any changes”.

Do you know ChatGPT’s response to a text where none of the rules applied?

“The original text without any changes”. Yes, the literal string.

View on HN

You know all the stories about the capricious djinn that grants cursed wishes based on the literal wording? That's what we have. Those of us who've been prompting models in image space for years now have gotten a handle on this but for people who got in because of LLMs, it can be a bit of a surprise.

One fun anecdote, a while back I was making an image of three women drinking wine in a fancy garden for a tarot card, and at the end of the prompt I had "lush vegetation" but that was enough to tip the women from classy to red nosed frat girls, because of the double meaning of lush.

The monkey paw curls a finger.

Programming is already the capricious djinn, only it's completely upfront as to how literally it interprets your commands. The guise of AI being able to infer your actual intent, which is impossible to do accurately, even for humans, is distracting tech folks from one of the main blessings of programming: forcing people to think before they speak and hone their intention.

That's kind of adorable, in an annoying sort of way

> I wonder why? It seems to work pretty well for me.

I read this as "what we do works just fine to not need to use JSON mode". We're in the same boat at my company. Been live for a year now, no need to switch. Our prompt is effective at getting GPT-3.5 to always produce JSON.

There's nothing to switch to. You just enable it. No need to change the prompt or anything else. All it requires is that you mention "JSON" in your prompt, which you obviously already do.

I think that’s only true when using ChatGPT via the web/app, not when used via API as they likely are. Happy to be corrected however.

If you don’t know, why speculate on something that is easy to look up in documentation?

https://platform.openai.com/docs/guides/text-generation/json...

You do need to change the prompt. You need to explicitly tell it to emit JSON, and in my experience, if you want it to follow a format you need to also provide that format.

I've found that this is pretty simple to do when you have a basic schema and there's no need to define one and enable function calling.

But in one of my cases, the schema is quite complicated, and "model doesn't produce JSON" hasn't been a problem for us in production. There's no incentive for us to change what we have that's working very well.

AmeliaBedeliaGPT

If you look at any of the cake decorating fail sites, humans make that sort of mistake all the time.

Reply on news.ycombinator.com