Hacker News Reader

Comment by ren_engineer | Hacker News Reader

I think it's worth noting the response explicitly said the paid API model hasn't been changed, so ChatGPT could have been changed through many different ways outside the core model

View on HN

This should always be the first explanation when you're speculating about something involving ChatGPT or Bing Sydney, if you aren't directly using the API and comparing temp=0 responses to identical canned prompts. There are lots of smaller models & filters and prompts involved, all of which are much easier to update than the behemoth underneath, and which do get updated frequently without notice. People jump the gun constantly on assuming it's the biggest model which changed.

What is your hypothesis for the layout/diagram of models feeding the gpt-4 output for chatgpt/gpt-4 vs api/gpt-4?

Components which are likely there and which could be affecting quality: the invisible-to-the-user 'system prompt' (probably with few-shot examples), retrieval from history (not necessarily exclusively your own, as a way to augment few-shot), cascaded models with a very cheap 'turbo' model to try to answer first, possibly regular finetunes of the main chat model (note that OP only specifies the API model hasn't changed, but the live chat models seem to change frequently), and a post-generation filtering model trying to reject offensive outputs.

We know MS Bing Sydney is doing at least 3 of those (prompt, cascade with Megatron, and finetuned rejection classifier for post-output filtering) on top of its GPT-4-finetune, so it's not a stretch to figure that OA is doing similar things.

Which one? The response wasn't clear on this - they could've meant gpt-4, or gpt-4-0314, or both. gpt-4-0314 is the "pinned" model that is not supposed to change, so I can believe it didn't change. gpt-4 is supposed to receive updates, so... are they saying it didn't?

Note that API users using Bring Your Own Key tools/chat frontends probably default to gpt-4, and not the pinned gpt-4-0314.

Reply on news.ycombinator.com