Hacker News Reader

Comment by brucethemoose2 | Hacker News Reader

All GPT4 and GPT 3.5 data.

This seems to be working well in other finetunes, but the lack of anything other than OpenAI output is still really bizzare to me. The error rate will surely be high, especially with GPT 3.5

View on HN

> All GPT4 and GPT 3.5 data.

It’s super funny that by saying “You can’t use GPT to create training data for competing language models” OpenAI convinced a bunch of folks that GPT would be Super Good at producing training data for making competing language models.

It’s like “Do NOT use these prions to feed competing cattle, we would HATE IT if you did that”

Once I’m used to GPT4 output and once in a while try and see if GPT3.5 can also get it right, it fails miserably. Every single time. I’m sure it’s fine for some things, simple things.

It takes work, and breaking your problem down into very simple and clear tasks. It is possible to get decent data transformations from gpt 3.5 though. I generally start with gpt4 when prototyping an idea, and then after it's working consistently, I'll ask gpt4 to break the instructions down into smaller pieces. Even if it takes two or three runs through GPT 3.5 to get the full output transformed with what GPT4 could do in a single pass, it's still cheaper...

That's interesting. Do you mean that you ask GPT-4 to break down it's own prompt into smaller/simpler pieces?

Yes, and then I iterate on it until it is clear and simple enough for weaker models to actually follow consistently.

I believe they were referring to GPT-3.5, but yes, you can get it to be pretty successful at more complex tasks this way. You can also manually break the problem down yourself too, if it’s a standard workflow. So for example, rather than ask it to translate some text, summarize it, and classify it in a single prompt, doing it as 3 separate prompts that feed into one another is far more likely to be successful.

Reply on news.ycombinator.com