Hacker News Reader

Comment by quickthrower2 | Hacker News Reader

When is fine tuning worth it, rather than just prompt engineering?

I think these are two very separate concepts.

What we are mostly seeing when it comes to fine-tuning is making a model promptable. Models like LLaMA or the original GPT3 weren't promptable. They were fine-tuned with demonstration data that looks like a prompt input, prompt output.

See below: { "instruction": "What would be the output of the following JavaScript snippet?", "input": "let area = 6 * 5;\nlet radius = area / 3.14;", "output": "The output of the JavaScript snippet is the radius, which is 1.91." }, [1]

Prompt engineering is really just carefully designing what inputs and outputs on a prompt-ready model work best.

I highly recommend skimming this RLHF article and looking for the parts where it talks about demonstration data [2]

1: https://github.com/sahil280114/codealpaca/blob/master/data/c...

2: https://huyenchip.com/2023/05/02/rlhf.html

Thanks for link 2 - it is worth a proper read! Read half of it already and it is very interesting and useful for understanding this.

Cheers!

Prompt engineering and fine tuning are in many cases alternative ways to achieve the same goal. You claim that the "original GPT3" wasn't promptable. I'm unsure which version you refer to, but I'm guessing you refer to text-davinci-003 and it was definitely promptable. For one app I used prompt engineering to make it behave like a spirit talking through a ouija board. For another, I used prompt engineering to make it act like a dystopian search engine from the future. So, yeah, it's promptable.

From what I've seen, it's when embeddings get too large for the token limit or the embeddings drive the cost up too much because you're always operating near the max token limit. In those cases, it may be worth the up front training cost and slightly higher per-token cost to dramatically reduce the amount of tokens in the average request. If you're building a higher throughput solution, the difference in cost can be quite large.

When you're starting to run into context limits.

It's worth it whenever you have a reasonable amount of training data. You can get substantial quality improvements automatically. Unless you're doing some kind of prompt-optimization, prompt-tuning is a lot of random guessing and trial-and-error. It's also most necessary when you have a smaller base model, as opposed to one of the big ones.

If you want to teach it eg. all of the text in your private training manuals and internal documentation, which wouldn't fit in the input token size.

Reply on news.ycombinator.com