Hacker News Reader

I have a question For people in the field!

Would GPT only with pretraining and no fine tuning be able to behave better when prompt is "let’s think step by step"?

Or this prompt only worked because a fine tuning dataset containing many "let’s think step by step" prompt was used?

Working with pure pre-trained models is quite hard, and takes some practice. The key part is that "let's think step by step" is a technique that is used by humans also, and therefore (I think) would be somewhat represented in the pre-training corpus. It would be somewhat harder to "activate" this mode of thinking than a "let's think step by step" would in a fine-tuned model, but it would be possible with some elbow grease.

Reply on news.ycombinator.com