Is there a consensus yet on when you should fine-tune vs when you should use prompt engineering?

It seems not that hard to include in your prompt “here is some examples, please write like this and follow the style set here”. While that may make every completion request more expensive (more tokens), it also seems like fine tuning these models can also be quite expensive.

I’m curious if there are other trade offs besides cost— maybe quality achieved is better with fine tuning? Very interested to see how it all plays out. On the one hand a massive model like gpt4 can probably be prompted to match any style quite well, albeit costly, vs fine tuning a cheap model may get exactly what you want.