> For translation, you are better off starting from a model specifically trained for that purpose
Yes, but wasn't the whole point of the recent LLM research to show that you didn't need to fine-tine for a specific task?
Yes, but wasn't the whole point of the recent LLM research to show that you didn't need to fine-tine for a specific task?
Remember that GPT3 is 175 billion parameters so many times bigger than both the above models (and gpt4 is rumoured to be bigger still), which also allows it to be more generalisable.
If GPT3 was trained at 7 billion parameters it might also lose it's language translation capabilities.