Hacker News Reader

Another dumb question: is it possible to "fine-tune" this model with a "corpus" of specialized data/jargon in another language ? I tried to do this with OpenAI API in order to improve the response on a specialized field (pediatric oncology) in another language (Italian) but the results were mixed (not to say disappointing).

View on HN

This is a common misunderstanding; fine-tuning a model does not mean teaching the model new information. Fine-tuning is used to adapt the model to perform better in a specific task or domain using the information it already has, in a specialized way (like a chatbot for question/answer interactions). Training a LLM on new data is extremely expensive so it's not possible.

Ok then. So how to teach the LLM new information/jargon ? I repeat: I tried to do this with GPT (fine-tuning and embeddings) and I got no improvements on jargon and re-training is out of reach, of course.

I imagine the path to doing this, assuming the model already has good Italian, is to train it on all educational materials that go into getting a degree in pediatric oncology in Italian. This would help the model to actually understand the jargon, by tying it together with its baseline knowledge of Italian.

Reply on news.ycombinator.com