Tune PaLM 2 with your own RLHF training data

Each product fine-tunes their own weights on top of the underlying model, but it seems like that fine-tuning is more about style than substance.

For example, I listened to a 90-minute interview about the MedPaLM 2 product. It achieved an accuracy of 85% on the MedQA dataset (median score for human medical students is 67.4%) and 72.3% on the MedMCQA dataset (median score for human medical students is 58.4%).

But it didn't get its medical knowledge and reasoning capabilities from the fine-tuning, since the underlying PaLM 2 model already had that.

The fine-tuning was focused on interaction style, more professional and objective, avoiding informal language or expressing personal opinions.

MedPaLM 2 was also fine-tuned to answer questions in a comprehensive, deliberate and informative way. For example, if a doctor asks MedPaLM 2 about the causes of a medical condition, MedPaLM 2 provides a list of possible causes, as well as information about the symptoms, diagnosis, treatment alternatives and side effects. The fine-tuning also helped it to express its degree of certainty or uncertainty.

The underlying PaLM 2 already had the knowledge and reasoning capability to perform well on standardized medical tests, and the fine-tuning helped it to express it more effectively.