OpenHermesPreferences: Dataset of ~1M AI preferences from teknium/OpenHermes-2.5

The dataset can be used for training preference models or aligning language models through techniques like Direct Preference Optimization (DPO).