Pearl-7B-slerp, an xtraordinary 7B model for maths

Pearl-7B-slerp is a merge of the following models using LazyMergekit:

mlabonne/OmniBeagle-7B WizardLM/WizardMath-7B-V1.1

Spherical Linear Interpolation (SLERP) serves as a technique for seamlessly interpolating between two vectors while maintaining a constant rate of change and upholding the geometric properties of the spherical space in which these vectors exist.

Opting for SLERP over traditional linear interpolation is motivated by various considerations. Linear interpolation in high-dimensional spaces may result in a reduction in the magnitude of the interpolated vector, diminishing the scale of weights. Additionally, in many cases, the alteration in the weights' direction conveys more meaningful information, such as feature learning and representation, compared to the magnitude of change.

The implementation of SLERP involves the following steps:

- Normalize the input vectors to unit length, ensuring they signify directions rather than magnitudes. - Calculate the angle between these vectors using their dot product. - If the vectors are nearly collinear, the method defaults to linear interpolation for efficiency. Otherwise, SLERP calculates scale factors based on the interpolation factor t (where t=0 corresponds to 100% of the first vector, and t=1 corresponds to 100% of the second vector) and the angle between the vectors. - Utilize these computed factors to weigh the original vectors, and then sum them to derive the interpolated vector.

Sounds cool, do you have any examples? Why is it xtraordinary? What's the prompt format?