Open-Llama: A Open Source Project for Training Language Models

Exciting news! Check out this model trained using the Open-Llama project at http://home.ustc.edu.cn/~sl9292 . This model is trained primarily on English and Chinese, but also has capabilities in other languages like Japanese and Korean.

Now, let's dive into Open-Llama. It's a truly open-source project for pre-training and instruct-tuning AI models. One of the key features of this project is its support for a wide range of model sizes, from 7B to 65B parameters.

What sets Open-Llama apart is the incorporation of performance acceleration via xformers from Llama, enabling 95% of the original Llama speed on the 65B models. In fact, for the 7B models, Open-Llama's performance surpasses the original Llama.

By providing full access to the codebase, we believe that Open-Llama will contribute greatly to the advancement of open-source AI technologies. We invite developers and researchers to join us on this exciting journey!