- it is NOT backed directly or owned by any VC funded company
- it is 100% OSS driven by the community (Apache 2 license)
- it’s currently the top OSS chat model that can be used commercially on the chatbot arena score board
- IMO it is undertrained, so expanding the training data alone will make it much better (however for the sake of this paper, we wanted to focus on architecture not training data, so we compared similarly trained models)
And yes we do have multiple experiments and plans to make it better. It’s a list, and we will not know which is final until we try. Individual members can go to great lengths on what they are working on
For better or worse, being truly OSS means our initiatives are more disorganized then a centrally planned org
To be fair, that filters the majority of models in the scoreboard.
Where we have more OSS models to choose from without weird rule lawyering gotchas. Or needing to be from a research institute / a license to download the weights
- integrating this with AI platform X/Y/Z
- setting up evals
- improving the code quality
- making a how to guide (it’s stuck on my todo list)
- helping with dataset
- doing silly experiments on how the architecture work (and if the changes give good result)
- etc etc
One of the community goals is to make this a model for EVERYONE on earth that means we need quality dataset for all the non English languages
So even on that level there are things to do
( find something that interest you on the community )