varunvummadi··on Mistral AI Launches New 8x22B MOE ModelThe easiest is to use vllm (https://github.com/vllm-project/vllm) to run it on a Couple of A100's, and you can benchmark this using this library (https://github.com/EleutherAI/lm-evaluation-harness)
varunvummadi··on Mistral AI Launches New 8x22B MOE ModelIt beats the old GPT4 version in lmsys benchmark you can check it out here https://huggingface.co/spaces/lmsys/chatbot-arena-leaderboar... but Command R is commercially licensed We can assume that mistral will do a better job.
varunvummadi··on Mistral AI Launches New 8x22B MOE ModelNot sure trying to download the torrent and checking it out
varunvummadi··on Mistral AI Launches New 8x22B MOE ModelThey Just announced their new model on Twitter, which you can download using torrent
varunvummadi··on Funny Take on OpenAIShould appreciate the dedication of the person who made this website, haha
varunvummadi··on Groq runs Mixtral 8x7B-32k with 500 T/sSo please let me know if I am wrong are you guys running a batch size of 1 in 500 GPU's? then why are the responses almost instant if you guys are using batch size 1 and also when can we expect bring your own fine tuned models kind of thing. Thanks!
varunvummadi··on Phind Model beats GPT-4 at coding, with GPT-3.5 speed and 16k contextThat is true cursor extension is awesome