No. Without the inference code, the best we can have are guesses on its implementation, so the benchmark figures we can get could be quite wrong. It does seem better than Llama2-70B in my tests[0], which rely on the work done by Dmytro Dzhulgakov[1] and DiscoResearch[2].

But the point of releasing on bittorrent is to see the effervescence in hobbyist research and early attempts at MoE quantization, which are already ongoing[3]. They are benefitting from the community.

[0]: https://twitter.com/espadrine/status/1733505332293288256

[1]: https://github.com/dzhulgakov/llama-mistral

[2]: https://huggingface.co/DiscoResearch/mixtral-7b-8expert

[3]: https://github.com/TimDettmers/bitsandbytes/tree/sparse_moe

Apparently you can download it via torrent[0].

https://venturebeat.com/ai/mistral-ai-bucks-release-trend-by...

Mistral doesn't release source code or data just weights. They did include some details of sliding window attention previously