Hacker News Reader

It’s legit and has been honed and refined for the past few years by the same Stanford PhD students (now graduated and working at CMU & Princeton).

The latest iteration has promising results for language but no one has yet trained and released a big (7B+) model to see how it scales.

The peer reviews on Mamba's ICLR2024 are pretty interesting, especially with how split the reviewers are: https://openreview.net/forum?id=AL1fq05o7H

Kind of weird we haven’t seen Mistral 7B Mamba. Until that’s out it’s untested vaporware.

Do you mean include it with the Mixtral "Mixture of Experts" model? I'm not sure Mistral Mamba makes sense, since it's a completely different architecture.

This isn’t a fine tune. It’s not even a transformer like Qwen, Llama, GPT, Gemini, or Mistral. It’s another approach to language models based on RNNs

Doesn't the point remain though -- when/where do we demo this?

There is Mamba 7B pre trained model on hugging face/ LM studio right now.