French AI startup Mistral secures €2B valuation | Hacker News Reader

French AI startup Mistral secures €2B valuation

ft.com

107 pointsadmp76 comments

Can someone explain to me what they're trying to do now?

They went from we're going to create a European competitor to OpenAI to we have a 7bn parameter model?

From their pitch deck (https://sifted.eu/articles/pitch-deck-mistral)

"By creating mistral.ai, we intend to train state-of-the-art models with counter-positions to closed-model current offerings. Our vision is to become a leading actor in the field, while developing a very valuable business around integrating these models in the European industry and beyond."

Have you used any of the Mistral 7b models? They're really good, I think the 7b model just shows what they're capable of. They also dropped an 8x7b mixture of experts model yesterday, that should be very interesting to try.

I have, through Ollama, and it would say their truthfulness is not great. You almost have to treat them with kid gloves and prompt them a certain way to get reasonable results (they're harder to steer than larger models). There's a reddit thread on this where people share tricks on how to handle 7B models.

https://www.reddit.com/r/LocalLLaMA/comments/18e929k/prompt_...

My feeling is 7B models should only be used for domains where reasoning and factual correctness are not needed, like summarization or generating creative variations of ideas. Code generation is ok too if one is willing to babysit it.

But otherwise, the answers can be misleadingly bad. Even reasoning fine-tuned models like Orca2 7B do very badly on simple math questions, despite Chain of Thought prompting.

They just released a much larger model as a torrent. Probably the first open source mixture-of-experts model. Waiting for them to drop the code and explain it.

Which one? Everything on their website is still about the 7B model. 7B is the only model available on their huggingface page too: https://huggingface.co/mistralai

Edit: I'm guessing this is the model referenced: "Mistral "Mixtral" 8x7B 32k model [magnet]" - https://news.ycombinator.com/item?id=38570537

That's it. The community is working on getting it implemented in the popular inference engines. It came out less then 48 hours ago so give it a few days and we should be able to run it and test it at various quantizations depending on how much compute power you have.

There’s an initial version of it already up and running:

https://huggingface.co/DiscoResearch/mixtral-7b-8expert

I am doubtful about these efforts because people are just guessing about how it is supposed to work since Mistral didn't release any code or information yet.

Yes, there are probably some implementation pieces missing. But Disco Research also punches way above their weight on open source LLM stuff, so I would gamble it’s not that far off. They also just released a chat fine-tune of the model just moments ago:

https://huggingface.co/DiscoResearch/DiscoLM-mixtral-8x7b-v2

People had Mixtral 8x7B running on Twitter within 3 hours of the torrent being posted.

It's already been independently benchmarked and finetunes are cooking.

> Can someone explain to me what they're trying to do now?

> "By creating mistral.ai, we intend to train state-of-the-art models with counter-positions to closed-model current offerings. Our vision is to become a leading actor in the field, while developing a very valuable business around integrating these models in the European industry and beyond."

Is this just a rhetorical question from your side? Because the last sentence gives you the answer. They do open models + focusing on integrating those open models into European industry. Seems pretty easy to grasp.

Seems similar to some Linux distros business model, give it away for free but the service is paid.

If you believe a few assumptions then foundation model companies like Mistral look very valuable even if they open their models: 1) few companies are be able to produce top performing foundation models at any given time, 2) this will continue to be true for a long time due to the required expertise, infrastructure, and cost, 3) this somehow translates into a capability commercial customers will need and pay for at scale.

That third step is a bit of a leap. I tend to think that as the open knowledge about how to train models improves the training "recipes" needed to get close to state of the art will get easier to replicate, not harder, with "good enough models" requiring less data and less hardware.

Maybe they could earn money by fine tuning and ensuring it works perfectly for commercial customers.

The open model is the base. But if you want to use it as a customer service bot, you need it to be safe and stop hallucinating.

https://archive.ph/2023.12.08-165411/https://www.ft.com/cont...

This is like dot-com bubble all over again...."we have this fancy and powerful new technology, you must invest in us." Good luck competing with Microsoft, Facebook and Google, because they are super-focused now on LLMs and generative AI and they are not going to stop.

Mistral's models outperform all of theirs at much smaller sizes and can be self-hosted.

Right now, but how long does it take for OpenAI, Facebook and Google's Bard/Gemini to catch up?

Considering Google been lying in their marketing videos about their AI capabilities, OpenAI seems to be falling apart from the inside and Facebook refuses to release open (not commercially-locked "open") models, there seems to be space for at least one honest and open AI company.

Google is trying and just can't? They are pretending they aren't behind despite setting a pile of cash on fire by spending exorbitant amounts on exaggerating their capabilities in basically fake videos.

Meanwhile you see Mistral casually dropping magnet links to weights with barely any instructions on how to use them on a Friday afternoon and you see reports of some very happy people finetuning on a civilian 4090 and getting GPT-4-quality performance in blind tests on Saturday morning.

Google is trying to catch up with OpenAI, not with open source 7B models.

Open Source 7B models have already caught up with OpenAI.

The current top of the open model 7B leaderboard beats GPT-3.5 and Bard while running on a laptop, smartphone, raspberry pi, or 10 year old graphics card.

Tim Dettmers just released code to get Mixtral 8x7b running in 4GB of RAM, the same amount required by Mistral 7b.

We are quite clearly a matter of weeks from Open Source being on par with GPT-4, likely before Google's Gemini Ultra is even released.

This is all to say nothing of the multi-model LLama-3 120B coming in 2 months according to insider leaks.

I’m very skeptical of LLM benchmarks because I’ve been testing models through Ollama and none of the open source 7b models are at even close to GPT4 (or Open AI’s GPT 3.5 implementation) in daily usage, despite what benchmarks say. LLM results are actually really hard to measure objectively because it depends on what kinds of questions you ask and what answers you expect.

I’ve just been finding lots of little errors and reasoning mistakes, and even in terms of producing useful results, they fall short of GPT4.

> Mistral's models outperform all of theirs at much smaller sizes and can be self-hosted.

This does not track with my experience. Mistral's models have bang for buck in that they are impressive for being so small, but they do not beat GPT4 at all.

Sure, but you can't host GPT-4.

No, but I look forward to the day I can trust Mistral results. Right now I have low confidence in them for any use that require at least a reasonable level of correctness. GPT4 on the other hand is correct enough for many use cases and much faster.

I’m also excited to see how new non transformer architectures like Mamba and state space models turn out.

> Good luck competing with Microsoft, Facebook and Google

Funny, considering people said the same thing about OpenAI and look who is eating FAANGs lunch now when it comes to AI?

Microsoft invested more than $10bn in OpenAI. Without Microsoft they wouldn't be able to thrive. Unless Elon and other investors were willing to invest billions.

So? People are investing in Mistral too (which this submission is about), albeit smaller sums, so far.

You said OpenAI is eating FAANG's lunch, which is only technically true because Microsoft isn't in that acronym.

OpenAI is part of the battle between big tech companies.

I strongly suspect that people are figuratively standing in line outside OpenAI headquarters with giant bags of money.

OpenAI is probably turning down multi-billion dollar investment offers on a weekly basis.

> Funny, considering people said the same thing about OpenAI and look who is eating FAANGs lunch now when it comes to AI?

Precisely how is OpenAI eating anyone's lunch? This is a serious question. Is OpenAI fantastically profitable? Does it have a proprietary moat that changes its margins in a way that no other competitor can overcome? Does it benefit from network effects that lead to a natural monopoly?

From my mole's eye view I see a raft of "use AI on task X" applications that are incremental improvements on the underlying products, not game-changing products themselves. I also see insane equity valuations, which look more like Gilded Age railroad speculation than actual proof it's different this time. [0]

Don't get me wrong. LLMs have some pretty amazing capabilities. The fact that they can perform what appears to be reasoning on huge pools of data is fascinating. But it seems premature to say that anyone has won here.

[0] https://www.smithsonianmag.com/history/robber-baron-gamble-r...

> Precisely how is OpenAI eating anyone's lunch? This is a serious question.

Because they made, maintain and offer GPT-4 which is currently the top-of-the-line LLM available. GPT-3 was also top-of-the-line when it launched, and they've proven they can iterate on the model and launch better ones.

Currently, both open and non-models are just about barely competitive with GPT-3.5, but none is close to GPT-4.

Even if they aren't profitable today, it's hard to imagine they won't be able to monetize things better, since their model is way ahead of any other model.

That is, they manage to get that far before the company collapses from the inside, which seems more and more likely everyday.

Isn't that just a win-on-execution model? Given that OpenAI does not exactly have stellar management and the fact we're not sure where the big economic wins for LLMs will occur it seems premature to declare them the winner.

Funny, considering people said the same thing about OpenAI and look who is eating FAANGs lunch now when it comes to AI?

I tried Bard yesterday (Im late like that) and I'm not switching back to a competitor anytime soon. The UX was smoother I found and i was blown away by the multimodal thing. It generated some python and the picture of a graph the python is supposed to draw, it was awesome experience. I told myself I trusted Google more and it integrates with their other apps so I could search my emails.

I mean, they are clearly not f-ing around. They're going Iron-Mike on it rather. Which is the point of parent comment.

The latest Bard is good, but at this time (Dec 2023), Bard is still not better than paid ChatGPT 4 in my experience.

I use both daily (I use ChatGPT on my phone and at home, Bard at work because work doesn't allow ChatGPT). ChatGPT 4 still provides the highest quality answers to the range of questions I'm interested in. Bard's getting better with Gemini but it's still not good.

Claude is great at summarizing PDFs and has one of the longest context windows, but the quality of answers it has to general questions is still lacking. Outside of summarization, I have little confidence in Claude's Q&A abilities.

I've also been using Perplexity.ai and I would say it hits a sweet spot and is impressive. It's essentially a RAG-based search engine that's fast and snappy and it works much better than Bing or ChatGPT Bing. Because it's primarily RAG driven, the content is primarily external and you get links to original sources so you can verify the answer. This means the level of hallucination is lower than foundational models. It's become a practical tool for me. It's CoPilot feature, which asks clarifying questions about your query, has been helpful for finding new products. For instance, I was looking for luggage packing cubes that convert to hanging cubes. It's a known product class but I didn't know the lingo, but over a few clarifying iterations, it was able to help me find the right product.

>I've also been using Perplexity.ai and I would say it hits a sweet spot and is impressive. It's essentially a RAG-based search engine that's fast and snappy and it works much better than Bing or ChatGPT Bing. Because it's primarily RAG driven, the content is primarily external and you get links to original sources so you can verify the answer. This means the level of hallucination is lower than foundational models.

That is what worries me about the Google Bard; it rarely gives references to its answers so I'm not sure if I can trust it or not but you can double check the answer by Google Bard searching it on Google Search and then giving you the source and its textual reference.

Where do you find the references from Bard?

There is the G button that you click after the answer appears which double-checks the result, but it only underlines parts of the answer that show it was or wasn't able to corroborate with search results. It doesn't seem to provide links to the relevant results themselves.

Whereas wiht Perplexity, the links are part of the answer.

G Double check button only gives one reference(at least in my examples) but it seems like Bard does not give references by design[1][2]. Websites are kinda getting ripped off, because Google is not referencing them and giving them exposure and traffic. But Bard is still in the experimental phase so I hope that will change.

[1] https://searchengineland.com/google-explains-why-bard-rarely...

[2] https://www.seroundtable.com/google-bard-wont-link-to-source...

We were the first investors in for-profit OpenAI.

OpenAI is a completely different story and setup than Mistral.

The localization of these broader services to trade and market demands is how they scale faster and deeper. Unless you think MS and Google and Amazon have Europe locked down now?

> Good luck competing with Microsoft, Facebook and Google

Well, that's what the funding is for. These guys already proved they have the necessary expertise - Mistral models are top notch. Now they need the money (and time) required to train on large datasets.

> Good luck competing with Microsoft, Facebook and Google, because they are super-focused now on LLMs and generative AI and they are not going to stop.

Remember when Google was gonna kick Facebook's ass in social networking? All four times?

What is the end game for companies releasing open source models? Any ideas for a sustainable business?

I know someone will hound me on the definition of open source but...

I'd bet on one of four things

1. Early access for a fee. Get the latest model at a cost, they're later fully open sourced.

2. Sell commercial use licenses. Get and validate the model works for you easily, then buy a license. Split by company revenue if you want to

3. Sell hosted versions - these things require annoying tech to host, paying for someone else to manage running machines can help (possibly combined with a license around not setting up a resale of the LLM)

4. Sell the service of producing (and possibly hosting) custom models. Prove your worth with the open source ones, so you're a trusted person to come to in order to get a model custom made.

I've also wondered this. Aside from just securing funding, I could see:

1. Offering a paid hosted solution down the road

2. Offering bigger / better models that cost money to use commercially

3. Consulting services around their models

4. Licensing their models to companies that want to host them as part of a paid offering

IMO the whole advantage of small, open source models is to avoid paying for hosted solutions and additional privacy / security, so it is a bit at odds with commercialization.

The potential for AI is one of those things that I think a lot of people are willing to fund without a return on their investment. The nations and businesses and individuals that leverage this technology will far outpace those that don't. It has value in and of itself to have access to the most powerful model capable of existing. There is a reason governments are starting to pass laws. Clearly this technology is seen as a huge asset to anyone who's paying attention. I am not making a moral argument here. I don't know if what's coming is good or bad. But it's coming with our without my approval.

To add to this, it is a valid strategy to give away something for free so attention - and therefore power - is taken away from a competitor. A quick way for everyone to forget OpenAI exists is for another nation or business conglomerate to release a model that is "good enough". This slows growth for businesses trying to profit from it.

As an open multi modal we went for a flat membership scheme we are introducing:

https://x.com/EMostaque/status/1729609312601887109?s=20

$1 => $100k+ flat for all models we release depending on company size/type, then upsell support, early access, custom models, other services.

I think that aligns with growth of market as a base to cover model costs and is a strong recurring revenue stream while being predictable.

This incentivises us to release more good models of every type to drive more subscriptions, much like Netflix with hit TV shows. So we are doing a dozen languages, common models improved as well as state of the art ones.

We also have commercial variants eg our www.stableaudio.com model that per licensing we can't release open.

Usually companies have an open core business model for source code, but models are a bit different: https://en.wikipedia.org/wiki/Open-core_model

It will be interesting to see how others do it, particularly with incredibly strong models from Chinese companies coming through.

7B open model as a proof of 'we know what we're doing, try it yourself with zero paperwork' and 80B/220B closed, paid 'so you're serious, yes we can, here's an openai-compatible API and our account number' model.

Previously: OpenAI Rival Mistral Nears $2B Valuation with Andreessen Horowitz Backing (6 days ago, 2 points, 0 comments)[0], OpenAI Rival Mistral Nears $2B Valuation with Andreessen Horowitz Backing (5 days ago, 9 points, 1 comment)[1]

[0]: https://news.ycombinator.com/item?id=38522873 [1]: https://news.ycombinator.com/item?id=38533725

Does anyone know the advantage of mistral over llama2? I understand the founders of mistral previously worked on llama.

Mistral 7b version is slightly better than 7b llama based on elo rating https://huggingface.co/spaces/lmsys/chatbot-arena-leaderboar...

Also less censored

there is also llama uncensored https://ollama.ai/blog/run-llama2-uncensored-locally

It's better than that. The best Llama-2 finetune is 88th on the 7B leaderboard, behind 87 Mistral finetunes.

Mistral 7B is above every Llama-2 finetune, including the 10x larger llama-2 70B, on the Open LLM Leaderboard.

https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderb...

(Note: Check "architecture" and uncheck precision "?" then ignore "tigerbot" as it was disqualified for cheating.)

Do LLM providers with smaller / less capable LLMs still have a USP or will it be like what we witnessed in the search space, where the #1 company has more than 90% market share, #2 has less than 5% market share and everyone else less than 1%?

If the model is closed and served through an API, probably not.

But at the moment Mistral seems to focus on smaller and efficient models that you can fine-tuned and run on your own infrastructure.

Mistral 7b was free but if they come up with larger capable models, they can offer them with a non-commercial free license and a sell a commercial license for companies that need to fine-tune it.

Looking at the performance of small fine-tuned models on specific tasks, I think most company will prefer taking that route rather than trusting OpenAI with their own data, the market is huge I think.

> trusting OpenAI with their own data

often that's a GDPR violation. there is not such thing as perfect data scrubbing.

You can definitely transfer data to another processor without breaking GDPR rules, even US ones. You need to take care, but it doesn't require perfect data scrubbing.

My hope, being fair to Mistral, is that they keep supporting the 7B model series, and release a set of products such as: one off non-commercial and commercial model purchases of more capable 13B, 20B, and 30B models. I would also like to see a subscription service so, if for example, I was using 13B commercially, I could pay a yearly fee for updates and support for that model.

EDIT: since I am retired and just obsess about LLMs and in general deep learning as a ‘gentleman scientist’ I hope Mistral has inexpensive non-commercial options for larger, more capable, and commercially supported models.

Hmm... I would think the on-premise search for companies falls within the "less than 1% for everybody else" category I mentioned in regards to search.

The revenue of Elastic Search is less than 0.5% of Google's revenue.

Isn't the recent AI regulation voted in Europe a problem for this company?

With that valuation, they could pick any country they like

> Isn't the recent AI regulation voted in Europe a problem for this company?

Maybe not that big of a problem but the EU legislation that just passed wants the companies building models to hand over technical documentation, comply with EU copyright laws and give detailed information about the data used for training the models. So basically I take it's there's going to be some annoying administrative work to do but it doesn't seem impossible to comply with.

"The European Union's landmark Artificial Intelligence (AI) Act may exempt open-source models from strict regulation, according to a leaked compromise proposal seen by Reuters." [1]

And Mistral has been betting on an open-source first approach. One might even wonder if there is some causation between the two decisions, one direction or the other.

[1]: https://www.reuters.com/technology/eus-ai-act-could-exclude-...

The EU would have to be crazy to not support smaller local European companies like Mistral (a French company). So good for their economy and securing their own AI infrastructure.

I have ranted about this before on HN, so bear with me here: all countries should take reasonable steps that they can afford to secure their own infrastructure: energy, water, food, some local manufacturing, IT, AI, education, etc. This “one world order” stuff is bull-shit. The world is a better place with many cultures, ethnicities, more self-sufficiency, etc. My wife and I like to travel to meet people with very different backgrounds, see new places. I cringe thinking of a “vanilla” world where large corporations and elite interests “homogenize” the world.

The EU hasn’t done much to build local Big Tech companies, why would it be different with AI?

I think the EU and its member countries are largely content with taxing American tech giants - no need to rock the boat.

It doesn't substantially change your question but it hasn't been voted on yet, an agreement between the negotiators for parliament and council has been made, it'll likely pass but we don't even have the actual text

As for the actual question, there was talk about foundational models being subject to less scrutiny than the applications using them. So it's quite possible it won't be a problem for Mistral or other LLM companies

But again, we don't actually know what the latest agreement was and what the new draft will be like

> Isn't the recent AI regulation voted in Europe a problem for this company?

The world is a big place. And moving the talent to a place less regulation-crazy than the EU is not very hard.

what is their moat over other companies releasing "open-source" models, a la stability and meta?

all the fanfare during their inception seems to have washed away tbh.

Did they release the source code or benchmarks for their new 8x model?

No. Without the inference code, the best we can have are guesses on its implementation, so the benchmark figures we can get could be quite wrong. It does seem better than Llama2-70B in my tests[0], which rely on the work done by Dmytro Dzhulgakov[1] and DiscoResearch[2].

But the point of releasing on bittorrent is to see the effervescence in hobbyist research and early attempts at MoE quantization, which are already ongoing[3]. They are benefitting from the community.

[0]: https://twitter.com/espadrine/status/1733505332293288256

[1]: https://github.com/dzhulgakov/llama-mistral

[2]: https://huggingface.co/DiscoResearch/mixtral-7b-8expert

[3]: https://github.com/TimDettmers/bitsandbytes/tree/sparse_moe

Apparently you can download it via torrent[0].

https://venturebeat.com/ai/mistral-ai-bucks-release-trend-by...

First-hand source: https://twitter.com/MistralAI/status/1706877320844509405

Mistral doesn't release source code or data just weights. They did include some details of sliding window attention previously

They released a reference implementation with Mistral 7B and a fork of a framework they probably used to train the Mixture of Experts for the new one.

But it's still early and I guess that they kept just enough mystery to kickstart the conversation around this new model!

Reply on news.ycombinator.com