Hacker News Reader

Comment by brandall10 | Hacker News Reader

I'm surprised this isn't firmly attached to the top of HN right now for the entire day.

This is a tiny company (appears 30 or so people?) that just scored a 2B valuation, produced easily the most performant 7B model and a 7B*8 MOE model that performs at the level of a 70B requiring the inference power of a 14B.

I feel this could be a potential bigger threat to OpenAI than Google or Anthropic. I gather with the huge recent investment they'll be able to a) scale out to a reasonable traffic load in the near future and b) attract the best and brightest researchers put off w/ various chest puffing and drama that has been front and center in this industry.

View on HN

It's because there have been quite a few big threads already:

Mixtral of experts - https://news.ycombinator.com/item?id=38598559 - Dec 2023 (272 comments)

Mistral-8x7B-Chat - https://news.ycombinator.com/item?id=38594578 - Dec 2023 (69 comments)

Mistral AI Valued at $2B - https://news.ycombinator.com/item?id=38593616 - Dec 2023 (221 comments)

Mistral's mixtral-8x7B-32kseqlen on Vercel - https://news.ycombinator.com/item?id=38584179 - Dec 2023 (30 comments)

French AI startup Mistral secures €2B valuation - https://news.ycombinator.com/item?id=38580758 - Dec 2023 (76 comments)

Mistral "Mixtral" 8x7B 32k model [magnet] - https://news.ycombinator.com/item?id=38570537 - Dec 2023 (236 comments)

I know these are not all the same exact story but the discussions end up being more or less generically the same, so we can call them all (or most of them) quasidupes.

Yes, I'm aware there's been a ton of recent chatter. Most of these are dupes though so it's really just Mixtral, their funding, and now their commercial offering.

My argument is opening up the commercial offering is perhaps the biggest story yet simply because of all excitement this company has been able to generate.

Agreed. By far the most impressive company out of the current AI wave.

They managed to put out the gold standard for 7B models in like 6 months, and are quickly moving up the scale.

I mocked the funding round back in March as being signs of hype, ($300m for a team of 3 with just an idea?) but clearly I didn't know the details. Really remarkable execution.

They may well be on their way to eat all use cases that don't need gpt-4 performance and hopefully soon tackle the big leagues as well. Exciting times!

In some ways I see this as a sign that LLMs possibly aren’t that hard to iterate on now that the cat is out of the bag. A small team can push the state of the art in just 6 months? It feels like this is more a circumstance of the state of LLMs presently and less a statement on Mistral. I wouldn’t it be surprised if we see 10+ other new $1B+ companies spring up because of how much room for improvement there is in the space.

Well said. OpenAI's biggest contribution wasn't serving GPT-4, it was proving that big LLMs are AGIs. Rushing to gold is easy; discovering it ain't. Since we've found a correct path, low-hanging fruit is plentiful. Throw more compute at it. Work smarter not harder (optimize). Graft on simple ideas for big gainz (e.g. RAG, MoE, chain-of-thought, etc). We moved from "GPT Who?" to...all of this, in a year. I now run an AGI at a bookish 13-year-old's level on my MacBook. The trend of hardware/software advances are astonishing. And this is just the beginning.

AGI, thanks 4 lmao. Either that or you're trolling.

I figure AGI will need a lot of parts to it, the same way our brain does, but having the language part figured out is I would guess is about 1/4 of what we need to solve to get there. So it's not AGI, but I would say it's a big chunk of what we need to get to AGI.

I have to disagree. I think language will be emergent from AGI, not the other way round.

LLMs are neat, but likely wont have a place in an AGI.

> I'm surprised this isn't firmly attached to the top of HN right now for the entire day.

It's kind of hard to tell what it is from both the blog post and their homepage. So only people really familiar with AI will grasp the relevance. But your comment certainly helps.

Mistral is in France though and the EU is throwing cold water on "AI" at the moment. I think this hurts them in the long run.

Or it could help them be more globally competitive because they’ll have built their product to accommodate the EU rules and will be ready if/when other countries introduce something similar.

Maybe in 1970 someone was arguing that Volvo is handicapped because Swedish safety regulations have become so stringent that their cars can’t compete on price elsewhere. Turns out it was exactly Volvo’s strength to be ahead of the curve on regulation.

Regulations are accumulative though, not just relative to your era. So a critique in 2023 would be more relevant than one in 1970 as the pure scale of it has increased dramatically.

I've seen a few graphs showing staffing counts and budgets for US regulatory agencies have grown exponentially since the 1970s (adjusted for inflation), while it was relatively flat before then in the 60s. Economic regulation growth in the US has grown steadily while social regulations (workplace, climate, healthcare, transportation etc) grew dramatically. Adding in TSA / Homeland security and it's a hockey stick.

https://regulatorystudies.columbian.gwu.edu/sites/g/files/za...

I'd be curious to see similar charts for the EU.

The EU’s new AI Act is the first regulatory action in the world targeted specifically at AI models. In that sense it’s now 1970 for this domain specifically.

In a democracy I'd expect regulation to grow alongside population and productivity. It would be interesting if it didn't, actually.

It's possible, but the AI space right now is all about the vibes and even the French President isn't feeling it[0]. The job market for AI/ML people is white hot right now and this is something Mistral has to try to explain around in an interview. I hope they do well despite it.

[0]https://www.ft.com/content/9339d104-7b0c-42b8-9316-72226dd4e...

> Turns out it was exactly Volvo’s strength to be ahead of the curve on regulation.

That's a weird analogy. I don't remember Volvo ever being that big. If anything they're bigger now than they used to be (I mean: I don't remember a world with lots of Volvo when I was a kid... I may be wrong though). And with about 650 000 vehicles sold per year they're not in the Top 25 car manufacturer worldwide by number of cars sold.

Yeah I think most people would consider Volvo - a standard bearer for safe vehicles for over 100 years - a success

Have to see the final text, of course, but open source models apparently got some breaks in the final agreement last Friday.

The EU makes in it’s proposed law a lot of exceptions for open models. It might be a big selling point for mistral (or Facebook) but maybe tricky to deal with for closed models like OpenAI.

Except that they're not throwing cold water on opensource models. So, maybe that helps them in the long run.

Also, I'm not sure how much the EU is really going to matter for the future of AI.

You don't really need to conform to their regressive tendencies if you don't want to do business with the EU. Or you could dumb your stuff down for them if it ever comes down to it.

Generally, it just wouldn't be worth it to care too much about what the EU thinks.

French engineering and computer science education really emphasizes math and theory. That is an advantage in AI.

Reply on news.ycombinator.com