Mistral AI raises $640M at $6B valuation
generalcatalyst.com
generalcatalyst.com
For companies that are built off these AI services, there's a very real fear that the prices we're getting on API calls aren't covering the costs of the requests themselves under some assumption that costs will magically go down, or we'll get locked in.
I hope the companies like Mistral and OpenAI know that if they're selling API access below cost they could be the origin of a __lot__ of companies closing and choose to make sure their operating costs are sustainable. (I understand creating new features and new models taxes additional capitol and that's a perfectly fine use of VC money. But selling a $1.00 for $0.90 is dangerous and bad.)
I'm guessing that's going to be the needle that finally pops this bubble, because as neat as a lot of shit going on with various models is, nobody is going to pay what it actually costs to run the things in order to have that neatness.
Currently their lowest-tier price is $20/month, and limits users to 80 queries per hour. If we assume that 7 cents number and our hypothetical user actually uses those 80 queries for whatever odd reason, that's $5.60 it costs OpenAI, which if that user used them for say, 8 hours (the typical work day), that's $44.80 for OpenAI on a user who's only paid $20. And that's one day, that $20 currently gets them a month of access to that 80 queries per hour. I hope for the dream of profitability that they would get throttled at some point. So the user could, in theory, already render their subscription a net loss for OpenAI with 4 workdays of steady usage.
And yes this is strictly back-of-napkin math here but it would explain why "OpenAI" and "bankrupt" keep appearing in the tech press over and over.
You're going to have to bring some actual numbers on that speculation.
You can actually see it in paid, web stacks. There’s free ones, esp FOSS, that are really good. So, the paid contenders try to be better at other things. There’s many of them in the market, too, despite over a decade of free alternatives.
It's a well-established, economically rational business model to subsidize prices in order to gain market share.
Uber has been very successful at it. Amazon has had great success. It's standard.
Investors know what they're doing and know that some companies won't be able to sustain it and will go out of business. It's a calculated, known risk.
And users don't really need to be concerned. Just enjoy the low prices while they last. If the service you use goes out of business, it's pretty easy to switch to another.
> It's a well-established, economically rational business model to subsidize prices in order to gain market share.
Because once they have the market share, they raise prices and/or make the product worse in order to get their money back.
> And users don't really need to be concerned. Just enjoy the low prices. If the service you use goes out of business, it's pretty easy to switch to another.
Well that's the question, isn't it? If the market stays healthy and they don't take the oxygen out of the room and kill the competition and switching costs stay low, then yes by all means enjoy the free ride but plan for its end. But... do plan for it.
They should only be able to do that if they have a considerable moat though?
Right but it means you can't depend on that price remaining low. If it's below cost, the price must rise. That may be important for anyone building businesses on top.
> it's pretty easy to switch to another.
Well that assumes the others have true lower prices.
Well yes but every business owner knows that, or should know that. That's a known business risk. We're all adults here.
If your business fails because you naively assumed that prices you rely on will never go up, then that's on you.
By taking outside money and subsidizing certain market players so that they can sell their goods at below market value (because that is what is happening) you are undermining the market and punishing players who are acting rationally.
This would be absolutely no different than taking tax money and subsidizing certain companies over others to destroy competition and then claiming that it was a market decision.
'We are all adults here' is not a magical phrase that makes cheating OK.
Know that they are buying services below cost? I don't think so.
It's a reasonable bet that compute costs generally trend downwards, and in a competitive market where the key thing you're paying for is compute that should also trend downwards.
That's different to finding out that everyone's been selling things way under the true cost.
To the contrary, of course you should know.
If you know anything about tech businesses, you know that startups being funded by massive amounts of VC keep prices artificially low initially, and that they then rise.
This is tech startup 101 stuff.
It's fine if you're not aware of this as an engineer. But if you're building a business and therefore have a business model where you've identified known risks, there's no excuse for you not to be aware of it.
Is openai selling below cost? Is the cost on azure lower than the true cost? ChatGPT free is obviously run subsidized but are the apis?
Can you tell me the actual cost to Microsoft of a gpt4 API call?
https://en.wikipedia.org/wiki/Enshittification
This is how the world gets shittier. It's a whole thing. It's a large part of why things are as bad as they are.
But hey, yeah, don't worry about it, they obviously know what they're doing.
If I launch a business that makes sense given the current pricing of LLMs, and the price then goes up by a factor of ten, my own unit economics may not make sense any more.
There's an Uber comparison here: back when Uber was cheap, some people made life decisions about where they lived and worked that were predicated on being able to afford a commute in an Uber (since public transport doesn't serve a lot of places, at least in the US).
This hurt a lot when the prices went up a few years later!
As a consumer it should definitely concern you if you're using poorly financed or unsustainable products. Maybe it's great now but I would rather have a slightly more expensive product that is more stable than one that suddenly had to make the experience a lot worse because they burned a lot of cash and had a down round.
Economically rational for the business owner, maybe, but it inevitably results in eventual enshittification for the consumer. (And Uber isn't a great example of this; its share price has only recently surpassed what it was at IPO 4 years ago, and many companies don't even do that well)
I'm not sure how well that works if vendors differentiate themselves through proprietary models and backend integrations. As far as I know, Mistral only releases weights for some of its models, with others being proprietary.
Don't actually spend the business effort to use them if it's not worth it, but have a business continuity plan to be able to switch over, especially if a "single source provider" going out of business or changing it's pricing could represent a existential thread to your business.
It's the same as any other tech vendor.
Imo. A company can use whatever wrapper services but the business they serve should be smarter.
So, in most cases they have no moat. They are going to close, and while some will close due to operational cost issues as you identified, a lot more will close since they have no defensible business/revenue model.
Or, even better if they can price future API price increases in their own pricing, today.
So at least for current gen, you're ok.
If it's the latter, way more startups on SOTA models will become unviable once the investor subsidies dry up.
(for example, deepinfra has wizardLM-2-8x22B at $0.65/1M output tokens, compared to $6/1M output tokens for 8x22B by Mistral - and of course Mistral has some more expensive, closed source models that perform better)
I'm working on a local product (coming soon™) that uses Mistral-7B-Instruct-v0.2 under the hood. Parsing screen reader text (and taking actions, etc.) via local LLMs is, in my humble opinion, the future of computing. Even though getting it right is hard, and there's lots of edge cases, it's pretty awesome when it works. Mistral has been (by far) the most reliable model to date. I'm likely going to be fine tuning it over the next few months, but here it is working on Windows and MacOS in both web browser[1] and file manager[2] contexts.
[1] https://vimeo.com/931907811
[2] https://dvt.name/wp-content/uploads/2024/04/image-11.png
Same reason Cohere has a weirdly-high valuation relative to revenue: it's a national champion.
edit: If you're that curious just call Hemant up and ask him yourself?
Mistral AI employee: https://x.com/dchaplot/status/1800601722722935153
TechCrunch: https://techcrunch.com/2024/06/11/paris-based-ai-startup-mis...
Wall Street Journal: https://www.wsj.com/tech/ai/french-startup-mistral-ai-raises...
For reference: https://en.wikipedia.org/wiki/Mistral_(wind)
(this kills the joke)
Bizzare