You want both a backup for OpenAI as well as negotiating leverage if OpenAI gets too powerful and this achieves both.
I'd be surprised if they didn't consider the notion that they are hitting to birds with one stone: OpenAI and Indie AI.
> Mistral Remove "Committing to open models" from their website
That was 5 hours ago.
Without having insider details it is hard to know why, but the coincidence of timing with the Microsoft deal is not lost on me. It could have even been a stipulation.
So... would Mistral deliberately sabotage their low-end models to appease Microsoft's cloud demand? I don't think so. Microsoft probably knows that letting Mistral fall behind would devalue their investment. It makes more sense to bolster the small models to increase demand for the larger ones, at least from where I'm standing.
This is why anti-CSAM measures policy is possible so compiled-release LLMs can have certain vector spaces removed before release; but apparently people are creating cracks for these types of locks?
Not sure where you are getting the CSAM bit. We aren’t that good at blanking out weights in any kind of model, certainly not good enough to lobotomize specific types of content.
The CSAM bit seems to then be propaganda from at least one AI company putting out PR to falsely quell people's concerns about their LLMs being able to generate content involving children that's sexualized.
I've yet to see details of how much compute-minimum server requirements are necessary to run LLMs. Maybe you know a source who's compiling a list in a feature matrix that includes such details?
Diversifying their AI bets definitely makes total sense. If this wasn't their strategy originally, it almost certainly became so the moment the OpenAI board fired Sam Altman.
It's easy to make simplistic judgements from the outside, but with the limited information we have, it does seem like Satya Nadella came out of this OpenAI debacle looking pretty competent.
It's hard to reconcile the fact that the Microsoft that handled the unexpected OpenAI issue so well is the same Microsoft that seems intent on literally setting fire to their flagship product! (Windows)
Step 1: Get the industry leaders to be purchasable via Azure. Step 2: Slowly build your own clone and start stealing user share even though your offering is still worse.
> Nadella [in December 2022] abruptly cut off Lee midsentence, demanding to know how OpenAI had managed to surpass the capabilities of the AI project Microsoft’s 1,500-person research team had been working on for decades. “OpenAI built this with 250 people,” Nadella said, according to Lee, who is executive vice president and head of Microsoft Research. “Why do we have Microsoft Research at all?”
> At the same time, even as the company began weaving OpenAI into the fabric of Microsoft’s products, Nadella decided not to abort Microsoft’s own research efforts in AI. During the tense exchange at the December meeting between the Microsoft CEO and Lee, other executives spoke up to defend the work of Microsoft’s researchers, including Mikhail Parakhin, who oversees Microsoft’s Bing search and Edge browser groups, Lee said. After grilling Lee in the meeting, Nadella called him privately, thanking him for the work Microsoft Research had done to understand and implement OpenAI’s work in a way that passed muster for corporate customers. Nadella said he saw Lee’s group as a “secret weapon.”
While this is entirely speculation, it's easy to imagine that there are many levels of PR magic going on here, to share a quote that on the surface feels "leaked" and "explosive" but, among investors and clients who read beyond the (very good) paywall, actually shores up a narrative that Microsoft has a capability that significantly augments OpenAI's, and allows the existence of MSR to become headline news without even needing a product release.
The Mistral deal feels like yet another step in this direction. Microsoft is not afraid of seeming "messy" in the press as long as it can control the narrative around its value-add to customers in the context of its partnerships. By contrast, the rest of FAANG's more consumer-facing positioning makes it a lot harder for them to maneuver in a similar way.
The answer to that is till Google released the Attention is All You Need paper in 2017 there were no breakthroughs allowing models as we have now to be built, OpenAI being a small and nible team picked up on which direction the wind is blowing with LLMs and quickly brought a product to market whilst MS just did what corps do - move slowly (same for Google etc).
The con is that MS attracts more attention from regulators.
Altman is out there trying to raise ridiculous sums to get away from Azure, didn't he make the first move here?
Basically something that is more than just another bump in the scorecard for GPT 5 over GPT 4. Otherwise it is still just a horse race between relatively interchangeable GPT engines.
Until OpenAI releases GPT 5 and it blows everyone away, OpenAI's leverage is constantly decreasing as the gap between their best model and everyone else's best model decreases.
There doesn't seem to be moats right now in this industry except for pure model performance.
Maybe someone should as ChatGPT what OpenAI should do to maintain long-term leadership in this industry?
They might, in an upside-down world where the Shockley Semiconductor board tried to fire Shockley, and where the Traitorous Eight not only didn't bail out but took his side.
And when I was a kid, it seemed like all the teachers thought it would be a waste of time to learn MacOS because "Apple would be bankrupt soon". (Given how much all the app UIs changed, right decision for the wrong reason).
Those integrated AI solutions will usually be done via enterprise deals where brand name is not quite as important. It will be done by people who care about cost, reliability and ease of use.
Think of nginx's dominance in web servers even though it has no name recognition among the general population. Or Stripe's payment system.
https://techcrunch.com/2024/02/15/no-gpt-trademark-for-opena...
Hard disagree. OpenAI's function calling is something no other commercial model provides, not even Gemini and Mistral Large.
Compute?
At least in the short term, it seems like the biggest wallets are going to win by default.
Hire Ilya, get him to hire as many of the best folks he can.
Stop selling GPUs. Hoard them. Introduce some subtle bug into the drivers that dramatically increases their rate of burn out.
Figure out some reasonable way to give attribution to original content creators, approximately solve the content ID problem of the AI age. Cut the content creators into the rev share in proportion to their data importance to the model. Make the content creators incredibly pissed off that their work is being stolen by big AI companies unfairly and encourage to them to sue the other big AI firms. Their content share multiplier increases if they get injunctions against LLM firms.
Convince politicians that the AI firms have performed an intellectual heist of epic proportions, and that they must not be allowed to even generate synthetic training data from poisoned models. With the content creators united behind you, convince congress that poisoned models must be destroyed, that even using synthetic training data from poisoned models must be illegal. Make them start over from a clean room with no copyrighted data.
do runs of cards for themselves with higher core counts and clock speed that they dont release to others.
And when such models become popular[0], all the artists now have no job and no way to get compensation for being unable to work through no fault of their own.
I don't think that's really a winning condition. It might make you feel better about the world, but the end result is still all the artists being out of work.
[0] some models are already trained that way, although I assume you're using the word "copyrighted" in the conventional sense of "neither public domain nor an open license", as e.g. all my MIT licensed stuff is still copyrighted but it's fine to use.
There is still also money to be made in producing physical art or performances, even when AI can produce amazing digital works.
> There is still also money to be made in producing physical art or performances, even when AI can produce amazing digital works.
Perhaps, but it may be akin to the way there is still money to be made from horse drawn carriages in city centres, even when cars displaced them over a century ago — a rare treat for special occasions, to demonstrate wealth.
Is it not the same with every leap in technology? There were professions like street lamp lighters, alarm services etc that have become redundant now?
More specifically, I was responding to the idea that "compensating creators whose works are used to train the models" would actually solve anything; to use your examples, it would be as if the literal luddites were suggesting passing laws saying that "all textile machines that work like humans need to compensate the humans they displace, and also you need to make your new machines from scratch without talking to any textile workers to make sure you don't cheat", and my response would be analogous to saying "there's already machines which don't work like humans, so you're going to be out of work and have no compensation".
The Luddite movement preceded The Communist Manifesto by about 30 years. Everything's sped up since then, so I'd be surprised if we have to wait 30 years for a political shift which is to AI what Communism was to industrialisation. I'm just hoping we don't get someone analogous to Stalin or Pol Pot this time.
I've thought the same thing. NVIDIA getting into AI seriously is a vertical integration play and they often do that -- like NVIDIA trying to buy ARM.
Well, they didn't stop selling GPU when cryptomining was going strong. Instead they continued to sell them (with a hefty markup, tho).
It's like selling shovels and picks during a gold rush.
And then OpenAI tripped and fell over a magic money printing factory, and the complaints are now in the set ["it's just a stochastic parrot", "it's so good it's a professional threat to $category", "they've lobotomised it", "they don't have a moat", "they're too expensive"].
As the saying goes, "Prediction is very difficult, especially if it’s about the future!"
OpenAI has 700 people, Microsoft has 220,000 people.
OpenAI is strong but they're still dependent on MSFT.
Microsoft will want to avoid things regulators in the current regime will go after.
This seems like a step towards both and ultimately good for developers as it seems likely to bring costs down by increasing competition.
This way Microsoft is less dependent on a single deal and can diversify their offering based on use cases.
> [EEE] describe its strategy for entering product categories involving widely used standards, extending those standards with proprietary capabilities, and then using those differences in order to strongly disadvantage its competitors.
But yeah, not sure it is at play here.
Where are the "widely used standards"? Where are the "extending the standards with proprietary capabilities"? Where is the "strongly disadvantaging competitors"?