Its really a matter of having the capital for training. Same with the Devin AI coder, its just VC pumped crap. Same with Mistral, they have no moat, and their researchers, as "prestigious" as they are, are completely undifferentiated.
Its really a matter of having the capital for training. Same with the Devin AI coder, its just VC pumped crap. Same with Mistral, they have no moat, and their researchers, as "prestigious" as they are, are completely undifferentiated.
Remember sometimes they joy comes from owning horses and being in the race even though horses are (almost) completely undifferentiated for the untrained eye.
Qwen has always out-performed other equivalent/contemporary models on Chinese-language tasks, so it wouldn't surprise me if it continued to do so vs LLaMa 3.
In 2 years, when compute costs are 10x cheaper or whatever, every developer at Mistral will be running a chatbot or flight planning team at American Airlines.
But the answer is bubbles. Any time sudden money is made in anything it attracts everyone from everywhere and it immediately becomes corrupted and full of scams and old money institutions. Suddenly people aren't becoming developers to innovate but to become personally financially stable. What started out as mostly uneducated hipsters and hacktivists disrupting and improving is now academia, major corporations, wealthy heirs with their WeWork NFT companies, etc. soaking up what's left of the funds, stagnating the industry, gatekeeping it, and playing a totally different (and highly political) game than we were playing in ~2008-2016.
When the tech world came crashing down in ~2016, at that time there was still a lot to disrupt: Pre-Tik Tok, largely still pre- crypto and AI. SaaS and mobile had reached a peak, and we were ready for something new, but I had no idea what was coming lol - Trump and Hilary and politics, then Covid, and now nobody has jobs like under Bush all over again, it's all politics and it's never been worse for a person's image to identify as a software engineer. This is how it was before it was cool though, nobody wanted to be a developer in ~2003.
But it's a necessary cycle, you can't just keep pumping money endlessly, it gets ridiculous quickly. There has to be periods of on and off and extreme hype cycles to see if something might be, and like a kite or firework some of those take off and impress us, but make no mistake they're all - necessarily - bubbles! Get in while it's hot, get out before it bursts :)
Meta does not have unlimited financial firepower to release models for free. Its like saying you can't compete against someone who burns money. In theory true, in practice the 'someone' can run out of money.
Chinese models can get state backing. Mistral has French state backing etc. There's plenty of money to go around for huge technologies like this.
Zuckerberg’s budget for side projects is bigger than most countries’ defence budgets.
https://www.fool.com/investing/2024/04/01/meta-platforms-has...