They invented the transformer architecture that powers GPT and other llms.
The fact Google did the initial work on transformers but only OpenAI was able to productize is an indictment of their stagnation more than an achievement.
If websites can't earn a living through ads because LLM's don't send them any traffic anymore, where do the LLM's ingest new information from?
The web continues to deteriorate because of this and nobody has solved this problem.
AI is messy
LLMs have to have something ‘true’ (or at least statistically probable) to validate against for training.
Taking the output from it and feeding it back into itself is, essentially, an Ouroboros. Or like locking a human in solitary.
The more/longer it happens, the more deranged it’s going to get.
It's culmination of decades of NLP researches, but it's also just East Asian predictive text percussively adapted for variable word length language. It must've been clear to whoever core people that it's fundamentally a sluggish demo or at most a cheap outright-sold commodity product.
IMO they would be right unless someone finds a reason that LLMs can't ever be self hosted in the way Google Search can't be. They must have just saw by the way it is that LLM is at best a regularization engine for that magic box.
What do you mean? Anthropic and Google both have widely used products based on transformers.
All of these companies are in the red, but OpenAI has the most revenue.
https://trends.google.com/trends/explore?date=today%205-y&ge...
2017, seven Google employees invent the transformer architecture and publish a paper. Google's investing heavily into ML, with their own custom 'TPU' chips and their 'Tensorflow' ML framework.
2019ish, Google has an internal chatbot they decide to do absolutely nothing with. Some idiot tells the press it's sentient, and they fire him.
2022, ChatGPT launches. It proves really powerful, a product loads of individuals and businesses are ready to pay for, and the value of the company skyrockets.
2023, none of the seven Transformer paper authors are at Google any more. Google rushes out Bard. Turns out they don't have a sentient super-intelligence after all. In fact it's badly received enough they end up needing to rebrand it a few months later.
Classic tortoise-and-hare situation - Google spent 5 years napping, then had to sprint flat out just to take third place.
At worst, Google is in a tie for first.
In the market, I would say both Anthropic and openAI have been able to do that much better than traditional big tech including Google.
E.g. when I start a search session to solve a programming problem (before llms), I will continually search different terms to get to my solution webpage. Then stop. This session metadata and the path I took is highly significant data that can be used to help llms recognise what research itself looks like.