But, according to the metrics, it barely edges out GPT-4 -- this mostly makes me _more_ impressed with GPT-4 which:
- came out 9 months ago AND
- had no direct competition to beat (you know Google wasn't going to release Gemini until it beat GPT-4)
Looking forward to trying this out and then seeing OpenAI's answer
We might soon get to a point where every player is using pretty much all the low-cost data there is. Everyone will use all the public internet data there is, augmented by as much private datasets as they can afford.
The improvements we can expect to see in the next few years look like a Drake equation.
LLM performance delta = data quality x data quantity x transformer architecture tweaks x compute cost x talent x time.
The ceiling for the cost parameters in this equation are determined by expected market opportunity, at the margin - how much more of the market can you capture if you have the better tech.
What? Why?
> data quality x data quantity x transformer architecture tweaks x compute cost x talent x time.
Google arguably has the most data (it's search index), the best data (ranked and curated already, along with data sets like books), the cheapest compute (they literally run their own cloud offering and are one of the biggest purchasers of H100s), and the oldest and most mature ML team.
Google has the best Internet search engine bar none and personally I'd not normally use Bing if not through ChatGPT.
It has Google Book, and I believe it has been scanning books for more than a decade now. It good to know that, so when the next time Mongol-like invasion happen (as happened to old City of Baghdad) all the books contents are well backup /s
It has Google Patent, and the original idea of patenting is for knowledge dissemination in return of royalty, and that knowledge would otherwise locked behind industry closed door.
It has Google Scholar, some of the papers are behind paywall but most of the contents are already cached somewhere (e.g. Pre-Print servers, Sci-Hub, online thesis portal).
It has Google Video aka YouTube that by watching all the uploaded videos within one hour duration to YT platform, will probably last more than your lifetime (assuming lifetime watching videos doing nothing else from cradle to grave non-stop without sleeping).
Ultimately it has Google mail or Gmail and to say that Google do not access the emails on its platform it's providing for free is naive and almost all my colleagues, friends, acquaintances (people that I know personally) have Gmail.
UK ex-PM (no prize of correctly guessing who) was once said on national TV that "Google probably know about him than he knows about himself" (TM).
Google once claimed that no one has moat on LLM but from the planet that I live none has organized the world's information like Google and ironically the CEO just reminded us in the Gemini video introduction that Google corporate mission statement is to organize the world's information and AI, LLM, RAG (insert your favourite acronym soup here) are the natural extensions of what they have been doing all along.