[1] I would expect real world-performance gap to be even larger if Mistral 7B is anything to go by. The fact that safety filters are opt-in is a huge benefit (even for safe applications).
Pretty sad for Google if their next big AI thing is already being beaten by a small company with a tiny fraction of their resources.
As for AI they have a lot of world-class talent but they somehow cannot transform that into a coherent product even though they did amazing stuff on the research side. It has been brought up many times on HN on how Google has a product-design problem and it seems to be true.
Google can't do anything about it either - consumers will prefer the chat approach and will go where it is offered. Google has to offer it, or people will go to ChatGpt and Bing, but at the same time they lose a lot of money each time people opt for the chat summary.
It is way expensive for Bing too, but they don't care because their market is growing and since the only thing people used to search Bing for was how to download Chrome and how to change the default browser, they aren't losing a lot this way.
I think search products will be OK - it will just be stranger than we expect.
Bing going legit and genuinely threatening google wasn’t on my bingo card for sure.
But those ads would be compelling and more valuable. There is no reason to believe the current inference LLM architecture is the optimal one (as shown by Mistral's MoE) - future inference will likely be cheaper than it is currently.
The sentiment that Google's goose is cooked I'm seeing in this thread seems like wish-fulfillment to me. Especially considering that the upstarts also don't have a clear road to profitability
That’s a large and difficult problem obviously and one Google is clearly well-positioned for if they can ever catch up on the model front.
Perhaps even sooner, given the pace tech is evolving nowadays comparing to when Google took over Yahoo's place.
Re: "distracting other companies", there's however this famous essay by Joel Spolsky (2002):
https://www.joelonsoftware.com/2002/01/06/fire-and-motion/
> Think of the history of data access strategies to come out of Microsoft. ODBC, RDO, DAO, ADO, OLEDB, now ADO.NET – All New! Are these technological imperatives? The result of an incompetent design group that needs to reinvent data access every goddamn year? (That’s probably it, actually.) But the end result is just cover fire. The competition has no choice but to spend all their time porting and keeping up, time that they can’t spend writing new features. Look closely at the software landscape. The companies that do well are the ones who rely least on big companies and don’t have to spend all their cycles catching up and reimplementing and fixing bugs that crop up only on Windows XP. The companies who stumble are the ones who spend too much time reading tea leaves to figure out the future direction of Microsoft. People get worried about .NET and decide to rewrite their whole architecture for .NET because they think they have to. Microsoft is shooting at you, and it’s just cover fire so that they can move forward and you can’t, because this is how the game is played, Bubby.
In fact Google did a lot against it. Google search was clean, fast, and with no ads, GMail had one of the best spam filters, and Google algorithms were highly resistant to SEO of the time. Time have passed, Google is now part of the problem, but it is not alone there, and for some time, Google really did good.
It's not really "available" though, is it? I won't buy any PR benchmarks until the models are publicly available, there's too much variation based on how the models need to toned back for safety reasons before being released to the public