Google Calls in Help from Larry Page and Sergey Brin for A.I. Fight
nytimes.com
nytimes.com
Also, it doesn’t inspire confidence in Pichai that Sergei and Brin were consulted about something that should be on the forefront of Google’s strategy. Maybe that’s too harsh but I just find all this surprising.
[1] https://ai.googleblog.com/2023/01/google-research-2022-beyon...
There is no reason why he would have ever needed to "muddle through" on his own. The article states Google acquired DeepMind an AI research lab in 2014 as well as the fact that Google has its own Advanced Technology Review Council. This is in addition to the insane hiring spree that the company went on in recent years where presumably they did or should have been hiring even more AI competencies. Given these things, having to call in the founders who stepped away from the company 3 years ago kind of make this CEO look incompetent.
A CEO getting in position after a company is well-established is often merely the "toppest" manager.
Big companies try to incubate startups but just can't leave shit alone. They have to micromanage everything under the terms of the core business. Startups get lucky from trying hair-brained ideas and being scrappy--not being micromanaged.
I'm working in a startup right now, building a new product that complements the core business. The new product is highly suspect, and on paper seems like it could be a real mess (handling real peoples' money). I'm interested to see how it adapts or pivots after meeting the customer, and I'm not willing to write it off because I cannot predict what it will develop into even if the original idea fails. It could all be a waste of time and money. It could also reveal another market. The original idea doesn't have to work out for the venture to be successful.
Google already has very large LLMs online for search and other applications.
A similar take on similar spin: https://twitter.com/westurner/status/1614002846394892288
From April 2022, "Pathways Language Model (PaLM): Scaling to 540 Billion Parameters for Breakthrough Performance" https://ai.googleblog.com/2022/04/pathways-language-model-pa... :
> In recent years, large neural networks trained for language understanding and generation have achieved impressive results across a wide range of tasks. GPT-3 first showed that large language models (LLMs) can be used for few-shot learning and can achieve impressive results without large-scale task-specific data collection or model parameter updating. More recent LLMs, such as GLaM, LaMDA, Gopher, and Megatron-Turing NLG, achieved state-of-the-art few-shot results on many tasks by scaling model size, using sparsely activated modules, and training on larger datasets from more diverse sources. Yet much work remains in understanding the capabilities that emerge with few-shot learning as we push the limits of model scale.
> […] More recent LLMs, such as GLaM, LaMDA, Gopher, and Megatron-Turing NLG, achieved [SOTA] few-shot results on many tasks by scaling model size, using sparsely activated modules, and training on larger datasets from more diverse sources. Yet much work remains [for few-shot LLMs] /2
DeepMind Dreamerv3 is more advanced than all the LLMs.
"Beyond Tabula Rasa: Reincarnating Reinforcement Learning" https://ai.googleblog.com/2022/11/beyond-tabula-rasa-reincar... :
> Furthermore, the inefficiency of tabula rasa RL research can exclude many researchers from tackling computationally-demanding problems. For example, the quintessential benchmark of training a deep RL agent on 50+ Atari 2600 games in ALE for 200M frames (the standard protocol) requires 1,000+ GPU days. As deep RL moves towards more complex and challenging problems, the computational barrier to entry in RL research will likely become even higher.
> To address the inefficiencies of tabula rasa RL, we present “Reincarnating Reinforcement Learning: Reusing Prior Computation To Accelerate Progress” at NeurIPS 2022. Here, we propose an alternative approach to RL research, where prior computational work, such as learned models, policies, logged data, etc., is reused or transferred between design iterations of an RL agent or from one agent to another. While some sub-areas of RL leverage prior computation, most RL agents are still largely trained from scratch. Until now, there has been no broader effort to leverage prior computational work for the training workflow in RL research. We have also released our code and trained agents to enable researchers to build on this work.
Feed-Forward with Prompt Engineering is like RL; which prompt elements should remain given objective or subjective error?
Looks like new competition in the dumbing down department.
Only 5 pages of search results vs regurgitated internet text.
A race to the bot. 'um...
Updated Jan. 21, 2023, 1:53 a.m. ET Last month, Larry Page and Sergey Brin, Google’s founders, held several meetings with company executives. The topic: a rival’s new chatbot, a clever A.I. product that looked as if it could be the first notable threat in decades to Google’s $149 billion search business.
Mr. Page and Mr. Brin, who had not spent much time at Google since they left their daily roles with the company in 2019, reviewed Google’s artificial intelligence product strategy, according to two people with knowledge of the meetings who were not allowed to discuss them. They approved plans and pitched ideas to put more chatbot features into Google’s search engine. And they offered advice to company leaders, who have put A.I. front and center in their plans.
The re-engagement of Google’s founders, at the invitation of the company’s current chief executive, Sundar Pichai, emphasized the urgency felt among many Google executives about artificial intelligence and that chatbot....(which shall remain name less.)