What the heck happened? Or was their reputation always just more hype than substance?
What the heck happened? Or was their reputation always just more hype than substance?
The TL;DR on his take is that there are organizational and cultural issues that prevent Google from focusing their research efforts in the way that is necessary for what he calls "big swings," like training GPT-3.
In regards to your second question, Google's reputation in ML is definitely not hype. Purely on the research side, Google has been behind some of the most important papers in modern ML, particularly around language model. The original Transformers paper, BERT, lots of work around neural machine translation, all of the work that DeepMind has done post-acquisition, and the list goes on. On the applied side, they also have some of the most successful/widely-adopted ML-powered products on the market (think RankBrain/anything involving a recommendation engine, Translate, Maps, a ton of functionality in Gmail, etc).
(The logic sort of makes sense. Glue sticks things together, and some glue is edible.)