Trying to transition a research org into a product org is going to be needlessly painful, especially since the research org needs to be firing on all cylinders in this hyper-competitive space.
Inventing things is no guarantee of success.
In spite of Xerox labs having pioneered the windowed OS GUI with mouse and Kodak having built the first portable mass-market digital camera.
They wouldn't have needed to enshittify or degrade their products to be successful with that, they just failed for other reasons.
Look what happened when Google tried to throw an LLM in to search. Absolute shitshow. That’s not ready to become any kind of product!
If they kill R&D now to focus on productizing something that is half baked, they will fail to develop those new inventions which might get us to AGI. When I worked at Google X Robotics I was hired on to the remnants of the last research team, which was dissolved six months after I started (I was moved to hardware test engineer). Our subteam really wanted to research multi-finger grippers but we got overruled, so the robot had to do everything with a two finger pinch gripper. Which is fine for research but absolutely unsuitable for real world tasks. It couldn’t even operate a spray bottle without special attachments and they thought it was going to clean people’s homes!
[1] I am sharing this one a lot lately but I’m very moved by Yann LeCun’s arguments about the limits of autoregressive approaches here. As a robotics engineer I have been dismayed at all the attention LLMs are getting despite serious limitations that make them generally unsuitable to solve some of the most important problems in robotics. https://youtu.be/1lHFUR-yD6I
tbf, Ilya was on a VC podcast not at a tech conf.
> Well he’s not omniscient.
Neither is Yann (who has since proposed a different architecture / vision which is yet to take off), but my comment was meant to highlight a recent claim from another accomplished researcher in the field.
I meant to counter-balance OP's point in that there are other equally accomplished individuals who aren't swayed by Yann's (and others accelerationists like Andrew Ng) arguments or claims.
This is exactly it. With the limitations ChatGPT is encountering around safety and hallucination, Google probably should've just said "we're working on something awesome - hold on" and kept plugging away before releasing, instead of ex-Product CEO making them release something now, even if half of the demo video is fake.
"Companies that mentioned AI in earnings saw their stocks rise 4.6% on average, a study from Wall Street Zen found."
https://markets.businessinsider.com/news/stocks/ai-stock-mar...
Maybe I'm too tired, but I don't understand this sentence.
That said, I love your farm robots.
Would seem far more sensible to allow Deepmind to continue to release hit after hit in the ML research world, and simply embed "fly on the wall" PM's into their org that can independently productionize any golden nuggets they happen to create.
Bad idea. People good or lucky enough to land in R&D like doing R&D. Force them to be product people, I expect most of them will leave.
--
[0] - Like Muffin from Bluey, https://youtu.be/hZVlBQXVtZA?t=8.
The chat service bit is them being chill, but the real spark was the model. I say they were lucky, because AFAIK back then no one expected LLMs to show so many and so advanced general capabilities. This took everyone by surprise, and since people could already play with it, ChatGPT took off on its own - it had so much real, transformative value, that it spread out with zero marketing. That's a rare, bona fide case of "word of mouth", it was just that useful. But that wasn't a strategy, that was luck.
To their credit though, OpenAI turned this early win into an opportunity and is excellent at exploiting it. Being small helps.
IMHO, in retrospect, the failure gradient of early LLMs is underappreciated in driving adoption.
Windows 95 failure: blue screen with inscrutable error code. Everyone noticed that.
LLM failure: run-around non-answer (user shrugs and tries again) or confident and plausible incorrect answer (user doesn't recognize this without research).
Essentially, the ways in which LLMs didn't work were the most hidden and hardest to discover failure mode.
Which was perfectly tuned for the "I'm going to try this thing for 5 minutes and be amazed" first impression.
Which allowed subsequent generations to backfill the capability gaps.
Tl;dr - We shouldn't underappreciate quiet-failing as a product adoption driver.
I'm certain it drives a lot of early user retention in the short term, but I feel strongly that this is ultimately a very myopic view which will prove catastrophic in the long term in much the same way that swallowing exceptions at runtime builds compounding technical debt you'll have to reckon with sooner or later
more broadly, there is just so much handwaving away all the black box parts of deep neural networks that are completely opaque and there seems to be very little interest in building the tooling to properly visualize, explore, and DEBUG latent space; until those priorities change this whole thing is a huge time bomb.
imagine if instead of coming with full memory dumps and diagnostic codes, BSODs just said "sorry, your computer had an oopsie!", and not a single engineer at Microsoft had a complete understanding of why the BSOD happened in the first place; sometimes it just does that! whoops!
So, MacOS? ;)
In all seriousness, I wasn't opining on the usefulness of opaque/hidden errors, but rather the effectiveness of them.
In an alternate reality where the first LLMs instead spit back an error reference instead of English, I don't think we would have seen nearly as rapid mass market adoption.
And, not to put too fine a point on it, early conversational LLMs and image diffusion models were literally trained so their junk output is as plausible as possible.
I also think people and society also give themselves way too much credit for their successes. There's plenty of smart hardworking people out there who continue to contribute but never stumble upon a unicorn. To a large extent its luck, a much larger contributor than people realize. All you can do is to play the game, consistently contribute and work hard on R&D and products and you improve your odds of stumbling upon success. But it's never guaranteed.
You'd think they'd have plenty of spare people with product launch experience.
Who do you think has that expertise? The people working on the model or the people studying users?
Without engineering, you don't have the capability.
Without product, you don't build something users are actually interested in.
I've seen too many engineering teams try to productize what they want, not what people not-them want, and then be flummoxed by lack of adoption.
Nothing sucks more than burning the midnight oil to nail a target... that ended up being 2m to the right of the actual target.
Must transfer value, and the guy in charge of the company is not good at allocating the company's resources to do that with an eye on long-term results.
Because the current way they were working squandered over a decade lead in the space. Deep Dream was 2015... Google Magenta was 2017...
.