Hacker News Reader

Comment by fdgsdfogijq | Hacker News Reader

Big Tech hasnt dominated anything. They got trounced by a 400 person startup. Alexa, Google Search, FB research all spend untold money on ML systems. Neither is releasing game changing products anywhere close to OpenAI.

View on HN

The Transformer architecture that powers GPT models is based on a now-famous Google Brain paper, Attention is All You Need. You're right that OpenAI has productized this research much faster than Big Tech, but the research itself came from established R&D in Academia and Big Tech.

Yes, but only one of the authors is still at google.

So if you work on a kickass product at FlorpCorp and then leave, FlorpCorp didn’t develop that product?

Yes, the people are where the value is, not a dumb fuck like Sundar.

Not if FlorpCorp didn't release it.

Well, Google had to figure out how to cancel itself somehow. That's the only Google project, aside from search, that has had staying power.

And Stable Diffusion is based on Google's Imagen paper.

The diffusion papers most analogous to the transformer paper are from Berkeley, Stanford, and Freiburg. Imagen wasn’t foundational in the way you’re suggesting.

OpenAI was backed by probably the biggest tech company around, Microsoft. Without the computing afforded by Azure, they would have problems training and running the LLM.

It is true that big companies have brands and existing revenue stream to protect (Google), so maybe only a little company with nothing to lose could do what OpenAI did.

And maybe MS found the perfect arrangement giving them an ability to adjust how closely to associate themself with the tech according to need.

But is this all necessarily a good thing? OpenAI gave MS access to GPT and MS clearly wants to use GPT to destroy Google, so Google now has respond aggressively, and now we have exactly the AI arms race that many predicted would lead to the worst outcomes.

In that case they might have picked the worst time to push through layoffs.

I disagree. All of the big players have been involved in some way, creating their own systems (which are quite competitive) as well as laying much of the foundation for existing models with research they published.

I wouldn't say big tech was "trounced" by OpenAI. Advances in LLMs are driven by the research community. OpenAI is a small, well-resourced organization with the sole purpose of quickly deploying and hyping state of the art AI models. Big tech naturally has a slower product development cycle - and will make full use of LLMs in due time.

> I wouldn't say big tech was "trounced" by OpenAI.

I mean not a single “big tech” company is offering any LLM products not powered by Open AI. It’s been almost 3 years since GPT-3 was released. I’m not sure they can catch up at this point. And I think the GPs point was a good one: big tech threw billions and billions at personal assistants. Nothing innovative was delivered.

Aren’t google search and google translate powered by transformer models?

Probably, but using a transformer isn’t Open AI’s advantage. A widely available language model with “emergent abilities” is Open AI’s advantage. The fact that it’s a transformer is more of an implementation detail.

Yes and tech was so path breaking and innovative that the company sold its 49% share for 10 billion dollar /s. Sam Altman is extremely experienced in the VC world and could have easily raised billions for potential value. He wouldn't have sold half of his company within 6 months of releasing chatGPT if there was any thing remotely defensible in the product

I think you forget what money and connections bring.

Anything that can be provided by a micro or small sized company can be provided by billion dollar corporations too.

Maybe size brings a drawback, in the same way a mega ship finds it difficult to turn around without a day's notice — but these companies can and will.

Microsoft once betted all chips on the internet being fad; look at them now.

I cant even discuss this without going powerlevel 9000+

Reply on news.ycombinator.com