Hacker News Reader

Comment by shrimpx | Hacker News Reader

Near-100% certainty that Altman and Brockman cofound a new AI company in the coming days. The question is will they be able to recruit a team that can actually build competitive models? Ilya Sutksevers don't grow on trees. Maybe they'll just get a team good enough to specialize Llama2, since Altman/Brockman seem to think what's lacking in this space is glitzy products, app stores, b2b integrations, etc. Maybe OpenAI starts to open source everything and Altman/Brockman can have their cake and eat it, too.

View on HN

You're massively overestimating the skills needed to build "competitive models". Researchers like Ilya were essential several years ago when OpenAI got started, because at that time nobody even knew what to build or how to go about it. It was all research, and OpenAI was solely focused on RL. But these days, LLM code is 95%+ data and infrastructure engineering and very little novel research. Even things like LLM speedups via new attention mechanisms are more engineering than they are research.

Above all else, what you need to build competitive models is a huge amount of money and access to compute. Money alone doesn't solve that either due to the global hardware shortage. You are competing for a globally limited pool of hardware with all other companies.

Google and Meta have more money and more compute than OpenAI. They have more data, better infrastructure, and more AI researchers. And they have been trying hard to catch up - so far unsuccessfully.

But they have the key problem of all established companies: they built the infrastructure for what is now the wrong thing.

This means to do the right thing at Google would require fighting battles on all sorts of fronts as established mini empires within attempt to latch on to this new perceived opportunity to expand.

to do the right thing at Google would require fighting battles

Determining what the right thing is the hard part. Unfortunately that’s also the case right now at OpenAI.

Google and Meta have less compute infrastructure purpose-built for training and testing LLMs than OpenAI does. It's something that OpenAI has been investing in for several years, while Google/Meta/etc have just started playing catch up very recently.

Researchers like Ilya were essential several years ago when OpenAI got started, because at that time nobody even knew what to build or how to go about it.

Nah Sam will just use GPT4 to code his vision. That's how it works now, we've been told.

It will be very ironic when their new startup gets dragged down by regulations due to them not having an AI license that sama pushed heavily in congress.

Nor do Andrej Karpathys (grow on trees).

OAI still have people who are true leaders in this deeply technical space - I don't see research wins going to a competitor unless they cultivate similar talent at the top.

It's also possible that Altman was privy to a breakthrough which makes it possible for him to execute from scratch on the AGI goal without a strong research org.

It's all speculation :)

> Maybe OpenAI starts to open source everything

I really doubt it

Sure. They just call Sundar.

Yo, bro what up. Want to grab lunch?

Microsoft would be horrified.

Would they tho? Altman and Brockman are not AI people, they’re business guys.

Brockman is also a software engineer. My understanding is that he spends most of his time on software problems.

gdb is easily one of the most intelligent engineers I've ever had the pleasure of interacting with.

Reply on news.ycombinator.com