Hacker News Reader

Comment by mannerheim | Hacker News Reader

Depends on what models we're talking about. I've heard reproducing ChatGPT would require on the order of ten million dollars' of compute.

View on HN

$10 MM of compute doesn't seem all that out-of-reach for most "mid-size" companies, especially if the result is economical.

I don't think the result is economical purely because if it were, Google would be monetising their own models by now (of course, maybe they could monetise it if they were willing to go with a paid instead of ad model for search).

I think there are a lot of assumptions that Google hasn't already integrated this type of technology into their search engine. I suspect they have, but they've done so conservatively, to avoid changing how their "Golden Goose" lays eggs.

That's cheap for an AI model training at big tech. Your average ads model engineer at those companies likely uses more compute each quarter.

It mostly suggests that we're in another AI hype bubble. MS and other big tech companies can easily replicate OpenAI results and do same type of research.

Well, there's not only monetary cost on compute, you need humans interacting with it for RLHF, it takes time as well, you need engineers, and at the end of it, if you're just copying the leader, you're always going to be behind and not going to be coming up with the latest and greatest.

I'm just disappointed the compute cost alone seems difficult to surmount for open source/community projects.

The number of companies ready to spend $40M/year of compute per "average ads model engineer" is exactly 0.

I know of 1 at least...

'compute' costs are tricky to convert to dollar values - everyone does it at 'what would it cost to rent this from AWS', but the reality is, people like AWS are prepared to give low priority access to unsold compute for almost any project.

Reply on news.ycombinator.com