Hacker News Reader

Grok4 was trained on 100k or 200k GPUs (as far as I understand)

Grok5 might need 1MM or 2MM.

So the question is what about metas / zucks plans? How many GPUs will Manhattan get? Looks like, that to get the next unlock you need crazy amounts of compute.

View on HN

Meta had the equivalent of about 600K H100 cards a year ago, but they were geographically distributed and used mostly for inference.

These giant data centres will allow these companies to put about a million in one location and possibly into a single giant training cluster.

Reply on news.ycombinator.com