xAI's Colossus: Most Powerful AI Cluster Online in 122 Days | Hacker News Reader

xAI's Colossus: Most Powerful AI Cluster Online in 122 Days

twitter.com

27 pointsgfortaine21 comments

Is there a reputable source for these claims? Given source has been proven to lie about scale, delivery, and functionality, especially for topics that might boost share prices. https://elonmusk.today/

“The utility company said that by August, xAI would have access to around 50 megawatts of power, or only enough for around 50,000 chips, and that an upcoming electric substation on the site would give him another 150 megawatts—enough to power 100,000 chips or more. But that wouldn’t happen until 2025, the utility company said” [1].

[1] https://www.theinformation.com/articles/why-musks-ai-rivals-...

This looks like quite a futile PR to downplay other players? MS already had more than 100k H100 before the beginning of this year and the same thing for Meta. Google has its own TPU cluster so it's not even possible to do apple-to-apple comparison.

Yeah the charts posted in comments on x didn’t look right to me either

Doesn't this literally not matter? We know that something Llama is being trained in about 3 months on Meta's current resources from their current datasets.

If they had less GPU power available, it would take longer but not dramatically so. Building a "more powerful AI cluster" doesn't give you a more powerful AI currently. In fact none of the Transformer architecture's seem to, and their instantiations seem to all be targeted towards "single H100 executions" for inference.

So this really does seem like nothing but hype: buying a lot of commercially available GPUs isn't accomplishing anything.

Right, but that means Meta can train 4 models a year. So, 4 opportunities for feedback. They however, need to release those to the public in order to get feedback on the limitations of the model, so that might limit it to training and releasing 2 models a year.

If you had more GPUs you could iterate faster...

Apparently running on many gas turbines (enough to power 50,000 homes), without any permits for all that pollution [1]. Also, most likely, hoovering up water from the often low Mississippi River

Just your modern day robber baron...

1. https://www.reuters.com/business/environment/musks-xai-opera...

Is this really the most powerful “AI cluster” or is this an exaggeration? I would expect that hyperscalers like Google or Amazon would actually have the largest amount of compute power to throw at AI. Or is that no longer true?

EDIT: I see a comment that has a graph of AI compute across different companies. I am not sure if this is trustworthy however, as the graph is itself just generated by an LLM (Claude):

https://x.com/AnthonyEveryWhr/status/1830680977103794177

I'd say this graph is absolute rubbish. The fact that the numbers are round numbers, decreasing in even steps, with no units, no comparison, incorrect scaling, and incorrect layout for a graph, suggests that this is little more than fanfiction.

The truth is that the hyperscalers are very closely guarding information around things like this. Who has the biggest "cluster" is impossible to answer. Also most hyperscalers will have many clusters, and what's more useful, the SUM or MAX of those clusters? At a small scale probably the MAX, but at a large scale the SUM because you can distribute across clusters without as much loss in overheads and because you have a range of jobs to run.

No the chart is wrong.

For these numbers to be correct Nvidia would have to be committing securities fraud on a scale never seen before.

The data in that graph is just complete misinformation.

https://engineering.fb.com/2024/03/12/data-center-engineerin...

This sounds like a notable achievement but they really ought to have chosen a different name.

https://en.wikipedia.org/wiki/Colossus_computer

Does xAI do anything interesting or are they just trying to catch up?

To be fair catching up is quite a feat in this game

That's undeniable, but it would be interesting to see something beyond "you can generate a picture" or "you can ask our model questions, though it may make things up".

Generating pictures isn’t even xAI’s model. It uses flux.

Grok-2 is rank 2 on LLM arena, it's basically as good as the best Gemini model. They already caught up. Only the latest ChatGPT model is a tiny bit better.

(x) Doubt

Colossus ... didn't I watch a movie about this?

Is this notable due to the lack of HN comments?

Reply on news.ycombinator.com