Hacker News Reader

Comment by rsynnott | Hacker News Reader

Okay, I mean I feel like they’re not that mysterious. Like, there are probably only five or six candidates.

Companies like Coreweave who lease accelerators may make this analysis less straight forward than it appears.

Even less straightforward since Nvidia funds Coreweave.

> Although the names of the mystery AI whales are not known, they are likely to include Amazon, Meta, Microsoft, Alphabet, OpenAI, or Tesla.

Why mention Microsoft twice?

The departments aren’t talking, so they accidentally made two orders.

I’d note Microsoft needs OpenAI a hell of a lot more than OpenAI needs Microsoft. I’d actually pivot that to be why mention OpenAI twice.

OpenAI doesn’t operate without the enormous amounts of funding MS gives it.

I think a lot of institutions and people would love the chance to give them money.

But how many of them have hot data centers to offer? Google is a direct competitor, so Oracle or Amazon are kinda the only other two big options to offer them what MS is right now.

If MS drops OpenAI, it's not like they can just seamlessly pivot to running their own data centers with no downtime, even with pretty high investment.

A relationship that’s mutually beneficial needn’t be symmetric. Microsoft’s relationship is fairly commoditized - money and GPUs. OpenAI controls the IP that matters.

I’d note that the supplier of GPUs is Nvidia, who also offers cloud GPU services and doesn’t have a stake in the GCP, Azure, AWS behemoth battle. I’d actually see that as a more natural less middle man relationship.

The real value azure brings is enterprise compliance chops. However IMO aws bedrock seems to be a more successful enterprise integration point. But they’re all commodity products and don’t provide the value OpenAI provides to the relationships.

How so? As far as I can tell, Microsoft has a large equity interest in OpenAI, and OpenAI has a lot of cloud credits usable on Microsoft’s cloud. I don’t think those credits are transferable to other providers.

The value in the proposition is OpenAI IP. Money and data centers are commodities easily replaced, especially when you hold the IP everyone wants a piece of.

The arrangement is mutually beneficial, but the owner of the IP holds the cards.

And regardless of who the 4 are, the two others in that list of six candidates are most likely not too far behind.

The most interesting thing to discover would be if one of them is that far behind, because they're succeeding on their own/someone not-Nvidia's silicon.

The public in-house projects that I'm aware of (but as far as I know haven't fully replaced demand for Nvidia GPUs) include:

- Google's "TPU" (in production, publicly rentable)

- Amazon/AWS's "Trainium" (in production, publicly rentable)

- Meta's "MTIA" (in production)

- Microsoft's "Maia 100" (I'm unclear on their status)

- Tesla's "D1" (I'm unclear on their status)

xAI 100K H100 cluster would be ~$2B

Is this trailing year NVIDIA sales, or order book ?

The numbers in the article? They're neither. They're the revenue for just Q2, not the full trailing year.

Are these distinct architectures, or is it an ARM situation where nearly everyone is gluing the same IP cores together in slightly different configurations?

Definitely not an ARM situation.

I'm not sure about all of them but Google's TPU is custom to them and not shared architecture.

They are distinct architectures, but mostly do the same thing. Pretty much all of them have a few small control cores that run matrix multiply and vector reduction units. The instruction set on all of them is different, but the broad strokes of the architecture are the same.

MTIA will be for inference initially. Another to add to the list is wafer maker Cerebras

https://www.forbes.com/sites/craigsmith/2024/08/27/cerebras-...

It's not just the hardware, it's the software stack too and my understanding is that they aren't very good. Even TPUs aren't great if you aren't either (1) doing something extremely standard and a little bit old (e.g. not forefront of research and the stack has already been optimized for your model) or (2) in Google with access to the people who build the stack.

Maybe it is working for Meta or Tesla where things can be vertically integrated, but for the public clouds, they have to buy NVIDIA for their customers.

Does TSM have the capacity to scale up anything new right now?

Let's name them, and why?

Anyone building the largest of LLMs including Alphabet, Meta, OpenAI, Anthropic

I think you should probably add all large cloud providers. Amazon and Microsoft should be in the list.

And we know OpenAI uses Microsoft for GPUs. My guess is Anthropic is similarly not owning their own data centers; didn’t they get a bunch of money from google? It’s probably being spent there.

They use AWS.

No they don’t.

Care to substantiate?

According to Wikipedia:

In September 2023, Amazon announced an investment of up to $4 billion, followed by a $2 billion commitment from Google in the following month.

They probably use both. Almost certainly most of this is credits in their cloud platforms.

I doubt Anthropic builds their own GPU datacenter.

They might buy some, but I think that Google, Meta, Microsoft and Amazon and will be the ones buying in large batches to enable companies like Anthropic (and themselves) to scale up to world wide inferencing demands, as well as generally offering the most efficient GPUs to their customers.

Think they're renting GPUs from the cloud providers?

Very plausible. I'm not sure at which point it makes economic sense to buy the GPUs and build out the infrastructure to continually be training something like Claude.

> Think they're renting GPUs from the cloud providers

It's a major reason why they raised with Amazon [0]

There are actually a LOT of other large companies that participated in Anthropic's round but haven't announced it publicly.

[0] - https://www.aboutamazon.com/news/company-news/amazon-anthrop...

They train on GCP now.

They also raised with Google, interestingly.

https://www.wsj.com/tech/ai/google-commits-2-billion-in-fund...

Yep! You're right! I think they were initially using AWS though.

Oh, I would not forget governments. The NSA basically paid for Kepler with a single purchase...

I believe the GP refers to Google, Facebook, Amazon and Microsoft.

And while yeah, that's probably them, I'd put a non-trivial chance of some government intelligence organization to make it into the top 3.

I’m guessing a massive amount is for inference for whatsapp, and the original goal was making for relevant instagram - and of course the massive Llama model training - my guess is Facebook is a relatively small component of Meta’s overall use of GPUs. Feed recommendations? (unless you were using facebook as a holder for Meta?)

It’s absolutely not WhatsApp. It’s their recommendation engines. They’ve publicly stated they’re buying enough GPUs to have the spare capacity to train another “reels” sized product for when the opportunity emerges.

(They absolutely use it as a holder for Meta)

ANd possibly some less-known companies, as fronts for the Chinese.

> less-known companies, as fronts for the Chinese.

Not at that size. That is VERY on the nose sanctions evasion.

Such sanctions evasions tend to use multiple smaller parties doing purchases and then reselling.

> I'd put a non-trivial chance of some government intelligence organization to make it into the top 3

Most likely DoE. TLAs purchase indirectly (or use other federal agencies in the DoD as a front)

Reply on news.ycombinator.com