Hacker News Reader

Comment by lordofgibbons | Hacker News Reader

Does this mean Google is giving up on TPUs?

TPUs were supposed to be their unfair advantage in the cloud ML/DL space. But from what I've experienced, and have heard from other engineers, there's always some subtle incompatibility with TPUs that requires modifying the training/eval scripts. I wonder why they didn't try to polish the rough edges with Pytorch, et al.

If they're admitting TPUs aren't their competitive advantage, then why not sell it to other hosting providers, or hell, even directly to ML scientists and enthusiasts? They'll finally get economies of scale, and take business (and mind share) away from NVidia's monopoly.

View on HN

There's a few comments to this effect in the thread, and I don't entirely understand where they're coming from. There's nothing in the article suggesting they've changed their strategy with TPUs in any way. The word TPU isn't even mentioned here. There's no suggestion they're actually using this internally either. There's no benchmarks showing that it's more cost-effective or scales better.

And isn't your second paragraph the obvious reason for why this product (A3) exists? It's something they expect to sell to cloud customers who have an existing GPU-based workflow, and just want to run it as-is as fast/cheap/scalable as possible, without worrying about compatibility, and making sure they can always move the workload to some other cloud provider or on-prem if needed.

It's like suggesting Sony releasing some of their games on the PC means they're deprecating Playstation.

(Maybe there would be more details in the IO talk. Does anyone know which one this announcement is from?)

The word TPU isn't even mentioned here.

A sentence that reads "I am going to eat nothing but vegetables from now on" doesn't mention meat, but you can infer that I won't eat meat again from the sentence.

A sentence that says Google are going all in on nVidea GPUs for AI doesn't need to mention TPUs to convey information about their future either.

> A sentence that says Google are going all in on nVidea GPUs for AI doesn't need to mention TPUs to convey information about their future either

Where are you reading that Google is going “all in” on nVidia GPUs? I don’t see that in the linked article at all.

These are clearly targeted at their cloud customers who have workloads tailored to GPUs. They’re supplying demand, as cloud providers do.

Companies can do more than thing at a time.

> A sentence that reads "I am going to eat nothing but vegetables from now on" doesn't mention meat, but you can infer that I won't eat meat again from the sentence.

TBF, there is no mention of anything remotely similar to "I am going to eat nothing but vegetables from now on".

Sure. That's why I mentioned multiple ways in which the article could have been relevant to TPUs, which you chose not to quote. But it didn't have any of those either. The sentence you're offering up as a demonstration is just something you made up that does not appear in the article.

If anything, this just reinforces the point I was making. There is nothing at all in the article supporting this narrative. So, where is this coming from? Why are you so intent on this idea that you're reduced to fabricating support for it?

At the very least we know that there's a team in Google that chose to build an AI supercomputer with non-Google hardware. They didn't, or wouldn't, work with the TPU team to do it, or they did and the TPU team couldn't get it to work. Or they could but something still made nVidia hardware more compelling. Every level of management involved were persuaded that this was the case even knowing it would send a message to people outside of Google about TPUs.

Andnfrom.all that we're meant to say it infers nothing about TPUs?

The comment quoted you, then made an analogy. Where is the fabrication?

That's not true.

Google is huge.

Just a few H100 doesn't represent anything huge in Google scale.

I also tried to find your analogy in that article and google announcement and it's not there.

Companies never announce change of direction like you seem to think. There is no positive outcome in doing so. Instead they announce the new thing and promise to continue to support the old thing and then just don’t.

>There's nothing in the article suggesting they've changed their strategy with TPUs in any way

Google owns and designs their own TPUs. They offer these TPUs in the cloud. I've seen many comments in here about how next-level TPUs are (despite zero evidence indicating that). Google even disclaims their TPU by saying that you shouldn't compare it with the H100 given node levels et al.

Their premiere offering is an nvidia H100 offering.

Yes, of course this is a pretty telling indication. If Google was all in on TPUs they'd be building mega TPU systems and pushing those. Instead they're pushing nvidia AI offerings.

>Does this mean Google is giving up on TPUs?

https://arxiv.org/abs/2304.01433 from April 4 of this year.

> I wonder why they didn't try to polish the rough edges with Pytorch, et al.

It's always funny to me when people have this blindspot - because TPUs aren't for you, they're for the ads org. Neither are PyTorch nor TF for that matter. They're more than happy to get external bug fixers but trust me those individual teams dgaf about external customers. They're not in the least bit community driven projects.

This is for GCP. Google themselves probably still trains on custom hardware but they don't offer their latest and greatest hardware on GCP.

Offering more options to customers is always better especially when Nvidia has great market share in this area. This is probably the reason why Microsoft is trying to help AMD catch up so their is more competition. AI GPU prices are insane compared to standard GPU because of the lack of competition.

I haven't heard anything about microsoft helping AMD, it sounds interesting. Do you mind linking an article?

It doesn't seems to be true.

There were articles that Microsoft was helping AMD, but the denied it.

https://arstechnica.com/gadgets/2023/05/microsoft-and-amd-ar...

This is thinking about the issue all wrong. Google's internal infrastructure is terrifyingly large. They won't "get scale" by selling TPUs. That would expand the scale of TPUs only slightly.

There’s demand in GCP for H100s so they offer them. I doubt Google itself is a big user.

If they are selling gpu compute, nobody wants to use a Google TPU, they want cuda

And they want to support people migrating from other cloud providers where they are already using nvidia/Cuda. Though it also helps support the opposite migration, they are the smaller cloud player trying to get customers, not the big one trying to constrain them as much yet.

> subtle incompatibility with TPUs that requires modifying the training/eval scripts

Do you have any more details or links to articles expanding on this?

Reply on news.ycombinator.com