Hacker News Reader

Comment by llm_nerd | Hacker News Reader

The overwhelming majority of AI compute is by either the few bigs in their own products, or by third parties that rent out access to compute resources from those same bigs. Extremely few AI companies are buying their own GPU/TPU buildouts.

Google says Ironwood will be available in the Google Cloud late this year, so it's relevant to just about anyone that rents AI compute, which is just about everyone in tech. Even if you have zero interest in this product, it will likely lead to downward pressure on pricing, mostly courtesy of the large memory allocations.

View on HN

It just seems like John Deere putting out a press-release about about a new sparkplug that is only useful to John Deere and can maybe be used on rented John Deere harvesters when sharecropping on John Deere-owned fields using John Deere GMO crops. I just don't see what's appealing about any of it. Not only is it a walled garden, you can't even own anything and are completely dependent on the whims of John Deere to not bulldoze the entire field.

It just seems like if you build on Tensor then sure, you can go home, but Google will keep your ball.

The reality is that for large scale AI deployment there's only one criterion that matters: what is the total cost of ownership? If TPUs are 1/30th the total perf but 1/50th the total price, then they will be bought by customers. Basically that simple.

Most places using AI hardware don't actually want to expend massive amounts of capital to procure it and then shove it into racks somewhere and then manage it over its total lifetime. Hyperscalers like Google are also far, far ahead in things like DC energy efficiency, and at really large scale those energy costs are huge and have to be factored into the TCO. The long dominant cost of this stuff is all operational expenditures. Anyone running a physical AI cluster is going to have to consider this.

The walled garden stuff doesn't matter, because places demanding large-scale AI deployments (and actually willing to spend money on it) do not really have the same priorities as HN homelabbers who want to install inefficient 5090s so they can run Ollama.

At large scales why shouldn't it matter whether you're beholden to Google's cloud only vs having options to use AWS or Oracle or Azure etc. There's maybe an argument to be made about price and efficiency of Google's data centers, but Google's cloud is far from notably cheaper than alternatives (to put it mildly) so that's a moot point if there's any efficiencies to be had Google's pocketing it themselves. I just don't see why anyone should care about this chip except Google themselves. It would be a different story if we were talking about a chip that had the option of being available in non-Google data centers.

Reply on news.ycombinator.com