Hacker News Reader

Comment by DrewADesign | Hacker News Reader

And this is exactly what the LLM provider industry is fighting tooth-and-nail. It’s not only because it doesn’t directly contribute to their bottom line, it also directly opposes the idea that LLMs are going to replace entire workers rather than enhance the abilities of individual workers. What we’re headed towards would have been a killer product and probably still shifted a bunch of capital to the bazillionaires had these companies set more realistic goals rather than banking that they’d be the ones that won the war that “changed everythingTM”.

View on HN

As long as Apple and Google put reasonable AI capabilities on device, then software engineers will use those capabilities when it makes sense (the article gives lots of good examples of capabilities that make sense to run locally). As the author notes, it's cheaper and more reliable to run these things locally.

That also doesn't preclude LLM services from being massively successful, they'll just have to justify the pricing and complexity that comes with their adoption, just like any other product.

> That also doesn't preclude LLM services from being massively successful, they'll just have to justify the pricing and complexity that comes with their adoption, just like any other product.

What is completely different from every other product is how much they’re spending, and how much they’re obligating themselves to spend going forward. I think there’s a very good chance that the existing providers could be miles underwater coming out of this. Even if the business is not the everything to everybody that they’re banking on it being, they still owe all of that money back to the people they borrowed it from, and they will be a lot less likely to float them cash to get them back to a normal operating mode if they burned the last ocean of cash promising the universe and winding up with “oh yeah, that’s pretty useful sometimes.”

Yeah that's a good callout for sure, the spending here is nuts so agree that it's not "just another business that has to price itself right to be competitive".

I guess if the time horizons is long, like 20 years, then maybe the spending, as it begins to amortize, gets more in line?

I was thinking that a comparison could be to cloud providers, each of which had to spend a lot of money to build out datacenter before making money. Difference there is AWS proved the product first, so when Microsoft and Google came along, they knew it would work and be profitable. With AI, nobody has proven it will work and be profitable, they're all competing for that at the same time which is a potentially dangerous mix for the reasons you cited.

The only way that this even vaguely works, best I can tell, would be on that decade-or-two timeline, but therein lies the problem: all this money getting pumped into data centers right now is going to produce data centers that are running old, inefficient, slow GPUs by 5-years-from-now standards. And GPUs are by far the most expensive part of these data centers… having the buildings is barely an asset. We’re investing all the money in right now’s technology in one of the fastest moving hardware segments and for some inexplicable reason, think that will lead to a sustainable advantage. What’s to stop someone 5 years from now, waiting for the dust to settle, then spending way less money for more compute and just mopping the floor with everybody in this sector… and that’s (unreasonably, IMO) assuming that local applications won’t become good enough to take too large a bite from their business before that.

And look at the difference in spending between their building out general-purpose-computing cloud data centers that even then, had potential use cases if the business failed. What are they going to do… start a massive, extremely expensive pre-rendered online gaming service? Only render Disney movies?

I dunno. None of this makes sense to me.

These datacenters are already running old, inefficient, slow GPUs from five years ago in addition to newly released cards, because anything newer than that is extremely bottlenecked and they need all the compute they can get. Why should it be any different in five years' time? Even nVidia is rumored to be about to bring back the RTX 3060 which is an Ampere architecture card that got released around 2021. It's just fine.

If those data centers were good enough, they’d save themselves a few billion dollars and just do more of the same, wouldn’t they? Many current video games struggle on the 3060— it’s like 10 times slower for interference than a 4090 even. They’re reintroducing it because their upstream business of selling brand new insanely expensive GPUs required for every new data center is making it impossible for people to buy GPUs for their home computers. It says nothing about data-center-class GPUs except that every company currently has a burning desire to only have the latest and greatest GPUs.

The new GPUs are a lot better than the old ones, to be sure. They're also a whole lot harder to get ahold of in quantity. That's no different than the reason for officially reintroducing the 3060.

That doesn’t make this more sustainable or viable in the long term, which is the entire point

Yeah, conceptually this isn't all that different from new VM SKUs coming out in clouds. The costs and rate of change for AI hardware may be higher, and perhaps enough higher to mess up the math, but conceptually its a model that has been proven to work.

Unless some disrutive technology comes along in 5 years time. And many are working on exactly that.

> they'll just have to justify the pricing

like by selling it at a loss to build dependencies and then jacking the price up year after year by whatever amount is just below the cost of removing the dependency

In an ideal world they will. In reality most will use online AI, because it's path of least resistance and more familiar.

> ...it also directly opposes the idea that LLMs are going to replace entire workers rather than enhance the abilities of individual workers.

Which also, as I feel the need to remind everyone every time it comes up, has not yet once been actually shown to be a workable strategy. For any worker in any industry.

And to be clear, I'm talking about a worker, sitting in a chair, replaced with an agent, sitting in... a server, I guess, where nothing else about the org has to be changed. That's what's being advertised and sold, and it has never to my knowledge actually happened.

mainframe industry vs personal computers.

If their product is "access to a big model running on a really big computer" (if we can count 'multiple data-centers' as a single enormous distributed computer), then the product "small, accessible device that everyone has" risks killing their cash cow.

Ironically enough, the first company to really focus on "an LLM in every phone" will have a good shot at actually being the ones that "changed everythingTM", in the way Microsoft changed the world from IBM mainframes to PCs, or Apple made smartphones a thing.

As an aside, the mainframe industry was profitable for decades before PCs took over. It’s not like they spent a zillion dollars ramping up at the same time.

The mainframe industry IS profitable.

Reply on news.ycombinator.com