Comment by dagmx | Hacker News Reader

Comment by dagmx | Hacker News Reader

On the one hand, I totally understand why they’re in this position: there’s no competition and there’s only demand.

On the other hand: I feel like this is a bubble that’s going to pop. Everyone is promising AI futures and I just see a skirmish to bring a product to market that will leave a lot of folks crashing.

Either way, NVIDIA wins.

I am amazed how much the rest of the silicon industry has slept on this.

I know AMD have their competition, but their GPU software division keeps tripping over itself.

Intel could have done more but got complacent for a solid few years.

Qualcomm feels like a competitor in the making with Nuvia but is late to the dance.

Apple is honestly the biggest competitor in terms of hardware and software ecosystem , but refuses to get into the data center.

And of course I’m leaving out the Chinese companies like Huawei who would be a force to be reckoned with if it weren’t for sanctions. But China is also a hotspot for AI/ML.

NVIDIA was first to run with the wind. Jensen Huang made a big bet on AI, sent all the researchers his cards, made the world run on CUDA. By the time AMD/Intel woke up to the AI tsunami, the sea had already turned green.

> I know AMD have their competition, but their GPU software division keeps tripping over itself.

They are actively stepping on every rake there is. Eg they just stopped supporting the drop-in-cuda project everyone is waiting for, due to there being "no business-case for CUDA on AMD GPUs" [0].

[0] https://github.com/vosen/ZLUDA?tab=readme-ov-file#faq

Does HIPIFY not meet your standards?

https://github.com/ROCm/HIPIFY

And Google has TPUs that can compete with Nvidia, but they want to hoard it all and don't want to sell outside of GCP.

How well Nvidia can hold its current valuation depends on how useful the new AI/LLM ecosystem will become. Just chat apps and summarizing documents aren't that IMO.

If Google can sell TPUs, perhaps they can add a trillion or two to their market cap?

Google seems to be perennially disinterested in selling products that require large scale physical operations. I'm surprised they're still in the Pixel phone game.

I’d say more that Google are caught between wanting to be a service provider and a hardware company.

Pixel phones extend their services.

Selling TPU units reduces the differentiator for their services.

Why would Google sell TPUs when it can rent them out via GCP?

Because some people want to buy, not rent them, and you want those people's money....?

I don't think AI is a bubble but Nvidia's stock price might be. They aren't the only company designing GPUs.

IMHO, NVidia isn’t just competing on GPU. It’s competing on ecosystem and infrastructure so I think they’re quite untouchable for the foreseeable future.

I think AI will burst though. So few companies have a novel use, but all of them are promising a future they can’t yet reliably deliver. How much capital runway is there on this?

I’d bet (figuratively of course, as if I had any money) that AI will see a lot of casualties by end of year.

I'm bullish on AI because there are millions of people turning their attention to it and making discoveries. The democratization brought by consumer hardware being able to use these things will create a cambrian explosion similar to the open source movement for software.

It's hard to imagine the future but the imagination of the masses never ceases to amaze.

Perhaps I am more jaded because I am partially in this space, and I have a natural aversion to anything pitched as "democratization" because it's a salespitch word for "We want you to believe it will be mass market someday"

That's not to say democratizing abilities is not a worthy goal. I just don't think many of the companies rushing into this space really have a product vision that will end up being mass adopted.

I mean democratization in that people can buy a 4080 or 4090 and run and thinker with AI/LLMs themselves.

I think that’s too much of a minority to hold up a market.

The real key will be client side inference. And as more chips go the route of the Apple ones with onboard inference, I think it’ll be more appealing to the mass market.

Running LLMs locally on high end GPUs is going to always be niche.

> Apple is honestly the biggest competitor in terms of hardware and software ecosystem

Are they? It's not that obvious that they could necessarily scale up their low-power integrated GPUs (as fast as they are) to be competitive in the datacentre. AMD and Intel at least are trying and have/are developing some products.

And I'm not sure about their software? What do they really have that Intel/AMD don't?

In terms of hardware , they absolutely could scale up higher in the same way Nvidia does: more chips with interconnect on a blade.

Given their power efficiency, they could just go wider and come out on top.

And in terms of software, metal compute is the only real competitor to CUDA backends. Between their PyTorch backends, MPX, and large memory, a lot of ML engineers are going with a combo of Macs locally and NVIDIA in the cloud these days.

Intel and AMD don’t factor in at all comparatively for ML use because Intel doesn’t have a GPU backend for most things people want, and AMD has repeatedly made a mess of rocm

Reply on news.ycombinator.com