Comment by bob1029 | Hacker News Reader

Comment by bob1029 | Hacker News Reader

Parent Full thread

I have a tiny suspicion that a new technique is going to emerge soon that will cause some frustration for Nvidia's investors. For instance, observing how little quantization error seems to matter makes me believe there are probably some other massive wins lurking throughout. Before I'd invest, I'd consider the "doomsday" scenario wherein someone develops a technique for running an OpenAI, GPT-4-scale model on a MacBook or iPhone.

I know it probably seems impossible to many on HN that there could be another bucket of 3-4 orders of magnitude sitting on the table, but progress over the last ~6 months seems to provide a compelling argument against that point of view.

There are also radically-different architectures that have seen zero serious effort put towards them. Mostly things that are CPU-bound. In my view, GPU is starting to cause more harm than good with regard to development of new concepts for problem solving. There are neural network architectures that simply don't work well across the PCIe bus. Eventually, someone is going to start playing around with these ideas as GPU scarcity rages onwards.

> I'd consider the "doomsday" scenario wherein someone develops a technique for running an OpenAI, GPT-4-scale model on a MacBook or iPhone.

That's not a doomsday scenario for Nvidia. There is essentially infinite demand for better AI, only limited by what can be provided at acceptable cost. If you can run GPT-4 on a macbook, you can do even better with a more massive model. If you can run a good image model on a macbook, then the next frontier is running a good video model, etc.

The real doomsday scenario for Nvidia is that there seems to very little differentiating their hardware. Their lead in the space exists because they developed good software support early, which lead to everyone standardizing on them. But AI is not like graphics where the problem domain is complex and the APIs are very ill-defined, and you can do all these tricks to make it faster and better. Instead, AI is almost entirely doing just a handful of very simple operations. Other vendors should be capable of providing good software support eventually, and at that point, what is Nvidia's moat? What justifies their margin?

I think Windows is a pretty good example of why just a set of software libraries can be a pretty formidable competitive edge.

The risk point is always "good enough."

Yes, you can run a better model if you have more hardware, but does a better model translate to a more compelling user experience?

Will people be willing to pay for slightly better AI images of giant cats using the Empire State Building as a scratching post if good-enough cats can be rendered locally.

> There is essentially infinite demand for better AI,

New paradigm stuff. Infinite demand. It will be able to create a perpetual motion machine easily.

Perpetual motion machine? Why bother with that when we can scale our EBITDA to infinity too!!!

E(BITDA) = MC^2

And 640k memory is enough for anyone!

Yeah but the challenge lately has been that efficiency gains have also made the capabilities of large compute clusters even more powerful. Transformers were introduced as an efficient alternative to RNNs and conv nets, then it turned out that however much they supercharged a single GPU, they were dramatically more powerful deployed at datacenter scale.

I thought the whole deal with machine learning was that inference is easy, but training is hard.

So creating the models from the data is what takes the billions of transistors on many cards.

running the models doesn't need so much hardware.

also... with respect to powerful processing - I haven't seen graphics cards tapering off. There's a ravenous demand for better graphics hardware each year. For every technique that is commoditized, the next year there's a new way of doing things that is better. I remember things like lighting, or realistic hair, or physics or whatever making a new graphics card better. Why wouldn't AI stuff be any different?

  inference is easy, but training is hard

That's often true for classification models (like image recognition), but it the generation models (SD, Llama and GPT-4) everyone is excited about use a lot of compute for inference.

It may be "a lot" in relative terms, but still a few order of magnitude lower than training.

1. There is not a 3-4 order of magnitude improvement on the table. 1? Maybe.

2. If AI gets more capable/efficient, the demand will increase not decrease.

Did weapons manufactures go out of business when machine guns were invented?

> Did weapons manufactures go out of business when machine guns were invented?

Some certainly did.

The obvious counter example is that our brains operate many orders of magnitudes more efficient and capable than what currently exists.

That alone proves your first point wrong.

Reply on news.ycombinator.com